Your LiveKit agent sends every turn to STT, addressed or not.

The audio pipeline

Five pipeline layers; only layer 5 answers who's being addressed.

A user turns to talk to someone nearby: VAD and STT keep running anyway. SAA gates before STT, deciding addressee every 250 ms.

How SAA joins a LiveKit room

A hidden SAA participant subscribes to your tracks and publishes decisions.

Calling start_attention_session adds that participant to your room. It can subscribe to tracks and publish data, never media, and emits a decision every 250 ms.

Integration: existing AgentSession

One callback expression to gate an AgentSession you already have running.

Add the gate without restructuring your entrypoint. Construct an AttentionEngine and wire three callbacks, shown below.

from saa_livekit import start_attention_session, AttentionEngine

async def entrypoint(ctx: JobContext):
    await ctx.connect()

    # Summon the hosted SAA participant into this room
    saa = await start_attention_session(
        room=ctx.room,
        participant_identity=ctx.room.local_participant.identity,
    )

    # Construct the engine that routes PredictionEvents to your callbacks
    engine = AttentionEngine(
        room=ctx.room,
        agent_identity=ctx.room.local_participant.identity,
    )

    # Gate: the entire addressee decision is one expression
    @engine.on_prediction
    async def _(p):
        voice.input.set_audio_enabled(p.aligned_class == 2)

    # Interrupt: stop TTS when a confident class-2 streak fires during playback
    @engine.on_interrupt
    async def _(ev):
        voice.interrupt()

    # Responding state: arm the interrupt detector during AI playback
    @session.on("agent_state_changed")
    def _(old_state, new_state):
        if new_state == "speaking":
            engine.responding_start()
        elif old_state == "speaking":
            engine.responding_stop()

    ctx.add_shutdown_callback(saa.stop)
    ctx.add_shutdown_callback(engine.stop)

Integration: greenfield agent

build_attention_entrypoint: a factory helper for teams starting from scratch.

build_attention_entrypoint returns a coroutine you can pass to your agent server. Give it your async on_turn handler; SAA handles the rest.

from livekit.agents import JobContext, AgentServer
from saa_livekit import build_attention_entrypoint, TurnReadyEvent

async def on_turn(ev: TurnReadyEvent, ctx: JobContext):
    # ev.audio_pcm16: int16 mono PCM at 16 kHz, ready for any LLM audio API
    if ev.context == "interjection_follow_up":
        # Turn was captured after a successful interjection;
        # route to a different LLM prompt (e.g. "briefly offer to help")
      ...
        return
    # Forward ev.audio_pcm16 to your STT pipeline or realtime model

# Plug the returned coroutine into your agent server
entrypoint = build_attention_entrypoint(
    on_turn=on_turn,
    # on_interrupt=handle_interrupt,   # optional
    # on_interjection=handle_interject, # optional
)

server = AgentServer()
server.rtc_session(entrypoint)
# or: WorkerOptions(entrypoint_fnc=entrypoint)

Speech-to-speech agents

Disable LiveKit VAD; let SAA own turn boundaries for realtime models.

Set turn_detection=None so the model hears only SAA-injected turns. attention_config can disable frame capture for audio-only pipelines.

from livekit.plugins import openai
from saa_livekit import build_attention_entrypoint, inject_realtime_turn, TurnReadyEvent

async def on_turn(ev: TurnReadyEvent, ctx):
    # inject_realtime_turn chunks the PCM at 100 ms intervals,
    # calls rt.push_audio for each chunk, rt.commit_audio,
    # then calls session.generate_reply()
    rt = ctx.proc.userdata["rt_session"]
    await inject_realtime_turn(rt, ev)

# Disable server VAD: SAA owns turn boundaries
model = openai.realtime.RealtimeModel(
    turn_detection=None,
)
session = model.session()
session.input.set_audio_enabled(False)  # model hears only SAA-injected audio

# Disable per-turn frame capture for pure audio pipelines
entrypoint = build_attention_entrypoint(
    on_turn=on_turn,
    attention_config={"frames_per_turn": 0},
)

Barge-in and proactive interjection

Two typed events: barge-in during playback and proactive interjection.

Call engine.responding_start() when playback begins and responding_stop() when it ends. This arms the interrupt branch.

Setup and requirements

One package, four environment variables, Python 3.10 or later.

Install saa-livekit-client and set the four environment variables below. SAA supports audio-only and performs best with video. Methodology: arXiv:2604.08412.

pip install saa-livekit-client

Common questions

What LiveKit agent developers ask before integrating.

Get started

Get your API key and start gating your LiveKit agent today.

Sign up for a free SAA_API_KEY and install saa-livekit-client. Need help with a nonstandard pipeline? Talk to us.

Related: how addressee detection works, Pipecat addressee detection integration, OpenAI Realtime addressee detection integration, voice infrastructure pipeline, engagement-control layer.

Last updated June 30, 2026