Engagement control for voice AI
Your voice agent responds to the wrong person.
VAD fires on any speech. ASR transcribes it. The LLM acts on it. None of those layers ask whether the device was being spoken to.
Root cause
The standard voice stack never asks who the speech was meant for.
Voice AI was designed for one speaker in a quiet room. Real deployments rarely are. Everything that clears VAD reaches ASR, regardless of who it was addressing.
That is why the problem survives VAD tuning, barge-in upgrades, and ASR improvements: none of them add the missing decision.
VAD fires on any voice, not on speech addressed to the device
VAD tells the system someone is speaking, not who they are addressing. Every turn that clears it still runs full-cost ASR and LLM inference.
Wake words detect a phrase, not who it was meant for
Any voice that says the trigger phrase fires the pipeline, regardless of who it was meant for.
Diarization labels who spoke; it does not decide who was addressed
Speaker diarization runs after ASR and labels who spoke. It cannot gate audio before recognition, so background speakers still get transcribed at full cost.
The pipeline treats everything that clears VAD as a command
That assumption holds in a quiet, single-speaker room. It breaks in any room with more than one voice, and the error compounds: ASR, the model, and TTS all run on speech that was never meant for the device.
Symptom, cause, and fix
What the wrong-person failure looks like end to end.
A caller at home: a family member speaks in the background, the TV is on. VAD fires on all of it, and the pipeline runs ASR and LLM inference on every turn.
How SAA fixes it
One layer, inserted before ASR, that decides whether the device is being addressed.
SAA is a hosted, pre-ASR layer that sits between your microphone and your existing ASR. For each utterance it returns one decision: addressed or not.
Audio streams to the hosted SAA service via thin SDK
A thin client SDK (JS/TS or Python) streams audio, and optionally video, to the hosted SAA service. No hardware, wake word, or GPU required.
Addressee decision returned in real time, before ASR
SAA classifies each utterance before ASR runs and returns the decision in real time. When uncertain, it fails closed: no audio reaches ASR.
Only addressed audio reaches your ASR and model
Everything else is suppressed before recognition runs. Your ASR, LLM, and TTS are unchanged; SAA is model-agnostic and sits in front of any speech stack you already run.
architecture
Hosted, pre-ASR, behind any model
The client SDK streams audio to the hosted SAA service and receives the addressee decision before transcription begins. A Data Processing Agreement is available for regulated deployments.
access
Eval tier is free. Licensing is per device.
Request an evalOr try the Cloud SDK to get a feel.
Why existing tools miss it
VAD, wake words, and diarization each answer a different question than addressee.
Each layer below solves a real problem. None of them decide whether the device is the one being addressed.
| Layer | Question it answers | Runs before ASR? | Addressee signal? | Trigger phrase needed? |
|---|---|---|---|---|
| Voice activity detection (VAD) | Is someone speaking? | Yes | No | No |
| Wake word detection | Was the trigger phrase said? | Yes | No | Yes |
| Speaker diarization | Who spoke, and when? | No (post-ASR) | No | No |
| Barge-in / endpointing | Did the user start speaking over the agent? | Partial | No | No |
| SAA addressee detection | Is this speech directed at the device? | Yes, in real time | Yes | No |
Where the problem shows up
The same structural failure takes a different form in each deployment environment.
Each environment below links to the specific failure mode and what SAA changes operationally.
Drive-thru / QSR
AI takes orders from backseat passengers, car radio, and crew
Robots & kiosks
Robot fires on bystanders in a lobby or corridor
Ambient clinical
Scribe attributes patient speech to the wrong role
In-cabin automotive
Assistant activates on passenger or rear-seat occupant
Aviation
Gate kiosk fires on PA announcements and traveling companions
Voice infrastructure
Agent stops on background speech, misses real interruptions
Contact center
Supervisor whisper triggers a false escalation on a live call
Wearables
Smart glasses wake on ambient conversation, not the wearer
Conferencing
Meeting AI folds side conversations into the official summary
For the concept behind all of these, see what is addressee detection and how SAA integrates.
Technical methodology
Measured on held-out multi-party sessions.
Our published paper (arXiv:2604.08412) evaluates SAA on audio-video and audio-only conditions. It fails closed under distribution shift; cross-lingual recall is still limited.
performance
Accurate on audio and video fusion
A video stream sharpens the addressee decision when cameras are available.
performance
Accurate on audio-only
No camera required. SAA classifies addressee from voice alone.
latency
Real-time decision
The addressee decision is returned before ASR begins.
design
Fails closed by default
Ambiguous speech is suppressed, not forwarded, so uncertainty never becomes a false activation.
signal model
Acoustic and prosodic, not language parsing
No wake word, no transcript required. Language-agnostic at the acoustic level; cross-lingual recall parity is a known open item under active work.
VAD tells you someone is speaking. Diarization tells you who spoke.
Neither tells you whether the device was the one being addressed.
Common questions
What teams ask when they first encounter this failure in production.
Why does my voice agent respond to bystanders?
The pipeline has no layer that decides whether speech is addressed to the device. Anything that clears VAD reaches ASR and your model as a candidate command.
We already upgraded our barge-in and VAD sensitivity. Why does the problem persist?
Barge-in and VAD control when the pipeline fires, not who it fires for. Addressee detection is the missing layer, and it runs before ASR.
Is this an ASR accuracy problem?
No. The recognizer works fine; nothing upstream decided if the speech was addressed to the device. The fix is a pre-ASR gate, not better transcription.
Will a wake word fix it?
Not reliably. A wake word detects a phrase, not who it was meant for, so it still misfires when background speech contains the trigger phrase and misses requests phrased without it. In multi-speaker rooms, any voice saying the phrase can activate the pipeline.
Does SAA replace our ASR, LLM, or turn-detection logic?
No. SAA is a pre-ASR gate between your microphone and your ASR. It forwards only addressed audio; your ASR, LLM, and turn-detection are unchanged.
How do I evaluate SAA on my own audio?
The evaluation tier is free. Request an eval for API keys and an eval kit, or try the self-serve Cloud SDK to get a feel first.
Get in touch
Run SAA against your own audio and see the addressee decisions before any commitment.
We provide API keys, an eval kit, and the client SDK to measure addressee decisions against your own deployment audio.
Related: how to reduce false triggers in voice AI · what background speech costs a voice agent · device-directed speech detection
Last updated June 29, 2026
Alternatively: [email protected]