Addressee detection for voice AI builders
Voice separation finds every voice, not who it's for.
Selective Auditory Attention (SAA) adds that missing decision, in real time, before ASR runs, and fails closed when unsure.
Definition
What is the cocktail party problem?
The cocktail party problem describes how people follow one voice among many in a noisy room. E. Colin Cherry named it in 1953.
Voice separation only clears the mix. A bystander saying your wake phrase to a friend still produces a clean signal, and ASR will act on it.
SAA answers what separation cannot: which voice is addressed to the device. It runs before ASR, in real time, and fails closed when unsure.
In the original 1953 formulation, the listener is always the intended recipient. In a deployed voice AI product that assumption breaks constantly: a drive-through assistant hears the driver, the back seat, the crew, and the radio at once; a lobby robot is surrounded by people talking to each other, not to it; an in-cabin assistant receives speech from every seat. The question is not only which voice, but whether any voice is addressing the device at all.
That second question, directed at the device or not, is the engagement-control problem. It needs a dedicated layer between the microphone and the recognition pipeline. Source separation, voice activity detection, and wake words each solve a slice of it. None of them close the loop on addressee intent, in real time, before transcription begins.
The gap in current voice stacks
No layer in your voice stack knows who is being addressed.
Separation, VAD, diarization, and wake words each solve a different piece of the audio problem.
| Layer | What it answers | When it runs | Decides who is addressed? |
|---|---|---|---|
| Source separation | Which individual voices are in the mix? | Pre-ASR | No |
| Voice activity detection (VAD) | Is someone speaking right now? | Pre-ASR | No |
| Speaker diarization | Who spoke, after the fact? | Post-ASR | No |
| Wake word detection | Was the trigger phrase said? | Pre-ASR | Trigger phrase only |
| SAA addressee detection | Is the device being addressed right now? | Pre-ASR, real time | Yes |
The two-stage solution
Where SAA sits: after separation, before ASR.
It decides, per voice, whether that voice is addressed to the device or to someone else in the room.
Drop in before your ASR, no pipeline changes
Audio and optional video stream to the hosted SAA service via a thin SDK (JavaScript/TypeScript or Python).
No model runs on your hardware. No GPU. No wake word. SAA inserts as a single stage between your microphone and ASR.
Addressee decision in real time, from acoustic signal alone
SAA analyzes acoustic and prosodic signals in the raw audio frames and returns addressed or not addressed, before ASR runs.
No trigger phrase. No text. No post-hoc LLM call.
Only addressed audio reaches your model; everything else is suppressed
Addressed audio forwards to your ASR and language model. Non-addressed speech is suppressed.
Uncertain audio fails closed: nothing is forwarded on a guess. You keep your existing ASR, model, and pipeline.
Performance
Reliable on the addressee decision, audio-only or fused with video.
Measured across held-out multi-party sessions with multiple concurrent speakers. Full methodology: arXiv:2604.08412.
Two caveats: it fails closed under distribution shift, and cross-lingual recall is a known limitation under active work.
performance
High accuracy with audio and video
Combining audio and video gives SAA the clearest signal for the decision.
performance
Accurate on audio alone
No camera required. The same decision runs on audio alone.
latency
Real-time decision
The routing decision returns in real time, before ASR begins processing.
design principle
Fails closed by design
Ambiguous speech is suppressed, not passed through. Silence beats a forwarded guess.
Deployment contexts
Every shared acoustic space is a cocktail party.
Different environments, same failure: a voice agent in a real acoustic space can hear audio it was never meant to act on, in one-to-one calls too.
STT, LLM, and TTS run on every second received, including audio addressed to no one. SAA gates it before ASR, so that compute never runs.
Drive-through ordering
Driver, backseat, crew, radio: four clean channels, and none of them are labelled "for the AI."
Without an addressee gate, the agent acts on whichever voice is loudest.
Service robots
A robot in a lobby hears dozens of conversations it is not part of. A wake word fires on bystanders; addressee detection does not.
In-cabin voice assistants
Beamforming and seat routing separate audio spatially, but not by intent.
When two seats speak at once, neither one says which occupant the device should answer.
Ambient clinical AI
Clinician, patient, family members: without addressee detection, the scribe captures all of them and attributes utterances to the wrong role.
Wrong attribution here is a documentation error in the chart.
Your agent does not have a separation problem.
Common questions
What voice AI builders ask about the cocktail party problem.
What is the cocktail party problem?
The cocktail party problem describes how people follow one voice among many in a noisy room. E. Colin Cherry named it in 1953.
For voice AI it has a second layer: the device must decide which voice, if any, is addressed to it, not just which voices are present.
Does source separation solve the cocktail party problem for voice AI?
Separation isolates voices. A cleanly isolated voice can still be addressed to another person, not the device.
SAA adds the missing decision: in real time, before ASR runs, it determines whether each voice is device-directed.
How is addressee detection different from voice separation?
Voice separation answers which voice is present. Addressee detection answers whether that voice is talking to the device.
Separation untangles mixed audio into clean channels. SAA classifies each channel by intent, using acoustic and prosodic signals rather than content, and forwards only the addressed one to ASR.
No text. No trigger phrase.
Can SAA handle four people talking at once?
Yes. SAA is designed for real multi-party environments with multiple concurrent speakers.
It decides which, if any, of the active voices is addressed to the device, and forwards only that audio onward.
When it cannot determine with confidence that any utterance is device-addressed, it fails closed: audio is not forwarded rather than passed through on a guess.
Where does this matter in production?
Any voice AI device in a shared acoustic space faces this: drive-throughs, lobbies, vehicle cabins, and exam rooms all carry voices the device was never meant to act on.
In each case the fix is a pre-ASR addressee gate that routes only addressed audio to the model.
Get in touch
Run SAA on audio from your own deployment.
Describe your product and the acoustic environment it runs in. We will follow up with next steps for a free evaluation; licensing is per device.
Last updated June 26, 2026