Definition, mechanism, and hosted implementation
Device-Directed Speech Detection
The binary, pre-ASR decision of whether a spoken utterance is addressed to the device.
Where it fits in the pipeline
The addressee question is logically prior to every other classification. No standard layer answers it.
VAD answers whether speech is present. ASR transcribes it. NLU classifies its intent. None determine who the utterance was addressed to.
Device-directed speech detection is the missing prior: the binary decision, made before transcription, of whether downstream models should act at all.
Without it, any ambient speech reaching the microphone moves through the full pipeline as if it were addressed to the device.
| Layer | What it determines | When it runs | Carries addressee signal? |
|---|---|---|---|
| Voice activity detection (VAD) | Is someone speaking? | Pre-ASR | No |
| Wake word detection | Was a trigger phrase said? | Pre-ASR | Phrase only, not intent |
| Speaker diarization | Who spoke, and when? | Post-ASR | No |
| Intent classification (NLU / LLM) | What did the utterance mean? | Post-ASR | No |
| Selective Auditory Attention (SAA) device-directed speech detection | Is this utterance addressed to the device? | Pre-ASR, in real time | Yes |
the gap
No existing standard layer answers the addressee question
VAD fires on any sound above a noise floor. Wake words require a fixed phrase and still misfire on ambient speech. Diarization and intent classification run after transcription, long after the addressee signal was needed. The gap sits at the very front of the pipeline, before ASR begins.
the cost
Downstream models act on speech they should never receive
Without device-directed speech detection, ASR transcribes ambient conversation, intent models produce actions from bystander speech, and the system responds to people it was not talking to. Tuning ASR accuracy or intent precision does not fix a missing addressee layer.
The implementation
SAA is the hosted pre-ASR inference layer for device-directed speech detection.
SAA scores each utterance in real time and returns one decision: device-directed or not. Addressed audio moves to ASR; uncertain audio fails closed.
The methodology and benchmark results are documented in a technical paper we published at arXiv:2604.08412.
Audio streams in
Your microphone audio streams to the hosted SAA service through a thin client SDK. No model runs on your hardware, and your ASR and model stack stay unchanged.
Detect device-directed speech before speech-to-text
SAA analyzes how the utterance is shaped, not what words it contains, in real time. No wake word, no language parsing.
Route only device-directed audio
Device-directed utterances go to your ASR and model stack; everything else is dropped. Uncertain audio fails closed: not forwarded, not transcribed, not acted on.
Downstream models see only the right speech
Your intent classifier, dialogue manager, and model stack receive only audio genuinely addressed to the device. Ambient conversation, bystander speech, and competing voices never reach transcription. The addressee problem is solved at the layer where the addressee signal still exists.
cost and resilience
Non-addressed audio runs the full model stack
A voice agent pays STT, LLM, and TTS compute on every segment it receives. SAA drops non-addressed segments before ASR, before they reach the model stack.
Example scene
The same room. Two conversations. One should reach the device.
A person asks their voice assistant to set a reminder. At the same moment, someone nearby is finishing a sentence to someone else. Both utterances reach the microphone in the same audio frame. Without device-directed speech detection, the voice assistant may transcribe both and act on the wrong one.
SAA scores each utterance against a learned decision boundary, using acoustic signals rather than word content, before either reaches ASR. It forwards only the one addressed to the device and fails closed on anything uncertain.
Published benchmark
Published results on held-out multi-party sessions.
SAA reliably tells whether speech is addressed to the device, with audio-plus-video fusion and with audio only, on held-out multi-party sessions (arXiv:2604.08412).
Two caveats apply: it fails closed under distribution shift, and cross-lingual recall is a known limitation under active work.
performance
Reliable with audio and video fusion
Held-out multi-party sessions (arXiv:2604.08412).
performance
Reliable with audio only
Real multi-speaker audio, no video stream (arXiv:2604.08412).
latency
Real-time decision
Before ASR begins. No GPU required.
design principle
Fails closed by design
Uncertain utterances are not forwarded: no transcription, no action.
signal basis
Acoustic, not content
No wake word. No fixed trigger phrase. Language-agnostic at the acoustic level. Cross-lingual recall parity is a known open item under active work.
Device-directed speech detection is the binary classification of whether a spoken utterance is addressed to the device.
Common questions
What researchers and voice AI engineers ask.
What does SAA stand for?
SAA stands for Selective Auditory Attention: attention labs' hosted, pre-ASR engagement-control layer for device-directed speech detection. Underlying research: arXiv:2604.08412.
What is device-directed speech detection?
Device-directed speech detection is the real-time binary decision of whether a spoken utterance is addressed to the device or to another person. Without it, a voice assistant cannot distinguish speech meant for it from ambient conversation.
How does device-directed speech detection differ from VAD, diarization, and intent classification?
Four different questions: VAD asks if speech is present, diarization asks who spoke, intent asks what it meant, and device-directed speech detection asks whether it was addressed to the device. If not, even correct intent classification produces the wrong action.
The decision must run before ASR because the addressee signal is acoustic and is lost once transcription begins.
Does device-directed speech detection require a wake word?
No. SAA uses acoustic signals rather than a fixed trigger phrase. Wake words require a preamble before every interaction and still misfire when background speech contains the phrase.
SAA analyzes how an utterance is shaped, not what words it contains, and fails closed when uncertain. Cross-lingual recall parity is a known open item under active work.
What signals does SAA use to detect device-directed speech?
SAA reads acoustic signals, not content: how an utterance is shaped, not what words it contains. This includes patterns like pitch, rhythm, energy, timing, and directional cues, rather than parsed language. The approach is language-agnostic at the acoustic level, though cross-lingual recall parity is a known open item under active work.
How do I cite the SAA paper?
Cite arXiv:2604.08412: "SAA (Selective Auditory Attention): Device-Addressed Speech Detection for Real-Time Voice AI," at arxiv.org/abs/2604.08412.
It contains the evaluation protocol, benchmark results, and both caveats: fails closed under distribution shift; cross-lingual recall is a known limitation under active work.
How accurate is SAA at detecting device-directed speech?
On held-out multi-party sessions, SAA reliably tells whether speech is addressed to the device, with audio plus video fusion and with audio only (arXiv:2604.08412).
Two caveats travel with both results: performance fails closed under distribution shift, and cross-lingual recall is a known limitation under active work.
What is the cost benefit of device-directed speech detection?
A voice agent pays STT, LLM, and TTS compute on every audio segment it receives, whether or not that segment was addressed to it.
SAA makes the addressee decision before ASR, so non-addressed audio is dropped before any model processes it. This applies to any environment with ambient speech: multi-person rooms, drive-thrus, and one-to-one conversations with background noise.
As a labeled estimate, high-ambient deployments can recover a meaningful share of model-stack spend. The resilience benefit is structural: spurious actions from unaddressed audio become impossible.
Get in touch
Request an eval or reach us directly.
We provide API keys, an eval kit, and direct engineering support. The eval tier is free; production licensing is per device. A DPA is available for regulated deployments. Contact us at [email protected].
Last updated June 27, 2026