JavaScript SDK

@attenlabs/saa-js streams your mic and webcam from the browser and classifies every frame as silent, human-directed, or device-directed. No wake word. You decide what reaches your language model; the SDK handles capture, streaming, and event typing.

Overview

Wraps the inference server for the browser: grabs mic and camera via getUserMedia, captures audio through an AudioWorklet and video through canvas, and manages the WebSocket. The model is versioned and hosted for you.

Install

npm install @attenlabs/saa-js

View the SDK on GitHub

Requires a browser: getUserMedia, AudioContext, AudioWorklet, Canvas, and WebSocket. Sign up for the dashboard to get a key now and pass it as the token option. Keep tokens secret; load from an environment variable rather than hardcoding.

import { AttentionClient } from "@attenlabs/saa-js";

const client = new AttentionClient({ token: "your-token" });

client.on("prediction", (e) => {
  const label = { 0: "silent", 1: "human", 2: "device" }[e.cls] ?? "?";
  console.log(`${label}  ${(e.confidence * 100).toFixed(0)}%  faces=${e.numFaces}`);
});

client.on("turnReady", (e) => {
  console.log(`turn ready (${e.durationSec.toFixed(2)}s)`);
  // e.audioBase64 is PCM16 @ 16 kHz, ready to forward to your LLM
});

const video = document.getElementById("preview");
await client.start({ videoElement: video });

Calling start() resolves your connection in two steps:

• POST to the broker's /allocate endpoint (default https://broker.attentionlabs.ai, override with url) with your token as a Bearer header; it returns a session WebSocket URL.
• The SDK opens that WebSocket, passing the token again as the subprotocol.

Each reconnect re-runs allocate for a fresh backend. Pass a ws:// or wss:// URL as url to skip the broker and connect directly.

Quickstart

Add a <video> element for camera preview, then initialize the client and start streaming. Listen for prediction and error at minimum: cls is 0 (silent), 1 (human-directed), or 2 (device-directed). Only forward class 2 to your language model.

import { AttentionClient } from "@attenlabs/saa-js";

const client = new AttentionClient({ token: "your-token-here" });

const CLASS_LABELS = { 0: "silent", 1: "human", 2: "device" };
const log = document.getElementById("log");

client.on("connected", () => { log.textContent += "connected\n"; });
client.on("started", () => { log.textContent += "warmup complete, streaming live\n"; });

client.on("prediction", (e) => {
  const label = CLASS_LABELS[e.cls] ?? "?";
  log.textContent = `${label}  ${(e.confidence * 100).toFixed(0)}%  faces=${e.numFaces}  src=${e.source}\n`;
});

client.on("turnReady", (e) => {
  log.textContent += `turn ready (${e.durationSec.toFixed(2)}s)\n`;
  // Forward e.audioBase64 to your LLM here
});

client.on("error", (e) => {
  log.textContent += `ERROR: ${e.title}: ${e.message}\n`;
});

const video = document.getElementById("preview");
await client.start({ videoElement: video });

Call mute() and markResponding(true) while your LLM is speaking, then unmute() and markResponding(false) when it's done. This stops the SDK from mistaking playback for a new prediction.

function onLlmSpeaking() {
  client.mute();
  client.markResponding(true);
}

function onLlmDone() {
  client.unmute();
  client.markResponding(false);
}

AttentionClient

The constructor accepts a token plus optional capture and threshold settings.

const client = new AttentionClient({
  token: "...",                  // auth token (Bearer to broker + WS subprotocol)
  url: undefined,                // broker base URL (default https://broker.attentionlabs.ai)
  video: {},                     // video capture config (VideoCaptureOptions)
  audio: {},                     // audio capture config (AudioCaptureOptions)
  workletUrl: undefined,         // override the AudioWorklet module URL
  initialThreshold: 0.7,         // device-class confidence threshold (0..1)
  enableAudio: true,             // set false to skip mic capture
  enableVideo: true,             // set false to skip webcam capture
  serverProfile: undefined,      // override server profile; auto "audio_only" when video off, "default" forces the full processor
});

VideoCaptureOptions: width (default 1920), height (default 1080), jpegQuality (0..1, default 0.5). AudioCaptureOptions: targetSampleRate (default 16000; audio always captures at 16 kHz mono).

Call client.start({ videoElement }); videoElement is required when video is enabled. Already have a stream? Pass start({ mediaStream }) to skip the SDK's own getUserMedia call.

Properties: isConnected (boolean, true when the WebSocket is open) and currentThreshold (number, the current confidence threshold).

Methods

JavaScript method names are camelCase.

Events

Register handlers with client.on("prediction", cb); the returned function unsubscribes the listener, or call client.off("prediction", cb).

Event types

Audio and video

Audio streams at 16 kHz mono in 100 ms chunks (1600 samples). Video frames capture as JPEG at about 4 fps (every 250 ms) and drop if the WebSocket send buffer exceeds 1 MB.

// The SDK binds the camera stream to your video element and
// captures both mic and webcam once you call start().
const video = document.getElementById("preview");
await client.start({ videoElement: video });

// Disable the camera with enableVideo: false when constructing the client.

Errors

Errors arrive as an error event: title, message, detail, and code (WebSocket close code). Auth failures show title "Auth Failed"; create a fresh token in the dashboard if yours is missing, expired, or revoked.

client.on("error", (e) => {
  console.log(`${e.title}: ${e.message}`, e.detail, e.code);
});

Looking for the shared connection model, integrations, or the Python client? See the documentation home and the Python SDK.