Table of Contents▼

CLIENT SDK

The Client SDK (@aether-stack-dev/client-sdk) runs in the browser and connects to a UE-hosted AStack gateway via WebSocket while your app embeds the signed renderer player separately.

INSTALLATION

$ npm install @aether-stack-dev/client-sdk

CONFIGURATION

// Session credentials are returned by your backend after it calls the control plane
const session = await fetch('/api/start-session', { method: 'POST' }).then(r => r.json());


const client = new AStackCSRClient({
  gatewayWsUrl: session.gatewayWsUrl,     // Preferred per-session WebSocket URL
  gatewayUrl: session.gatewayUrl,         // HTTPS gateway origin fallback
  sessionToken: session.sessionToken,     // Auth token from the control plane
  rendererPlayerUrl: session.rendererPlayerUrl,
  sampleRate: 24000,                      // Audio sample rate (default: 24000)
  fps: 30,                                // Animation frame rate (default: 30)
  enableImageCapture: true,               // Enable camera capture (default: true)
  imageCaptureInterval: 5000,             // Capture interval ms (default: 5000)
});
gatewayWsUrlPreferred WebSocket URL for the UE-hosted customer gateway
gatewayUrlHTTPS gateway origin used when gatewayWsUrl is omitted
sessionTokenSession token returned by the control plane
rendererPlayerUrlSigned Pixel Streaming URL for iframe or tab embedding
workerUrlTemporary migration alias for gatewayUrl; avoid in new code
enableImageCaptureSend camera frames for vision processing
autoReconnectAuto-reconnect on disconnect (default: true)
maxRetriesMax reconnection attempts (default: 5)
reconnectDelayBase delay between reconnects in ms, with exponential backoff (default: 1000)
audioProcessorUrlPath to AudioWorklet processor file (default: '/audio-processor.js')
debugEnable debug logging (default: false)

METHODS

connect(): Promise<void>Establish WebSocket connection to the gateway and authenticate
startCall(options?: CallOptions): Promise<void>Initialize audio context, request mic/camera, and start streaming through the session gateway. Options: providers, systemPrompt, priorContext, configOverrides
stopCall(): voidStop mic/camera capture and release local media resources
disconnect(): voidStop call and close WebSocket connection
sendText(message: string): voidSend text input to the AI (instead of voice)
getGatewayStatus(): GatewayConnectionStatusReturn gateway connection/auth state
getRendererPlayerUrl(): string | nullReturn the signed renderer player URL, if provided
getRendererPlayerState(): RendererPlayerStateReturn renderer player URL/status state separately from gateway auth
getCallStatus(): CallStatusReturns 'idle' | 'starting' | 'active' | 'stopping' | 'error'
isConnected(): booleanReturns true if WebSocket is open
destroy(): Promise<void>Disconnect and clean up all resources
getReconnectAttempts(): numberReturns current reconnection attempt count

STARTCALL OPTIONS

await client.startCall({
  providers: {
    asr: 'self',       // 'self' | 'deepinfra' (default: 'self')
    llm: 'self',       // 'self' | 'deepinfra' (default: 'self')
    tts: 'self',       // 'self' | 'deepinfra' (default: 'self')
  },
  systemPrompt: 'You are a helpful assistant.',
  configOverrides: { EOU_SILENCE_MS: 800 },
});

EVENTS

The client extends EventEmitter. Listen for events with client.on(event, callback).

connected() => void
WebSocket connection established
disconnected() => void
WebSocket connection closed
reconnecting(attempt: number) => void
SDK is attempting to reconnect (with attempt count)
reconnected() => void
SDK successfully reconnected after connection loss
error(error: Error) => void
Connection, audio, or processing error
callStarted() => void
Call session started on the worker
callStopped() => void
Call session stopped
speechStarted() => void
User started speaking (VAD detected)
interim(text: string) => void
Interim/partial transcript from ASR
transcript(text: string) => void
Final user speech transcription
utteranceEnd() => void
User finished speaking (EOU detected)
response(text: string) => void
AI response text
responseComplete() => void
Full AI response text/control cycle completed
asrError(message: string) => void
ASR processing error occurred
modelStatus(status: { model_loaded?: boolean }) => void
Model loading status update
creditsExhausted() => void
Account credits have been exhausted
sessionExpired(reason: string) => void
Session has expired (with reason)
gatewayStatusChanged(status) => void
Gateway connection/auth state changed
rendererPlayerStateChanged(state) => void
Renderer player URL/status state changed

EXAMPLE

client.on('transcript', (text) => {
  console.log('User said:', text);
});


client.on('response', (text) => {
  console.log('AI:', text);
});


client.on('error', (error) => {
  console.error('AStack error:', error.message);
});

AUDIO PLAYER

The AudioPlayer is a standalone PCM helper. Hosted call audio comes from the signed renderer player over WebRTC, not from the gateway WebSocket.

import { AudioPlayer } from '@aether-stack-dev/client-sdk';


const player = new AudioPlayer(24000); // sample rate


player.enqueue({ audio: arrayBuffer });
player.clearQueue();
await player.destroy();


// Events: playbackStarted, playbackEnded, queueEmpty, error

REACT HOOKS

Import from @aether-stack-dev/client-sdk/react.

useAStackCSR

const {
  client,         // AStackCSRClient | null
  isConnected,    // boolean
  gatewayStatus,  // gateway connection/auth state
  rendererPlayerUrl, // signed Pixel Streaming/player URL
  callStatus,     // 'idle' | 'starting' | 'active' | 'stopping' | 'error'
  transcript,     // string — latest user speech
  response,       // string — latest AI response
  error,          // Error | null
  connect,        // () => Promise<void>
  disconnect,     // () => void
  startCall,      // () => Promise<void>
  stopCall,       // () => void
  sendText,       // (message: string) => void
} = useAStackCSR({
  session,                 // Full session API response
  autoConnect: false,       // auto-connect on mount (default: false)
});

RENDERER PLAYER

const { rendererPlayerUrl } = useAStackCSR({ session });


return rendererPlayerUrl ? (
  <iframe title="AStack renderer" src={rendererPlayerUrl} />
) : null;

ERROR HANDLING

import { AStackError, ErrorCodes } from '@aether-stack-dev/client-sdk';


client.on('error', (error) => {
  if (error instanceof AStackError) {
    switch (error.code) {
      case ErrorCodes.CONNECTION_LOST:
        // Reconnect logic
        break;
      case ErrorCodes.MEDIA_ACCESS_DENIED:
        // Prompt user for mic/camera permissions
        break;
    }
  }
});

ERROR CODES

INVALID_API_KEYAPI key is invalid or missing
INVALID_TOKENSession token is invalid
AUTHENTICATION_FAILEDAuthentication rejected by server
SESSION_EXPIREDSession has expired
CONNECTION_LOSTWebSocket connection dropped
NETWORK_ERRORNetwork connectivity issue
MEDIA_ACCESS_DENIEDMic/camera permission denied
AUDIO_ERRORAudio processing failure
RATE_LIMIT_EXCEEDEDToo many requests
INSUFFICIENT_CREDITSAccount has no remaining credits