Table of Contents▼

CLIENT SDK

The Client SDK (@aether-stack-dev/client-sdk) runs in the browser and connects to a UE-hosted AStack gateway via WebSocket while your app embeds the signed renderer player separately.

INSTALLATION

$ npm install @aether-stack-dev/client-sdk

END-USER NOTICES

Show a processing notice before calling startCall() or requesting microphone access. Suggested wording:

This conversation uses your microphone. Your audio, messages and any images you share are processed to generate replies. The character's voice, face and replies are AI-generated and may be inaccurate. See our Privacy Policy for how data is handled.

Link to your own privacy policy, adapt the notice to your data flow and obtain any consent your application requires. For Deepgram BYOK, also disclose processing under that Deepgram account. Browser microphone permission alone does not replace these notices.

AStack account holders must accept the current Terms and Privacy Policy in the dashboard before purchases or paid sessions. The session API returns legal_acceptance_required (403) until they do.

CONFIGURATION

// Session credentials are returned by your backend after it calls the control plane
const session = await fetch('/api/start-session', { method: 'POST' }).then(r => r.json());


const client = new AStackCSRClient({
  sessionId: session.sessionId,
  gatewayTransport: session.gatewayTransport,
  gatewayWsUrl: session.gatewayWsUrl,     // Preferred per-session WebSocket URL
  gatewayUrl: session.gatewayUrl,         // HTTPS gateway origin fallback
  sessionToken: session.sessionToken,     // Auth token from the control plane
  rendererPlayerUrl: session.rendererPlayerUrl,
  sampleRate: 24000,                      // Audio sample rate (default: 24000)
  fps: 30,                                // Animation frame rate (default: 30)
  enableImageCapture: true,               // Enable camera capture (default: true)
  imageCaptureInterval: 5000,             // Capture interval ms (default: 5000)
});
gatewayWsUrlPreferred session WebSocket URL returned by the control plane
gatewayUrlHTTPS gateway origin used when gatewayWsUrl is omitted
sessionTokenSession token returned by the control plane
rendererPlayerUrlSigned Pixel Streaming URL for iframe or tab embedding
workerUrlTemporary migration alias for gatewayUrl; avoid in new code
enableImageCaptureSend camera frames for vision processing
autoReconnectAuto-reconnect on disconnect (default: true)
maxRetriesMax reconnection attempts (default: 5)
reconnectDelayBase delay between reconnects in ms, with exponential backoff (default: 1000)
audioProcessorUrlOptional self-hosted AudioWorklet URL. The SDK bundles the worklet and loads it from a Blob URL; set this only when your CSP blocks blob: scripts.
debugEnable debug logging (default: false)

METHODS

connect(): Promise<void>Establish WebSocket connection to the gateway and authenticate
startCall(options?: CallOptions): Promise<void>Initialize audio context and start streaming. The voiceAgent option explicitly selects self-hosted inference or Deepgram BYOK without carrying a key.
stopCall(): voidStop mic/camera capture and release local media resources
disconnect(): voidStop call and close WebSocket connection
sendText(message: string): voidSend text input to the AI (instead of voice)
getGatewayStatus(): GatewayConnectionStatusReturn gateway connection/auth state
getRendererPlayerUrl(): string | nullReturn the signed renderer player URL, if provided
getRendererPlayerState(): RendererPlayerStateReturn renderer player URL/status state separately from gateway auth
getCallStatus(): CallStatusReturns 'idle' | 'starting' | 'active' | 'stopping' | 'error'
isConnected(): booleanReturns true if WebSocket is open
destroy(): Promise<void>Disconnect and clean up all resources
getReconnectAttempts(): numberReturns current reconnection attempt count

STARTCALL OPTIONS

With the upcoming server-configured session contract, persona fields are pinned by your backend. Send only fields permitted by its agentConfig, or use startCall() without private values. Browser configuration requires an explicit backend opt-in.

await client.startCall({
  voiceAgent: { mode: 'self_hosted' },
  providers: {
    asr: 'self',       // 'self' | 'deepinfra' (default: 'self')
    llm: 'self',       // 'self' | 'deepinfra' (default: 'self')
    tts: 'self',       // 'self' | 'deepinfra' (default: 'self')
  },
  voice: 'selected-voice', // Only when the backend permits voice overrides.
});

EVENTS

The client extends EventEmitter. Listen for events with client.on(event, callback).

connected() => void
WebSocket connection established
disconnected() => void
WebSocket connection closed
reconnecting(attempt: number) => void
SDK is attempting to reconnect (with attempt count)
reconnected() => void
SDK successfully reconnected after connection loss
error(error: Error) => void
Connection, audio, or processing error
callStarted() => void
Call session started on the worker
callStopped() => void
Call session stopped
speechStarted() => void
User started speaking (VAD detected)
interim(text: string) => void
Interim/partial transcript from ASR
transcript(text: string) => void
Final user speech transcription
utteranceEnd() => void
User finished speaking (EOU detected)
response(text: string) => void
AI response text
responseComplete() => void
Full AI response text/control cycle completed
asrError(message: string) => void
ASR processing error occurred
modelStatus(status: { model_loaded?: boolean }) => void
Model loading status update
creditsExhausted() => void
Credits ran out (close 4010); the session has ended
sessionExpired(reason: string) => void
Session ended without reconnecting: 'session_expired' (close 4009) or 'credits_exhausted' (close 4010)
gatewayStatusChanged(status) => void
Gateway connection/auth state changed
rendererPlayerStateChanged(state) => void
Renderer player URL/status state changed

EXAMPLE

client.on('transcript', (text) => {
  console.log('User said:', text);
});


client.on('response', (text) => {
  console.log('AI:', text);
});


client.on('error', (error) => {
  console.error('AStack error:', error.message);
});

AUDIO PLAYER

The AudioPlayer is a standalone PCM helper. Hosted call audio comes from the signed renderer player over WebRTC, not from the gateway WebSocket.

import { AudioPlayer } from '@aether-stack-dev/client-sdk';


const player = new AudioPlayer(24000); // sample rate


player.enqueue({ audio: arrayBuffer });
player.clearQueue();
await player.destroy();


// Events: playbackStarted, playbackEnded, queueEmpty, error

REACT HOOKS

Import from @aether-stack-dev/client-sdk/react.

useAStackCSR

const {
  client,         // AStackCSRClient | null
  isConnected,    // boolean
  gatewayStatus,  // gateway connection/auth state
  rendererPlayerUrl, // signed Pixel Streaming/player URL
  callStatus,     // 'idle' | 'starting' | 'active' | 'stopping' | 'error'
  transcript,     // string — latest user speech
  response,       // string — latest AI response
  error,          // Error | null
  connect,        // () => Promise<void>
  disconnect,     // () => void
  startCall,      // () => Promise<void>
  stopCall,       // () => void
  sendText,       // (message: string) => void
} = useAStackCSR({
  session,                 // Full session API response
  autoConnect: false,       // auto-connect on mount (default: false)
});

RENDERER PLAYER

const { rendererPlayerUrl } = useAStackCSR({ session });


return rendererPlayerUrl ? (
  <iframe title="AStack renderer" src={rendererPlayerUrl} />
) : null;

ERROR HANDLING

import { AStackError, ErrorCodes } from '@aether-stack-dev/client-sdk';


client.on('error', (error) => {
  if (error instanceof AStackError) {
    switch (error.code) {
      case ErrorCodes.CONNECTION_LOST:
        // Reconnect logic
        break;
      case ErrorCodes.MEDIA_ACCESS_DENIED:
        // Prompt user for mic/camera permissions
        break;
    }
  }
});

ERROR CODES

INVALID_API_KEYAPI key is invalid or missing
INVALID_TOKENSession token is invalid
AUTHENTICATION_FAILEDAuthentication rejected by server
SESSION_EXPIREDSession has expired
CONNECTION_LOSTWebSocket connection dropped
NETWORK_ERRORNetwork connectivity issue
MEDIA_ACCESS_DENIEDMic/camera permission denied
AUDIO_ERRORAudio processing failure
RATE_LIMIT_EXCEEDEDToo many requests
INSUFFICIENT_CREDITSAccount has no remaining credits