CLIENT SDK

The Client SDK (@aether-stack-dev/client-sdk) runs in the browser and connects to an AStack worker via WebSocket for real-time voice, text, and video AI.

INSTALLATION

$ npm install @aether-stack-dev/client-sdk

CONFIGURATION

// workerUrl is dynamic — returned by the server SDK when you create a session
const { sessionToken, workerUrl } = await fetch('/api/start-session').then(r => r.json());

const client = new AStackCSRClient({
  workerUrl: workerUrl,                   // Dynamic per-session URL from server
  sessionToken: sessionToken,             // Auth token from server SDK
  sessionId: 'sess_...',                  // Alternative: direct session ID
  sampleRate: 24000,                      // Audio sample rate (default: 24000)
  fps: 30,                                // Animation frame rate (default: 30)
  enableImageCapture: true,               // Enable camera capture (default: true)
  imageCaptureInterval: 5000,             // Capture interval ms (default: 5000)
});

workerUrlRequired. WebSocket URL of the AStack worker

sessionTokenAuth token generated by server SDK

enableImageCaptureSend camera frames for vision processing

autoReconnectAuto-reconnect on disconnect (default: true)

maxRetriesMax reconnection attempts (default: 5)

reconnectDelayBase delay between reconnects in ms, with exponential backoff (default: 1000)

audioProcessorUrlPath to AudioWorklet processor file (default: '/audio-processor.js')

debugEnable debug logging (default: false)

METHODS

connect(): Promise<void>Establish WebSocket connection and authenticate

startCall(options?: CallOptions): Promise<void>Initialize audio context, request mic/camera, start streaming. Options: providers, systemPrompt, priorContext, configOverrides, disableA2F

stopCall(): voidStop audio/video capture, reset blendshapes

disconnect(): voidStop call and close WebSocket connection

sendText(message: string): voidSend text input to the AI (instead of voice)

getCallStatus(): CallStatusReturns 'idle' | 'starting' | 'active' | 'stopping' | 'error'

getCurrentBlendshapes(): number[]Returns current 52-value blendshape array

isConnected(): booleanReturns true if WebSocket is open

destroy(): Promise<void>Disconnect and clean up all resources

getReconnectAttempts(): numberReturns current reconnection attempt count

STARTCALL OPTIONS

await client.startCall({
  providers: {
    asr: 'self',       // 'self' | 'deepinfra' (default: 'self')
    llm: 'self',       // 'self' | 'deepinfra' (default: 'self')
    tts: 'self',       // 'self' | 'deepinfra' (default: 'self')
  },
  systemPrompt: 'You are a helpful assistant.',
  disableA2F: false,
});

EVENTS

The client extends EventEmitter. Listen for events with client.on(event, callback).

connected() => void

WebSocket connection established

disconnected() => void

WebSocket connection closed

reconnecting(attempt: number) => void

SDK is attempting to reconnect (with attempt count)

reconnected() => void

SDK successfully reconnected after connection loss

error(error: Error) => void

Connection, audio, or processing error

callStarted() => void

Call session started on the worker

callStopped() => void

Call session stopped

speechStarted() => void

User started speaking (VAD detected)

interim(text: string) => void

Interim/partial transcript from ASR

transcript(text: string) => void

Final user speech transcription

utteranceEnd() => void

User finished speaking (EOU detected)

response(text: string) => void

AI response text

responseComplete() => void

Full AI response cycle completed (all chunks sent)

asrError(message: string) => void

ASR processing error occurred

blendshapeUpdate(blendshapes: number[]) => void

52 ARKit blendshape values for avatar animation

playbackStarted() => void

Audio playback began

playbackEnded() => void

Audio playback finished

modelStatus(status: { model_loaded?: boolean }) => void

Model loading status update

creditsExhausted() => void

Account credits have been exhausted

sessionExpired(reason: string) => void

Session has expired (with reason)

EXAMPLE

client.on('transcript', (text) => {
  console.log('User said:', text);
});

client.on('response', (text) => {
  console.log('AI:', text);
});

client.on('blendshapeUpdate', (blendshapes) => {
  // blendshapes is number[52] — ARKit blendshape weights (0-1)
  // Use with VRMAvatar for lip sync
});

client.on('error', (error) => {
  console.error('AStack error:', error.message);
});

AUDIO PLAYER

The AudioPlayer handles audio playback and blendshape synchronization. It is used internally by the client but can also be used standalone.

import { AudioPlayer } from '@aether-stack-dev/client-sdk';

const player = new AudioPlayer(24000); // sample rate

player.enqueue({ audio: arrayBuffer, blendshapes: frames });
player.clearQueue();
await player.destroy();

// Events: blendshapeUpdate, playbackStarted, playbackEnded, queueEmpty, error

REACT HOOKS

Import from @aether-stack-dev/client-sdk/react.

useAStackCSR

const {
  client,         // AStackCSRClient | null
  isConnected,    // boolean
  callStatus,     // 'idle' | 'starting' | 'active' | 'stopping' | 'error'
  blendshapes,    // number[] (52 values)
  transcript,     // string — latest user speech
  response,       // string — latest AI response
  error,          // Error | null
  characterConfig, // CharacterConfig — spread on VRMAvatar
  connect,        // () => Promise<void>
  disconnect,     // () => void
  startCall,      // () => Promise<void>
  stopCall,       // () => void
  sendText,       // (message: string) => void
} = useAStackCSR({
  workerUrl: workerUrl,    // Dynamic per-session URL from server
  sessionToken: sessionToken,
  autoConnect: false,       // auto-connect on mount (default: false)
});

AVATAR COMPONENTS

import { VRMAvatar } from '@aether-stack-dev/client-sdk/react';

// 3D VRM avatar with Three.js
<VRMAvatar
  blendshapes={blendshapes}    // number[52] from useAStackCSR
  width={400}                   // pixels (default: 400)
  height={400}                  // pixels (default: 400)
  modelUrl="https://xwqtyabmavpacejuypyp.supabase.co/storage/v1/object/public/vrm-templates/5705015963733407866.vrm"
/>

ERROR HANDLING

import { AStackError, ErrorCodes } from '@aether-stack-dev/client-sdk';

client.on('error', (error) => {
  if (error instanceof AStackError) {
    switch (error.code) {
      case ErrorCodes.CONNECTION_LOST:
        // Reconnect logic
        break;
      case ErrorCodes.MEDIA_ACCESS_DENIED:
        // Prompt user for mic/camera permissions
        break;
    }
  }
});

ERROR CODES

INVALID_API_KEYAPI key is invalid or missing

INVALID_TOKENSession token is invalid

AUTHENTICATION_FAILEDAuthentication rejected by server

SESSION_EXPIREDSession has expired

CONNECTION_LOSTWebSocket connection dropped

NETWORK_ERRORNetwork connectivity issue

MEDIA_ACCESS_DENIEDMic/camera permission denied

AUDIO_ERRORAudio processing failure

RATE_LIMIT_EXCEEDEDToo many requests

INSUFFICIENT_CREDITSAccount has no remaining credits