CLIENT SDK

The Client SDK (@aether-stack-dev/client-sdk) runs in the browser and connects to an AStack worker via WebSocket for real-time voice, text, and video AI.

INSTALLATION

$ npm install @aether-stack-dev/client-sdk

CONFIGURATION

const client = new AStackCSRClient({
  workerUrl: 'wss://worker.astack.dev',  // Required
  sessionToken: 'st_...',                 // Auth token from server SDK
  sessionId: 'sess_...',                  // Alternative: direct session ID
  sampleRate: 24000,                      // Audio sample rate (default: 24000)
  providers: {                            // AI provider selection
    asr: 'self',                          // 'self' | 'deepinfra' (default: 'self')
    llm: 'self',                          // 'self' | 'deepinfra' (default: 'self')
    tts: 'self',                          // 'self' | 'deepinfra' (default: 'self')
  },
  fps: 30,                                // Animation frame rate (default: 30)
  enableImageCapture: true,               // Enable camera capture (default: true)
  imageCaptureInterval: 5000,             // Capture interval ms (default: 5000)
});

workerUrlRequired. WebSocket URL of the AStack worker

sessionTokenAuth token generated by server SDK

providersASR/LLM/TTS provider: 'self' or 'deepinfra'

enableImageCaptureSend camera frames for vision processing

METHODS

connect(): Promise<void>Establish WebSocket connection and authenticate

startCall(): Promise<void>Initialize audio context, request mic/camera, start streaming

stopCall(): voidStop audio/video capture, reset blendshapes

disconnect(): voidStop call and close WebSocket connection

sendText(message: string): voidSend text input to the AI (instead of voice)

getCallStatus(): CallStatusReturns 'idle' | 'starting' | 'active' | 'stopping' | 'error'

getCurrentBlendshapes(): number[]Returns current 52-value blendshape array

isConnected(): booleanReturns true if WebSocket is open

destroy(): Promise<void>Disconnect and clean up all resources

EVENTS

The client extends EventEmitter. Listen for events with client.on(event, callback).

connected() => void

WebSocket connection established

disconnected() => void

WebSocket connection closed

error(error: Error) => void

Connection, audio, or processing error

callStarted() => void

Call session started on the worker

callStopped() => void

Call session stopped

transcript(text: string) => void

User speech transcription

response(text: string) => void

AI response text

blendshapeUpdate(blendshapes: number[]) => void

52 ARKit blendshape values for avatar animation

playbackStarted() => void

Audio playback began

playbackEnded() => void

Audio playback finished

modelStatus(status: { model_loaded?: boolean }) => void

Model loading status update

EXAMPLE

client.on('transcript', (text) => {
  console.log('User said:', text);
});

client.on('response', (text) => {
  console.log('AI:', text);
});

client.on('blendshapeUpdate', (blendshapes) => {
  // blendshapes is number[52] — ARKit blendshape weights (0-1)
  // Use with VRMAvatar or TalkingHeadAvatar for lip sync
});

client.on('error', (error) => {
  console.error('AStack error:', error.message);
});

AUDIO PLAYER

The AudioPlayer handles audio playback and blendshape synchronization. It is used internally by the client but can also be used standalone.

import { AudioPlayer } from '@aether-stack-dev/client-sdk';

const player = new AudioPlayer(24000); // sample rate

player.enqueue({ audio: arrayBuffer, blendshapes: frames });
player.clearQueue();
await player.destroy();

// Events: blendshapeUpdate, playbackStarted, playbackEnded, queueEmpty, error

REACT HOOKS

Import from @aether-stack-dev/client-sdk/react.

useAStackCSR

const {
  client,         // AStackCSRClient | null
  isConnected,    // boolean
  callStatus,     // 'idle' | 'starting' | 'active' | 'stopping' | 'error'
  blendshapes,    // number[] (52 values)
  transcript,     // string — latest user speech
  response,       // string — latest AI response
  error,          // Error | null
  connect,        // () => Promise<void>
  disconnect,     // () => void
  startCall,      // () => Promise<void>
  stopCall,       // () => void
  sendText,       // (message: string) => void
} = useAStackCSR({
  workerUrl: 'wss://worker.astack.dev',
  sessionToken: 'st_...',
  autoConnect: false,  // auto-connect on mount (default: false)
});

AVATAR COMPONENTS

import { VRMAvatar, TalkingHeadAvatar } from '@aether-stack-dev/client-sdk/react';

// 3D VRM avatar with Three.js
<VRMAvatar
  blendshapes={blendshapes}    // number[52] from useAStackCSR
  width={400}                   // pixels (default: 400)
  height={400}                  // pixels (default: 400)
  modelUrl="/models/avatar.vrm" // VRM/GLB model path
/>

// Ready Player Me avatar with TalkingHead
<TalkingHeadAvatar
  blendshapes={blendshapes}
  width={400}
  height={400}
  avatarUrl="https://models.readyplayer.me/..."
/>

ERROR HANDLING

import { AStackError, ErrorCodes } from '@aether-stack-dev/client-sdk';

client.on('error', (error) => {
  if (error instanceof AStackError) {
    switch (error.code) {
      case ErrorCodes.CONNECTION_LOST:
        // Reconnect logic
        break;
      case ErrorCodes.MEDIA_ACCESS_DENIED:
        // Prompt user for mic/camera permissions
        break;
    }
  }
});

ERROR CODES

INVALID_API_KEYAPI key is invalid or missing

INVALID_TOKENSession token is invalid

AUTHENTICATION_FAILEDAuthentication rejected by server

SESSION_EXPIREDSession has expired

CONNECTION_LOSTWebSocket connection dropped

NETWORK_ERRORNetwork connectivity issue

MEDIA_ACCESS_DENIEDMic/camera permission denied

AUDIO_ERRORAudio processing failure

RATE_LIMIT_EXCEEDEDToo many requests

INSUFFICIENT_CREDITSAccount has no remaining credits