WebRTC Voice Agent

This example builds a low-latency voice agent in the browser, with WebRTC between the browser and your server and a WebSocket from the server to the Grok API. It's a starting point when voice latency in the browser matters most.

Note: These are example implementations for learning and development, and they aren't production-ready without additional hardening.

This example demonstrates a WebRTC-based voice agent that provides:

• Low-latency voice communication via WebRTC
• Real-time bidirectional audio streaming
• Connection quality monitoring (bitrate, jitter, packet loss)
• Server-side relay between WebRTC (browser) and WebSocket (XAI API)

Browser (WebRTC Client)
    ↓ WebRTC (PCM16 via DataChannel)
    ↓
Node.js Server (Relay)
    ↓ WebSocket (PCM16 pass-through)
    ↓
XAI API

The server acts as a relay:

• Client → Server: WebRTC DataChannel with PCM16 audio
• Server → XAI: WebSocket with PCM16 audio format
• No audio conversion needed: PCM16 used throughout

• Node.js 18 or higher
• XAI API key
• Chrome or Edge browser (WebRTC support)

cd server
# Create .env file with your XAI_API_KEY
echo "XAI_API_KEY=your_key_here" > .env
./start.sh

The server will start on port 8000 (configurable via .env).

cd client
./start.sh

The client will start on port 5173.

Navigate to http://localhost:5173 and click START to begin voice conversation.

Node.js + TypeScript server that:

• Accepts WebRTC connections from browsers
• Handles signaling (SDP offer/answer, ICE candidates)
• Relays PCM16 audio via DataChannel
• Connects to XAI API via WebSocket
• Collects connection quality stats

Tech Stack: Express, werift (pure JS WebRTC), WebSocket, TypeScript

See server/README.md for details.

React + TypeScript web application that:

• Establishes WebRTC connection to server
• Captures microphone audio via WebRTC
• Displays real-time transcripts
• Shows WebRTC connection quality stats
• Provides debug console for messages

Tech Stack: React, Vite, WebRTC API, TypeScript

See client/README.md for details.

• Client-server WebRTC connection
• DataChannel for audio and control messages
• STUN/TURN support for NAT traversal (configurable)
• Connection quality monitoring
• PCM16 audio format (no codec conversion needed)

• Real-time voice interaction
• Server-side Voice Activity Detection (VAD)
• Conversation transcripts
• Interruption handling
• Debug console for monitoring

• WebRTC badge in header
• Connection quality indicator
• Stats panel (bitrate, jitter, packet loss, RTT)
• Microphone level indicator
• Live transcript display (persists between sessions)

Edit server/.env:

# Required
XAI_API_KEY=your_xai_api_key_here

# Optional
PORT=8000
VOICE=ara
INSTRUCTIONS="You are a helpful voice assistant. You are speaking to a user in real-time over audio. Keep your responses conversational and concise since they will be spoken aloud."
API_URL=wss://api.x.ai/v1/realtime

The client automatically connects to http://localhost:8000 (or VITE_API_BASE_URL if set).

Both client and server include an ENABLE_TURN flag to optionally enable TURN servers for restrictive network environments:

Location:

• Client: client/src/hooks/useWebRTC.ts (line 13)
• Server: server/src/rtc-peer.ts (line 9)

Default: false (STUN only, recommended for most networks)

When to enable:

• Restrictive corporate firewalls that block UDP
• Networks that require TURN relay
• Testing TURN server functionality

Note: Enabling TURN may add 10-15 seconds to connection time if TURN server is slow/unreachable. Most connections work fine with STUN only.

Recommended: Chrome or Edge for best compatibility.

Both client and server use TypeScript for type safety and better developer experience.

cd server
npm run dev  # Run with ts-node (no build step)
npm run watch  # Watch mode for TypeScript

cd client
npm run dev  # Vite dev server with HMR