Voice

session.update

Update session configuration such as system prompt, voice, audio format, turn detection, and tools.

input_audio_buffer.append

Append chunks of base64-encoded audio data to the input buffer. The server does not send back a corresponding message.

input_audio_buffer.commit

Commit the audio buffer as a user message. Only available when turn_detection type is null. Confirmed by input_audio_buffer.committed from the server.

conversation.item.create

Create a new conversation item. Can be a user text message, an assistant text message for history seeding, a function call for seeding tool-use history, or a function call output.

input_audio_buffer.clear

Clear the input audio buffer. Use this to discard any pending audio data without committing it.

conversation.item.delete

Delete a conversation item by ID. The server confirms deletion with a conversation.item.deleted event.

response.create

Request the server to create a new assistant response. This is handled automatically when using server-side VAD.

response.cancel

Cancel an in-progress response. In VAD mode, interruptions are automatic — use this for manual cancel in non-VAD mode.

session.created

Sent automatically on WebSocket connection. Contains the session configuration.

conversation.created

The first message on connection. Notifies the client that a conversation session has been created.

session.updated

Acknowledges the client's session.update message that the session has been configured.

input_audio_buffer.speech_started

Notifies that the server's VAD detected the start of speech. Only available with server_vad turn detection.

input_audio_buffer.speech_stopped

Notifies that the server's VAD detected the end of speech. Only available with server_vad turn detection.

input_audio_buffer.committed

Input audio buffer has been committed as a user message.

input_audio_buffer.cleared

Confirms the input audio buffer has been cleared.

conversation.item.deleted

Confirms a conversation item has been deleted.

conversation.item.added

A new user or assistant message has been added to the conversation history.

conversation.item.input_audio_transcription.completed

Audio transcription for the user's input has been completed.

response.created

A new assistant response turn is in progress. Audio deltas from this turn share the same response_id.

response.output_item.added

A new assistant response item is added to the message history.

response.output_item.done

An output item is complete.

response.content_part.added

A content part starts within an output item.

response.content_part.done

A content part finishes.

response.output_audio_transcript.delta

Streaming text transcript delta of the assistant's audio response.

response.output_audio_transcript.done

The audio transcript for this assistant turn has finished generating.

response.output_audio.delta

Streaming base64-encoded audio delta of the assistant's response.

response.output_audio.done

Audio generation for this assistant turn has finished.

response.text.delta

Text-mode output delta (when using text modality).

response.function_call_arguments.delta

Streaming function call arguments.

response.function_call_arguments.done

A function call has been triggered with complete arguments. Your code should execute the function and return results via conversation.item.create with type function_call_output.

mcp_list_tools.in_progress

MCP tool discovery has started.

mcp_list_tools.completed

MCP tool discovery succeeded.

mcp_list_tools.failed

MCP tool discovery failed.

response.mcp_call_arguments.delta

MCP call arguments streaming.

response.mcp_call_arguments.done

MCP call arguments finalized.

response.mcp_call.in_progress

MCP server HTTP call starting.

response.mcp_call.completed

MCP tool execution succeeded.

response.mcp_call.failed

MCP tool execution failed.

response.done

The assistant's response is completed. Sent after all audio and transcript deltas. Ready for the client to add a new conversation item.

error

Sent when an error occurs. Contains error code and message. Most errors are recoverable and the session stays open.

text.delta

Send a chunk of text to be synthesized. Text is processed incrementally — audio generation begins as soon as enough text is buffered. Individual deltas are capped at 15,000 characters.

text.done

Signal that all text for this utterance has been sent. The server will finish generating audio and send audio.done. After receiving audio.done, you can start a new utterance with another text.delta.

audio.delta

A chunk of base64-encoded audio data. Decode and append to your audio buffer or pipe directly to playback. The format matches the codec and sample_rate specified in the query parameters.

audio.done

Audio generation for this utterance is complete. The connection remains open for multi-utterance — send another text.delta to start a new synthesis, or close the connection.

error

An error occurred during synthesis. The connection may be closed after this message.

Binary frame (audio)

Send raw audio as binary WebSocket frames in the encoding specified by the encoding query parameter. Audio should be streamed in real-time-paced chunks (e.g. 100 ms at a time). No base64 encoding — send raw bytes directly.

audio.done

Signal that all audio has been sent. The server flushes any remaining buffered audio, emits final transcript events, and sends a transcript.done event. After receiving transcript.done, the client can begin sending audio for a new utterance without reconnecting.

transcript.created

Sent immediately after the WebSocket connection is established and the server is ready to receive audio. **Wait for this event before sending audio** — the server needs to initialize its ASR backend.

transcript.partial

A transcript result for a portion of the audio stream. Uses two boolean fields — is_final and speech_final — to convey three states:
Interim (is_final=false, speech_final=false): Partial transcript — text may change. Sent every ~500 ms when interim_results=true.
Chunk final (is_final=true, speech_final=false): Chunk fully transcribed — text is locked. Emitted every ~3 seconds during continuous speech.
Utterance final (is_final=true, speech_final=true): Speaker stopped talking — VAD detected silence exceeding the endpointing threshold. Contains the complete stitched utterance.

transcript.done

Sent after the client sends audio.done. Signals the end of the session turn. If there is un-finalized audio remaining after the last speech_final event, text and words contain the stitched transcript for that audio. If a speech_final=true event already covered all the audio (common when the speaker paused before audio.done), text and words will be empty — the full transcript was already delivered in the preceding transcript.partial. The duration field always reflects the total audio processed. After this event, the server resets its pipeline and is ready for a new turn.

error

Sent when an error occurs during the session. The WebSocket remains open — the client can continue sending audio after receiving an error. Common causes: invalid JSON text frame, internal pipeline errors.