time multimodal-Client events

Client events for the Qwen-Omni-Realtime API.

See Real-time (Qwen-Omni-Realtime).

session.update

After establishing a WebSocket connection, send this event first to update the default configurations for the session. The server validates the parameters upon receiving the session.update event. If the parameters are invalid, the server returns an error. If the parameters are valid, the server updates the configurations and returns the complete configuration. type *string* (required) The event type. The value must be session.update.

{
 "event_id": "event_ToPZqeobitzUJnt3QqtWg",
 "type": "session.update",
 "session": {
 "modalities": \[
 "text",
 "audio"
 \],
 "voice": "Chelsie",
 "input_audio_format": "pcm",
 "output_audio_format": "pcm",
 "instructions": "You are an AI customer service specialist for a five-star hotel. Accurately and friendly answer customer inquiries about room types, facilities, prices, and booking policies. Always respond with a professional and helpful attitude. Do not provide unverified information or information outside the scope of hotel services.",
 "turn_detection": {
 "type": "server_vad",
 "threshold": 0.5,
 "silence_duration_ms": 800
 },
 "enable_search": true,
 "search_options": {
 "enable_source": true
 },
 "tools": \[
 {
 "type": "function",
 "function": {
 "name": "get_current_weather",
 "description": "Useful for querying the weather in a specific city.",
 "parameters": {
 "type": "object",
 "properties": {
 "location": {
 "type": "string",
 "description": "The city or district, such as Beijing, Hangzhou, or Yuhang District."
 }
 },
 "required": \["location"\]
 }
 }
 }
 \],
 "seed": 1314,
 "max_tokens": 16384,
 "repetition_penalty": 1.05,
 "presence_penalty": 0.0,
 "top_k": 50,
 "top_p": 1.0,
 "temperature": 0.9
 }
}

      **session **`*object*`* *(optional) The session configuration.

Properties

**modalities **array* *(optional) The model output modality settings. Valid values:

• ["text"] Outputs text only.
• ["text","audio"] (default) Outputs text and audio.

**voice **string* *(optional) The voice for the audio generated by the model. For supported voices, see Voice list. Default voices:

• Qwen3.5-Omni-Realtime series: Tina
• Qwen3-Omni-Flash-Realtime: Cherry
• Qwen-Omni-Turbo-Realtime: Chelsie

**input_audio_format **string* *(optional) The format of the user's input audio. Currently, only pcm is supported.

**output_audio_format **string* *(optional) The format of the model's output audio. Currently, only pcm is supported.

**smooth_output **boolean \| null* *(optional) This parameter is effective only for Qwen3-Omni-Flash-Realtime series models. Whether to enable a colloquial response style. Valid values:

• true (default): Enables colloquial responses.
• false: Enables more formal, written responses. ** Content that is difficult to read aloud may not perform well.
• null: The model automatically selects a colloquial or formal response style.

<b>instructions **string* *(optional) The system message, which sets the model's objective or role.

**turn_detection **object* *(optional) The configuration for Voice Activity Detection (VAD). Set this to null to disable VAD and trigger model responses manually. If you do not provide this field, VAD is enabled with the default parameters. Properties

**type **string* *(optional) The VAD type. Valid values:

• server_vad (default): Detects the end of user speech based on acoustic features.
• semantic_vad: Detects the end of user speech based on semantic validity. This filters out meaningless speech, such as backchannels and background noise. This option is supported only by the qwen3.5-omni-realtime model.

**threshold **float* *(optional) The sensitivity of VAD. A lower value makes VAD more sensitive and more likely to recognize faint sounds as speech, including background noise. A higher value makes VAD less sensitive and requires clearer and louder speech to trigger. The value must be in the range of \[-1.0, 1.0\]. The default value is 0.5.

**silence_duration_ms **integer* *(optional) The minimum duration of silence required after speech ends. A timeout triggers a model response. A lower value results in a faster response but may cause false triggers during short pauses in speech. The default value is 800. The value must be in the range of [200, 6000].

**enable_search **boolean* *(optional) This parameter is effective only for the Qwen3.5-Omni-Realtime model. Whether to enable web search. Default: false. When enabled, the model can autonomously determine whether a search is needed to answer the user's real-time questions. ** tools and enable_search are incompatible. Do not enable both at the same time.

<b>search_options **object* *(optional) The web search option configuration. This takes effect only after you enable enable_search. Properties

**enable_source **boolean* *(optional) Whether to return a list of search result sources. Set to true to enable.

**tools **array* *(optional) A list of tool definitions. When configured, the model can decide whether to call a tool based on user input. Properties

**type ***string* (required) The value must be function.

**function.name ***string* (required) The name of the custom tool function. Use the same name as the function, such as get_current_weather or get_current_time.

**function.description ***string* (optional) A description of the tool function's capabilities. The model uses this field to decide whether to use the tool function.

**function.parameters ***object* (optional) A description of the tool function's input parameters. The model uses this field to extract the input parameters. If the tool function does not require input parameters, do not specify this field. Properties

**type ***string* (required) The value must be object.

**properties ***object* (optional) Describes the name, data type, and description of each input parameter. The key is the parameter name. The value is an object that contains the type and description.

**required ***array* (optional) Specifies which input parameters are required.

 **temperature **`*float*`* *(optional) The sampling temperature, which controls the diversity of the content generated by the model. A higher temperature results in more diverse content. A lower temperature results in more deterministic content. The value must be in the range of \[0, 2). Because both temperature and top_p control content diversity, set only one of them. Default value:

• qwen3.5-omni-realtime series: 0.7

• qwen3-omni-flash-realtime series: 0.9

• qwen-omni-turbo-realtime series: 1.0 ** qwen-omni-turbo models <b>do not support modification**. **top_p **float* *(optional) The probability threshold for nucleus sampling, which controls the diversity of the content generated by the model. A higher top_p results in more diverse content. A lower top_p results in more deterministic content. The value must be in the range of (0, 1.0]. Because both temperature and top_p control content diversity, set only one of them. Default value:

• qwen3.5-omni-realtime series: 0.8

• qwen3-omni-flash-realtime series: 1.0

• qwen-omni-turbo-realtime series: 0.01 ** qwen-omni-turbo models <b>do not support modification**. **top_k **integer* *(optional) The size of the candidate set for sampling during the generation process. For example, if you set this parameter to 50, only the 50 tokens with the highest scores in a single generation are used to form the candidate set for random sampling. A larger value increases the randomness of the generated content. A smaller value increases the determinism of the generated content. If you set this parameter to null or a value greater than 100, the top_k policy is not used. In this case, only the top_p policy is effective. The value must be greater than or equal to 0. Default value:

• qwen3.5-omni-realtime series: 20

• qwen3-omni-flash-realtime series: 50

• qwen-omni-turbo-realtime series: 20 ** qwen-omni-turbo models <b>do not support modification**. **max_tokens *integer* (optional) The maximum number of tokens to return for this request. ** The max_tokens setting does not affect the model's generation process. If the number of tokens generated by the model exceeds max_tokens, the response returns truncated content. The default value and the maximum value are both the maximum output length of the model. See Model list. The max_tokens parameter is suitable for scenarios where you need to limit the word count, such as generating summaries or keywords, controlling costs, or reducing response time. qwen-omni-turbo models <b>do not support modification. **repetition_penalty **float* *(optional) Controls the degree of repetition in consecutive sequences during model generation. A higher repetition_penalty value reduces the repetition of the generated content. A value of 1.0 indicates no penalty. The value must be greater than 0. repetition_penalty default value:

• qwen3.5-omni-realtime series: 1.0

• qwen3-omni-flash-realtime series: 1.05

• qwen-omni-turbo-realtime series: 1.05 ** qwen-omni-turbo models <b>do not support modification**. presence_penalty *float** *(optional) Controls the degree of repetition in the content generated by the model. Valid range: [-2.0, 2.0]. A positive value reduces repetition. A negative value increases repetition. presence_penalty default value:

• qwen3.5-omni-realtime series: 1.5

• qwen3-omni-flash-realtime series: 0.0

• qwen-omni-turbo-realtime series: 0.0 Scenarios: A higher presence_penalty is suitable for scenarios that require diversity, interest, or creativity, such as creative writing or brainstorming. A lower presence_penalty is suitable for scenarios that require consistency or specialized terminology, such as technical documents or other formal documents. ** qwen-omni-turbo models <b>do not support modification**. **seed *integer* (optional) Setting the seed parameter makes the model generation process more deterministic. This is typically used to ensure that the model's results are consistent across runs. If you pass the same seed value in each model call and keep other parameters unchanged, the model returns the same result. The value must be in the range of 0 to 231−1. The default value is -1. ** qwen-omni-turbo models <b>do not support modification.

response.create

The response.create event instructs the server to create a model response. In VAD mode, the server automatically creates responses, so you do not need to send this event. In a tool calling scenario, after you return the tool result using conversation.item.create, send this event to trigger the final model response.

The server responds with a response.created event, one or more item and content events such as conversation.item.created and response.content_part.added, and finally a response.done event to indicate that the response is complete. **type **string* *(required) The event type. The value must be response.create.

{
 "type": "response.create",
 "event_id": "event_1718624400000"
}

response.cancel

The client sends this event to cancel an ongoing response. If no response is in progress, the server responds with an error event. *type *string* *** (required) The event type. The value must be response.cancel.

{
 "event_id": "event_B4o9RHSTWobB5OQdEHLTo",
 "type": "response.cancel"
}

input_audio_buffer.append

Appends audio bytes to the input audio buffer. *type *string* *** (required) The event type. The value must be input_audio_buffer.append.

{
 "event_id": "event_B4o9RHSTWobB5OQdEHLTo",
 "type": "input_audio_buffer.append",
 "audio": "UklGR..."
}

     **audio **`*string*`* *** (required)** The Base64-encoded audio data.

input_audio_buffer.commit

Commits the user input audio buffer. This creates a new user message item in the conversation. If the buffer is empty, the server returns an error event.

• VAD mode: The client does not need to send this event. The server automatically commits the audio buffer.

• Manual mode: The client must commit the audio buffer to create a user message item.

Committing the input audio buffer does not create a response from the model. The server responds with an input_audio_buffer.committed event.

If the client sent an input_image_buffer.append event, the input_audio_buffer.commit event also commits the image buffer. *type *string* *** (required) The event type. The value must be input_audio_buffer.commit.

{
 "event_id": "event_B4o9RHSTWobB5OQdEHLTo",
 "type": "input_audio_buffer.commit"
}

input_audio_buffer.clear

Clears audio bytes from the buffer. The server responds with an input_audio_buffer.cleared event. *type *string* *** (required) The event type. The value must be input_audio_buffer.clear.

{
 "event_id": "event_xxx",
 "type": "input_audio_buffer.clear"
}

input_image_buffer.append

Adds image data to the image buffer. Images can come from local files or be captured in real time from a video stream.

The following limits apply to image inputs:

• The image format must be JPG or JPEG. For optimal performance, a resolution of 480p or 720p is recommended, with a maximum of 1080p.

• The size of a single image cannot exceed 500 KB before Base64 encoding.

• The image data must be Base64-encoded.

• Send images to the server at a recommended frequency of 1 image per second.

• Before you send an input_image_buffer.append event, you must send at least one input_audio_buffer.append event.

The image buffer is committed with the audio buffer using the input_audio_buffer.commit event. *type *string* *** (required) The event type. The value must be input_image_buffer.append.

{
 "event_id": "event_xxx",
 "type": "input_image_buffer.append",
 "image": "xxx"
}

     **image **`*string*`* *** (required)** The Base64-encoded image data.

conversation.item.create

The client sends this event to return the execution result of a tool function to the server. After the model triggers a tool call, you must execute the tool function locally. Then, send the result back using this event. Finally, send a response.create event to trigger the model to generate the final response. Note

Currently, only items of the function_call_output type are supported. type *string* (required) The event type. The value must be conversation.item.create.

{
 "event_id": "event_55099cddb51b4f208cb95d1a994eef80",
 "type": "conversation.item.create",
 "item": {
 "id": "item_2a80d7682b4e473c9c2154da135041e9",
 "type": "function_call_output",
 "call_id": "call_62c24725afdb4c2680ac54",
 "output": "The weather in Beijing today is changing from haze to clear, with a temperature of 4/-4°C and a light breeze."
 }
}

     **item **`*object*`** (required)** The conversation item to create. This cannot be empty.

Properties

**id ***string* (optional) The conversation item ID. The client can specify an ID to align with the local state. If you do not provide an ID, the server generates one.

**type ***string* (required) The conversation item type. Currently, only function_call_output is supported.

**call_id ***string* (required) Corresponds to the call_id returned in the response.function_call_arguments.done event.

**output ***string* (required) The execution result of the tool function.