Skip to content

Speech-to-speech

Select a model for use cases like conversational speech and speech translation.

S2S (speech-to-speech) vs. pipeline

There are two approaches to building voice applications:

S2SPipeline (ASR + LLM + TTS)
LatencyLow -- single-model stream processingHigher -- three-stage serial processing
Audio understandingEnd-to-end -- perceives tone and emotion and responds accordinglyConverts to text before processing, losing subtle audio cues
Voice customizationSelection of preset voices via a system promptVoice cloning and voice design (CosyVoice)
  • Use S2S for low latency, audio-aware responses, and interactive conversation.

  • Use a pipeline when you need to customize voices or select best-in-class ASR, LLM, and TTS models for each stage.

Real-time or file mode?

  • Real-time (WebSocket) : For real-time voice interaction such as voice assistants, call centers, and simultaneous interpretation. Supports streaming audio input and speech output. Model names contain -realtime.

  • File mode (HTTP): Trades higher latency for better results, ideal for video dubbing, podcast translation, and offline content processing. Supports function calling (Qwen3.5-Omni, Qwen3-Omni-Flash), web search (Qwen3.5-Omni), thinking mode (Qwen3-Omni-Flash), and Livetranslate.

Function calling

Allows the model to perform actions---such as querying a knowledge base, checking a schedule, or triggering a workflow---based on audio and visual input. Supported by Qwen3.5-Omni (in WebSocket and HTTP modes) and Qwen3-Omni (in HTTP mode).

Not supported by real-time models or the Livetranslate model.

Allows the model to retrieve real-time information to answer questions about current events, stock prices, and weather. Qwen3.5-Omni-Plus and -Flash (in WebSocket and HTTP modes) support web search. The model autonomously decides whether to search.

Not supported by Qwen3-Omni-Flash and the Livetranslate model.

Thinking mode

Use Qwen3-Omni (HTTP mode) when answer quality outweighs latency. The model performs step-by-step reasoning before replying, ideal for video analysis and batch labeling.

Voice generation is not supported in thinking mode.

Speech translation

All three model series support speech translation:

  • Qwen3-Livetranslate: Supports 17 languages and 5 Chinese dialects with a latency of approximately 3 seconds. In file mode, it uses video input to provide more accurate context-aware translations. For 7 of these languages, the output is text-only (no speech).

  • Qwen3.5-Omni: Supports 29 output languages and 8 Chinese dialects. Offers audio and video understanding and web search. Use a system prompt to inject terminology and domain context. Supports both real-time and file modes.

  • Qwen3-Omni-Flash: Supports 11 output languages and 8 Chinese dialects. Use a system prompt to inject terminology and domain context. Supports both real-time and file modes. More cost-effective.

Note

We recommend Qwen3-Livetranslate for quickly building translation applications, Qwen3.5-Omni for the highest quality and broadest language coverage, and Qwen3-Omni-Flash for cost-sensitive scenarios. Supported languages

LanguageQwen3-LivetranslateQwen3.5-OmniQwen3-Omni-Flash
EnglishSupportedSupportedSupported
Chinese (Mandarin)SupportedSupportedSupported
CantoneseSupportedSupportedSupported
Sichuan dialectSupportedSupportedSupported
ShanghaineseSupportedSupportedSupported
Beijing dialectSupportedSupportedSupported
Tianjin dialectSupportedSupportedSupported
Nanjing dialect--SupportedSupported
Shaanxi dialect--SupportedSupported
Minnan dialect--SupportedSupported
FrenchSupportedSupportedSupported
GermanSupportedSupportedSupported
RussianSupportedSupportedSupported
ItalianSupportedSupportedSupported
SpanishSupportedSupportedSupported
PortugueseSupportedSupportedSupported
JapaneseSupportedSupportedSupported
KoreanSupportedSupportedSupported
ThaiText-onlySupportedSupported
IndonesianText-onlySupported--
VietnameseText-onlySupported--
ArabicText-onlySupported--
HindiText-onlySupported--
TurkishText-onlySupported--
Finnish--Supported--
Polish--Supported--
Dutch--Supported--
Czech--Supported--
Urdu--Supported--
Tagalog--Supported--
Swedish--Supported--
Danish--Supported--
Hebrew--Supported--
Icelandic--Supported--
Malay--Supported--
Norwegian--Supported--
Persian--Supported--
GreekText-only----

"Supported" means the model provides both speech and text output. "Text-only" means the model provides text output but no speech.

Qwen3.5-Omni supports 113 input languages and dialects.

The legacy qwen-omni-turbo model supports only Chinese and English.

ModelAPIInputFunction callingWeb searchThinking mode
qwen3.5-omni-plus-realtimeWebSockettext, audio, imageSupportedSupported--
qwen3.5-omni-plusHTTPtext, audio, image, videoSupportedSupported--
qwen3.5-omni-flash-realtimeWebSockettext, audio, imageSupportedSupported--
qwen3.5-omni-flashHTTPtext, audio, image, videoSupportedSupported--
qwen3-omni-flash-realtimeWebSockettext, audio, image, video------
qwen3-omni-flashHTTPtext, audio, image, videoSupported--Supported
qwen3-livetranslate-flash-realtimeWebSocketaudio, image------
qwen3-livetranslate-flashHTTPaudio, video------

All models

Qwen3.5-Omni

ModelAPIInputFunction callingWeb searchThinking mode
qwen3.5-omni-plus-realtimeWebSocketText, audio, image, videoSupportedSupported--
qwen3.5-omni-plus-realtime-2026-03-15WebSocketText, audio, image, videoSupportedSupported--
qwen3.5-omni-plusHTTPText, audio, image, videoSupportedSupported--
qwen3.5-omni-plus-2026-03-15HTTPText, audio, image, videoSupportedSupported--
qwen3.5-omni-flash-realtimeWebSocketText, audio, image, videoSupportedSupported--
qwen3.5-omni-flash-realtime-2026-03-15WebSocketText, audio, image, videoSupportedSupported--
qwen3.5-omni-flashHTTPText, audio, image, videoSupportedSupported--
qwen3.5-omni-flash-2026-03-15HTTPText, audio, image, videoSupportedSupported--

Qwen3-Omni

ModelAPIInputFunction callingWeb searchThinking mode
qwen3-omni-flash-realtimeWebSocketText, audio, image, video------
qwen3-omni-flash-realtime-2025-12-01WebSocketText, audio, image, video------
qwen3-omni-flash-realtime-2025-09-15WebSocketText, audio, image, video------
qwen3-omni-flashHTTPText, audio, image, videoSupported--Supported
qwen3-omni-flash-2025-12-01HTTPText, audio, image, videoSupported--Supported
qwen3-omni-flash-2025-09-15HTTPText, audio, image, videoSupported--Supported

Qwen3-Livetranslate

ModelAPIInputLanguages
qwen3-livetranslate-flash-realtimeWebSocketAudio18
qwen3-livetranslate-flash-realtime-2025-09-22WebSocketAudio18
qwen3-livetranslate-flashHTTPAudio, video18
qwen3-livetranslate-flash-2025-12-01HTTPAudio, video18

Legacy models

These models are no longer updated. For new projects, use Qwen3.5-Omni.

ModelInputAPI
qwen2.5-omni-7bText, audio, image, videoHTTP
qwen-omni-turboText, audio, image, videoHTTP
qwen-omni-turbo-latestText, audio, image, videoHTTP
qwen-omni-turbo-2025-03-26Text, audio, image, videoHTTP
qwen-omni-turbo-realtimeText, audioWebSocket
qwen-omni-turbo-realtime-latestText, audioWebSocket
qwen-omni-turbo-realtime-2025-05-08Text, audioWebSocket

Mirror of Alibaba Cloud Model Studio docs for reference and RAG. Not affiliated with Alibaba Cloud.