time speech recognition API reference-Python SDK

The parameters and interfaces of the Paraformer real-time speech recognition Python SDK. Important

This document applies only to the Chinese mainland (Beijing) region. To use the models, you must use an API key from the Chinese mainland (Beijing) region. User guide: For model descriptions and selection guidance, see Real-time speech recognition - Fun-ASR/Paraformer.

Prerequisites

• You have activated the Model Studio and created an API key. Export it as an environment variable (not hard-coded) to prevent security risks. Note

For temporary access or strict control over high-risk operations (accessing/deleting sensitive data), use a temporary authentication token instead.

Compared with long-term API keys, temporary tokens are more secure (60-second lifespan) and reduce API key leakage risk.

To use a temporary token, replace the API key used for authentication in your code with the temporary authentication token.

• Install the latest version of the DashScope SDK.

Model list

	paraformer-realtime-v2	paraformer-realtime-8k-v2
Scenarios	Scenarios such as live streaming and meetings	Recognition scenarios for 8 kHz audio, such as telephone customer service and voicemail
Sample rate	Any	8 kHz
Language	Chinese (including Mandarin and various dialects), English, Japanese, Korean, German, French, and Russian Supported Chinese dialects: Shanghainese, Wu, Minnan, Northeastern, Gansu, Guizhou, Henan, Hubei, Hunan, Jiangxi, Ningxia, Shanxi, Shaanxi, Shandong, Sichuan, Tianjin, Yunnan, and Cantonese	Chinese
Punctuation prediction	✅ Supported by default. No configuration is required.	✅ Supported by default. No configuration is required.
Inverse Text Normalization (ITN)	✅ Supported by default. No configuration is required.	✅ Supported by default. No configuration is required.
Custom vocabulary	✅\ See Customize hotwords	✅\ See Customize hotwords
Specify recognition language	✅\ Specify the language using the language_hints parameter.	❌
Emotion recognition	❌	✅ (Click to view usage) Emotion recognition has the following constraints: - Applies only to the paraformer-realtime-8k-v2 model. - You must disable semantic punctuation (controlled by the request parameter semantic_punctuation_enabled). Semantic punctuation is disabled by default. - The emotion recognition result is displayed only when the is_sentence_end method of RecognitionResult returns True. To obtain the emotion detection results, retrieve the emotion and the emotion confidence level of the current sentence from the emo_tag and emo_confidence fields of the single-sentence information (Sentence) , respectively.

Getting started

The Recognition class provides methods for non-streaming and bidirectional streaming calls. You can select the appropriate method based on your requirements:

• Non-streaming call: Recognizes a local file and returns the complete result at once. This is suitable for processing pre-recorded audio.

• Bidirectional streaming call: Recognizes an audio stream and outputs the results in real time. The audio stream can come from an external device, such as a microphone, or be read from a local file. This is suitable for scenarios that require immediate feedback.

Non-streaming call

This method submits a real-time speech-to-text task for a local file. The process is blocked until the complete transcription result is returned.

Instantiate the Recognition class, set the request parameters, and call the call method to perform recognition or translation and obtain the RecognitionResult. Click to view the complete example The audio file used in the example is: asr_example.wav.

HELPCODEESCAPE-python
from http import HTTPStatus
from dashscope.audio.asr import Recognition


# import dashscope
# dashscope.api_key = "apiKey"

recognition = Recognition(model='paraformer-realtime-v2',
                          format='wav',
                          sample_rate=16000,
                          # The "language_hints" parameter is supported only by the paraformer-realtime-v2 model.
                          language_hints=['zh', 'en'],
                          callback=None)
result = recognition.call('asr_example.wav')
if result.status_code == HTTPStatus.OK:
    print('Recognition result:')
    print(result.get_sentence())
else:
    print('Error: ', result.message)

print(
    '[Metric] requestId: {}, first package delay ms: {}, last package delay ms: {}'
    .format(
        recognition.get_last_request_id(),
        recognition.get_first_package_delay(),
        recognition.get_last_package_delay(),
    ))

Bidirectional streaming call

This method submits a real-time speech-to-text task and returns real-time recognition results through a callback interface.

1. Start streaming speech recognition

   Instantiate the Recognition class, bind the request parameters and the callback interface (RecognitionCallback), and call the start method to start streaming speech recognition.

2. Streaming

   Repeatedly call the Recognition class's send_audio_frame method to send the binary audio stream from a local file or a device (such as a microphone) to the server in segments.

   As audio data is sent, the server uses the RecognitionCallback callback interface's on_event method to return the recognition results to the client in real time.

   We recommend that the duration of each audio segment sent is about 100 milliseconds, and the data size is between 1 KB and 16 KB.

3. End processing

   Call the stop method of the Recognition class to stop speech recognition.

   This method blocks the current thread until the on_complete or on_error callback of the callback interface (RecognitionCallback) is triggered.

Click to view the complete example

Recognize speech from a microphone

HELPCODEESCAPE-python
import os
import signal  # for keyboard events handling (press "Ctrl+C" to terminate recording)
import sys

import dashscope
import pyaudio
from dashscope.audio.asr import *

mic = None
stream = None

# Set recording parameters
sample_rate = 16000  # sampling rate (Hz)
channels = 1  # mono channel
dtype = 'int16'  # data type
format_pcm = 'pcm'  # the format of the audio data
block_size = 3200  # number of frames per buffer


def init_dashscope_api_key():
    """
        Set your DashScope API-key. More information:
        https://github.com/aliyun/alibabacloud-bailian-speech-demo/blob/master/PREREQUISITES.md
    """

    if 'DASHSCOPE_API_KEY' in os.environ:
        dashscope.api_key = os.environ[
            'DASHSCOPE_API_KEY']  # load API-key from environment variable DASHSCOPE_API_KEY
    else:
        dashscope.api_key = '<your-dashscope-api-key>'  # set API-key manually


# Real-time speech recognition callback
class Callback(RecognitionCallback):
    def on_open(self) -> None:
        global mic
        global stream
        print('RecognitionCallback open.')
        mic = pyaudio.PyAudio()
        stream = mic.open(format=pyaudio.paInt16,
                          channels=1,
                          rate=16000,
                          input=True)

    def on_close(self) -> None:
        global mic
        global stream
        print('RecognitionCallback close.')
        stream.stop_stream()
        stream.close()
        mic.terminate()
        stream = None
        mic = None

    def on_complete(self) -> None:
        print('RecognitionCallback completed.')  # recognition completed

    def on_error(self, message) -> None:
        print('RecognitionCallback task_id: ', message.request_id)
        print('RecognitionCallback error: ', message.message)
        # Stop and close the audio stream if it is running
        if 'stream' in globals() and stream.active:
            stream.stop()
            stream.close()
        # Forcefully exit the program
        sys.exit(1)

    def on_event(self, result: RecognitionResult) -> None:
        sentence = result.get_sentence()
        if 'text' in sentence:
            print('RecognitionCallback text: ', sentence['text'])
            if RecognitionResult.is_sentence_end(sentence):
                print(
                    'RecognitionCallback sentence end, request_id:%s, usage:%s'
                    % (result.get_request_id(), result.get_usage(sentence)))


def signal_handler(sig, frame):
    print('Ctrl+C pressed, stop recognition ...')
    # Stop recognition
    recognition.stop()
    print('Recognition stopped.')
    print(
        '[Metric] requestId: {}, first package delay ms: {}, last package delay ms: {}'
        .format(
            recognition.get_last_request_id(),
            recognition.get_first_package_delay(),
            recognition.get_last_package_delay(),
        ))
    # Forcefully exit the program
    sys.exit(0)


# main function
if __name__ == '__main__':
    init_dashscope_api_key()
    print('Initializing ...')

    # Create the recognition callback
    callback = Callback()

    # Call recognition service by async mode, you can customize the recognition parameters, like model, format,
    # sample_rate
    recognition = Recognition(
        model='paraformer-realtime-v2',
        format=format_pcm,
        # 'pcm', 'wav', 'opus', 'speex', 'aac', or 'amr'. You can check the supported formats in the document.
        sample_rate=sample_rate,
        # 8000 or 16000 is supported.
        semantic_punctuation_enabled=False,
        callback=callback)

    # Start recognition
    recognition.start()

    signal.signal(signal.SIGINT, signal_handler)
    print("Press 'Ctrl+C' to stop recording and recognition...")
    # Create a keyboard listener until "Ctrl+C" is pressed

    while True:
        if stream:
            data = stream.read(3200, exception_on_overflow=False)
            recognition.send_audio_frame(data)
        else:
            break

    recognition.stop()

Recognize a local audio file

The audio file used in the example is: asr_example.wav.

HELPCODEESCAPE-python
import os
import time
from dashscope.audio.asr import *

# If you have not configured the API key in the environment variable, uncomment the following line of code and replace apiKey with your API key.
# import dashscope
# dashscope.api_key = "apiKey"

from datetime import datetime

def get_timestamp():
    now = datetime.now()
    formatted_timestamp = now.strftime("[%Y-%m-%d %H:%M:%S.%f]")
    return formatted_timestamp

class Callback(RecognitionCallback):
    def on_complete(self) -> None:
        print(get_timestamp() + ' Recognition completed')  # recognition complete

    def on_error(self, result: RecognitionResult) -> None:
        print('Recognition task_id: ', result.request_id)
        print('Recognition error: ', result.message)
        exit(0)

    def on_event(self, result: RecognitionResult) -> None:
        sentence = result.get_sentence()
        if 'text' in sentence:
            print(get_timestamp() + ' RecognitionCallback text: ', sentence['text'])
            if RecognitionResult.is_sentence_end(sentence):
                print(get_timestamp() +
                    'RecognitionCallback sentence end, request_id:%s, usage:%s'
                    % (result.get_request_id(), result.get_usage(sentence)))


callback = Callback()

recognition = Recognition(model='paraformer-realtime-v2',
                          format='wav',
                          sample_rate=16000,
                          # The "language_hints" parameter is supported only by the paraformer-realtime-v2 model.
                          language_hints=['zh', 'en'],
                          callback=callback)

recognition.start()

try:
    audio_data: bytes = None
    f = open("asr_example.wav", 'rb')
    if os.path.getsize("asr_example.wav"):
        while True:
            audio_data = f.read(3200)
            if not audio_data:
                break
            else:
                recognition.send_audio_frame(audio_data)
            time.sleep(0.1)
    else:
        raise Exception(
            'The supplied file was empty (zero bytes long)')
    f.close()
except Exception as e:
    raise e

recognition.stop()

print(
    '[Metric] requestId: {}, first package delay ms: {}, last package delay ms: {}'
    .format(
        recognition.get_last_request_id(),
        recognition.get_first_package_delay(),
        recognition.get_last_package_delay(),
    ))

Concurrent calls

In Python, because of the Global Interpreter Lock (GIL), only one thread can execute Python code at a time (although some performance-oriented libraries may remove this limitation). If you want to better utilize the computing resources of a multi-core computer, we recommend that you use multiprocessing or concurrent.futures.ProcessPoolExecutor. Multi-threading can significantly increase SDK call latency under high concurrency.

Request parameters

Request parameters are set in the constructor (init) of the Recognition class.

Parameter	Type	Default	Required	Description
model	str	-	Yes	The model used for real-time speech recognition. For more information, see Model List .
sample_rate	int	-	Yes	The audio sampling rate in Hz. This parameter varies by model: - paraformer-realtime-v2 supports any sample rate. - paraformer-realtime-8k-v2 supports only an 8000 Hz sample rate.
format	str	-	Yes	The format of the audio to be recognized. Supported audio formats: pcm, wav, mp3, opus, speex, aac, and amr. ** **Important ** opus/speex: Must be encapsulated in Ogg. wav: Must be PCM encoded. amr: Only the AMR-NB type is supported.
vocabulary_id	str	-	No	The ID of the hotword vocabulary. This parameter takes effect only when it is set. Use this field to set the hotword ID for v2 and later models. The hotword information for this hotword ID is applied to the speech recognition request. For more information, see Custom hotwords.
disfluency_removal_enabled	bool	False	No	Specifies whether to filter out disfluent words: - true - false (default)
language_hints	list[str]	["zh", "en"]	No	The language code for recognition. If you cannot determine the language in advance, leave this parameter unset for automatic detection. Currently supported language codes: - zh: Chinese - en: English - ja: Japanese - yue: Cantonese - ko: Korean - de: German - fr: French - ru: Russian This parameter applies only to multilingual models. For more information, see Model list .
semantic_punctuation_enabled	bool	False	No	Specifies whether to enable semantic sentence segmentation (disabled by default): - true: Uses semantic segmentation (disables VAD segmentation). - false (default): Uses VAD segmentation. Semantic segmentation provides higher accuracy and is ideal for meeting transcription. VAD segmentation has lower latency and is ideal for interactive scenarios. Applies to v2 and later models.
max_sentence_silence	int	800	No	The VAD sentence segmentation silence threshold (ms). If silence after a speech segment exceeds this value, the sentence ends. Range: 200-6000 ms. Default: 800 ms. Applies only when semantic_punctuation_enabled is false (VAD mode) and model is v2 or later.
multi_threshold_mode_enabled	bool	False	No	Specifies whether to prevent VAD from over-segmenting long sentences (disabled by default). Applies only when semantic_punctuation_enabled is false (VAD mode) and model is v2 or later.
punctuation_prediction_enabled	bool	True	No	Specifies whether to automatically add punctuation to results (enabled by default): - true (default) - false Applies to v2 and later models only.
heartbeat	bool	False	No	Specifies whether to maintain a persistent server connection: - true: Keeps connection alive when sending silent audio continuously. - false (default): Connection times out after 60s even with silent audio. Silent audio: audio with no sound signal. Generate it with editing software (Audacity, Adobe Audition) or FFmpeg. Applies to v2 and later models only. When using this field, the SDK version must be 1.23.1 or later.
inverse_text_normalization_enabled	bool	True	No	Specifies whether to enable Inverse Text Normalization (ITN). When enabled, Chinese numerals are converted to Arabic numerals (enabled by default). Applies to v2 and later models only.
callback	RecognitionCallback	-	No	RecognitionCallback interface .

Key interfaces

Recognition class

The Recognition class is imported using from dashscope.audio.asr import *. Member method Method signature Description call

def call(self, file: str, phrase_id: str = None, **kwargs) -\> RecognitionResult

A non-streaming call that uses a local file. This method blocks the current thread until the entire audio file is read. The file must have read permissions. The recognition result is returned as a RecognitionResult type. start

def start(self, phrase_id: str = None, **kwargs)

Starts speech recognition. This is a callback-based streaming real-time recognition method that does not block the current thread. It must be used with send_audio_frame and stop. send_audio_frame

def send_audio_frame(self, buffer: bytes)

Pushes an audio stream. The audio stream pushed each time should not be too large or too small. We recommend that each audio packet has a duration of about 100 ms and a size between 1 KB and 16 KB. You can obtain the recognition results through the on_event method of the callback interface (RecognitionCallback). stop

def stop(self)

Stops speech recognition. This method blocks until the service has recognized all received audio and the task is complete. get_last_request_id

def get_last_request_id(self)

Gets the request_id. This can be used after the constructor is called (the object is created). get_first_package_delay

def get_first_package_delay(self)

Gets the first packet delay, which is the latency from sending the first audio packet to receiving the first recognition result packet. Use this after the task is completed. get_last_package_delay

def get_last_package_delay(self)

Obtains the last packet delay, which is the time taken from sending the stop instruction to receiving the last recognition result packet. Use this after the task is completed.

Callback interface (RecognitionCallback)

During a bidirectional streaming call, the server uses callbacks to return key process information and data to the client. You must implement a callback method to process the returned information and data. Click to view the example

HELPCODEESCAPE-python
class Callback(RecognitionCallback):
    def on_open(self) -> None:
        print('Connection successful')

    def on_event(self, result: RecognitionResult) -> None:
        # Implement the logic for receiving recognition results

    def on_complete(self) -> None:
        print('Task completed')

    def on_error(self, result: RecognitionResult) -> None:
        print('An exception occurred: ', result)

    def on_close(self) -> None:
        print('Connection closed')


callback = Callback()

**Method**   **Parameter**   **Return value**   **Description**

def on_open(self) -\> None

None None This method is called immediately after a connection is established with the server.

def on_event(self, result: RecognitionResult) -\> None

result: RecognitionResult None This method is called when the service sends a response.

def on_complete(self) -\> None

None None This method is called after all recognition results have been returned.

def on_error(self, result: RecognitionResult) -\> None

result: Recognition result None This method is called when an exception occurs.

def on_close(self) -\> None

None None This method is called after the service has closed the connection.

Response results

Recognition result (RecognitionResult)

RecognitionResult represents the recognition result of either a single real-time recognition in a bidirectional streaming call or a non-streaming call. Member method Method signature Description get_sentence

def get_sentence(self) -\> Union\[Dict\[str, Any\], List\[Any\]\]

Gets the currently recognized sentence and timestamp information. In a callback, a single sentence is returned, so this method returns a Dict[str, Any] type. For more information, see Sentence. get_request_id

def get_request_id(self) -\> str

Gets the request_id of the request. is_sentence_end

@staticmethod
def is_sentence_end(sentence: Dict\[str, Any\]) -\> bool

Determines whether the given sentence has ended.

Sentence (Sentence)

The members of the Sentence class are as follows:

Parameter	Type	Description
begin_time	int	The start time of the sentence, in ms.
end_time	int	The end time of the sentence, in ms.
text	str	The recognized text.
words	A list of Word timestamp information (Word)	Word timestamp information.
emo_tag	str	The emotion of the current sentence: - positive: Positive emotion, such as happy or satisfied - negative: Negative emotion, such as angry or sad - neutral: No obvious emotion Emotion recognition has the following constraints: - Applies only to the paraformer-realtime-8k-v2 model. - You must disable semantic punctuation (controlled by the request parameter semantic_punctuation_enabled). Semantic punctuation is disabled by default. - The emotion recognition result is displayed only when the is_sentence_end method of RecognitionResult returns True.
emo_confidence	float	The confidence level of the recognized emotion for the current sentence. The value ranges from 0.0 to 1.0. A larger value indicates a higher confidence level. Emotion recognition has the following constraints: - Applies only to the paraformer-realtime-8k-v2 model. - You must disable semantic punctuation (controlled by the request parameter semantic_punctuation_enabled). Semantic punctuation is disabled by default. - The emotion recognition result is displayed only when the is_sentence_end method of RecognitionResult returns True.

Word timestamp information (Word)

The members of the Word class are as follows:

Parameter	Type	Description
begin_time	int	The start time of the word, in ms.
end_time	int	The end time of the word, in ms.
text	str	The word.
punctuation	str	The punctuation.

Error codes

If an error occurs, see Error messages for troubleshooting.

If the problem persists, join the developer group to report the issue. Provide the Request ID to help us investigate the issue.

More examples

For more examples, see GitHub.

FAQ

Features

Q: How to maintain a persistent connection with the server during long periods of silence?

Set heartbeat parameter to true and send silent audio continuously. Silent audio: audio with no sound signal. Generate it with editing software (Audacity, Adobe Audition) or FFmpeg.

Q: How to convert an audio format to the required format?

You can use the FFmpeg tool. For more information, see the official FFmpeg website.

HELPCODEESCAPE-bash
# Basic conversion command (universal template)
# -i: Specifies the input file path. Example: audio.wav
# -c:a: Specifies the audio encoder. Examples: aac, libmp3lame, pcm_s16le
# -b:a: Specifies the bit rate (controls audio quality). Examples: 192k, 320k
# -ar: Specifies the sample rate. Examples: 44100 (CD), 48000, 16000
# -ac: Specifies the number of sound channels. Examples: 1 (mono), 2 (stereo)
# -y: Overwrites an existing file (no value needed).
ffmpeg -i input_audio.ext -c:a encoder_name -b:a bit_rate -ar sample_rate -ac num_channels output.ext

# Example: WAV to MP3 (maintain original quality)
ffmpeg -i input.wav -c:a libmp3lame -q:a 0 output.mp3
# Example: MP3 to WAV (16-bit PCM standard format)
ffmpeg -i input.mp3 -c:a pcm_s16le -ar 44100 -ac 2 output.wav
# Example: M4A to AAC (extract/convert Apple audio)
ffmpeg -i input.m4a -c:a copy output.aac  # Directly extract without re-encoding
ffmpeg -i input.m4a -c:a aac -b:a 256k output.aac  # Re-encode to improve quality
# Example: FLAC lossless to Opus (high compression)
ffmpeg -i input.flac -c:a libopus -b:a 128k -vbr on output.opus

Q: Can I view the time range for each sentence?

Yes. Results include start/end timestamps for each sentence to determine time ranges.

Q: How do I recognize a local file (recorded audio file)?

There are two ways to recognize a local file:

• Directly pass the local file path: This method returns the complete recognition result after the file is fully processed. It is not suitable for scenarios that require immediate feedback.

  Pass the file path to the call method of the Recognition class to directly recognize the audio file. For more information, see Non-streaming call.

• Convert the local file into a binary stream for recognition: This method returns recognition results as a stream while the file is being processed. It is suitable for scenarios that require immediate feedback.

  You can use the send_audio_frame method of the Recognition class to send a binary stream to the server for recognition. For more information, see bidirectional streaming call.

Troubleshooting

Q: Why there is no recognition result?

1. Verify audio format and sampleRate/sample_rate match parameter constraints. Common errors:

   • The audio file has a .wav extension but is in MP3 format, and the format parameter is incorrectly set to `mp3`.

   • The audio sample rate is 3600 Hz, but the sampleRate/sample_rate parameter is incorrectly set to 48000.

   Use ffprobe to check audio info (container, encoding, sample rate, channels):

   HELPCODEESCAPE-sh
   ffprobe -v error -show_entries format=format_name -show_entries stream=codec_name,sample_rate,channels -of default=noprint_wrappers=1 input.xxx

2. When you use the paraformer-realtime-v2 model, check whether the language set in language_hints matches the actual language of the audio.

   For example, the audio is in Chinese, but language_hints is set to en (English).

3. If all the preceding checks pass, you can use custom hotwords to improve the recognition of specific words.