time speech recognition API reference-Performance optimization for real-time speech recognition in high-concurrency scenarios

Paraformer real-time speech recognition uses WebSocket for streaming audio. Creating and destroying a connection per request wastes resources and increases latency. The DashScope Java SDK provides two resource reuse mechanisms: a connection pool (built-in) and an object pool (based on commons-pool2). Important

To use a model in the China (Beijing) region, go to the API key page for the China (Beijing) region. User guide: For model descriptions and selection guidance, see Real-time speech recognition - Fun-ASR/Paraformer.

How it works

The Java SDK uses two pooling layers:

• Connection pool: Built-in OkHttp3 pool that reuses WebSocket connections, reducing network handshake overhead. Enabled by default.

• Object pool: Based on commons-pool2, maintains a set of Recognition objects with pre-established connections. Borrowing from the pool eliminates connection setup latency and reduces first-packet delay.

Prerequisites

Before you begin:

• An API key

• DashScope Java SDK version 2.16.9 or later (install the latest version)

Add dependencies

Add dashscope-sdk-java and commons-pool2:

Maven

Add the following to the <dependencies> section of pom.xml:

HELPCODEESCAPE-xml
<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>dashscope-sdk-java</artifactId>

    <version>the-latest-version</version>
</dependency>

<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-pool2</artifactId>

    <version>the-latest-version</version>
</dependency>

Run mvn clean install or mvn compile to update dependencies.

Gradle

Add the following to the dependencies block of build.gradle:

HELPCODEESCAPE-groovy
dependencies {
    // Replace with 2.16.9 or later. See: https://mvnrepository.com/artifact/com.alibaba/dashscope-sdk-java
    implementation group: 'com.alibaba', name: 'dashscope-sdk-java', version: 'the-latest-version'

    // Replace with the latest version. See: https://mvnrepository.com/artifact/org.apache.commons/commons-pool2
    implementation group: 'org.apache.commons', name: 'commons-pool2', version: 'the-latest-version'
}

Run:

HELPCODEESCAPE-bash

./gradlew build --refresh-dependencies

# Windows
gradlew build --refresh-dependencies

Configure the connection pool

Set these environment variables:

Environment variable	Description	Default
DASHSCOPE_CONNECTION_POOL_SIZE	Connection pool size. Set to >2× peak concurrency.	32
DASHSCOPE_MAXIMUM_ASYNC_REQUESTS	Maximum asynchronous requests. Set to match DASHSCOPE_CONNECTION_POOL_SIZE.	32
DASHSCOPE_MAXIMUM_ASYNC_REQUESTS_PER_HOST	Maximum asynchronous requests per host. Set to match DASHSCOPE_CONNECTION_POOL_SIZE.	32

Configure the object pool

Set this environment variable:

Environment variable	Description	Default
RECOGNITION_OBJECTPOOL_SIZE	Object pool size. Set to 1.5--2× peak concurrency.	500

Important

• Object pool size (RECOGNITION_OBJECTPOOL_SIZE) must be ≤ connection pool size (DASHSCOPE_CONNECTION_POOL_SIZE). Otherwise, calling threads block when the connection pool is full.

• Object pool size must not exceed your account's QPS limit.

Create the object pool:

HELPCODEESCAPE-java
class RecognitionObjectPool {
    // ... other code omitted here, see the full code for the complete example
    public static GenericObjectPool<Recognition> getInstance() {
        lock.lock();
        if (recognitionGenericObjectPool == null) {
            // Set pool size here or via the RECOGNITION_OBJECTPOOL_SIZE environment variable.
            // Set to 1.5–2x your server's maximum concurrent connections.
            int objectPoolSize = getObjectivePoolSize();
            System.out.println("RECOGNITION_OBJECTPOOL_SIZE: "
                    + objectPoolSize);
            RecognitionObjectFactory recognitionObjectFactory =
                    new RecognitionObjectFactory();
            GenericObjectPoolConfig<Recognition> config =
                    new GenericObjectPoolConfig<>();
            config.setMaxTotal(objectPoolSize);
            config.setMaxIdle(objectPoolSize);
            config.setMinIdle(objectPoolSize);
            recognitionGenericObjectPool =
                    new GenericObjectPool<>(recognitionObjectFactory, config);
        }
        lock.unlock();
        return recognitionGenericObjectPool;
    }
}

Recommended pool sizes

These configurations are based on tests running only Paraformer real-time speech recognition on Alibaba Cloud servers. High concurrency may cause processing delays.

Maximum concurrency: the number of simultaneous recognition tasks (equals the number of worker threads).

Server specification (Alibaba Cloud)	Maximum concurrency	Object pool size	Connection pool size
4-core 8 GiB	100	500	2000
8-core 16 GiB	200	500	2000
16-core 32 GiB	400	500	2000

Borrow, use, and return Recognition objects

Borrow an object

HELPCODEESCAPE-java
recognizer = RecognitionObjectPool.getInstance().borrowObject();

If in-use objects exceed pool capacity, the system creates a new Recognition object. This requires re-initialization and a new WebSocket connection, so it does not benefit from pooling.

Perform speech recognition

Call the call or streamCall method.

Return the object after a successful task

Return the object after recognition completes:

HELPCODEESCAPE-java
RecognitionObjectPool.getInstance().returnObject(recognizer);

Important

Do not return Recognition objects from incomplete or failed tasks.

Invalidate the object after a failed task

If an exception interrupts the task, close the connection and invalidate the object:

HELPCODEESCAPE-java
// Close the connection
recognizer.getDuplexApi().close(1000, "bye");
// Invalidate the object in the pool
RecognitionObjectPool.getInstance().invalidateObject(recognizer);

No additional handling is required for TaskFailed errors.

Complete code

HELPCODEESCAPE-java
package org.alibaba.bailian.example.examples;

import com.alibaba.dashscope.audio.asr.recognition.Recognition;
import com.alibaba.dashscope.audio.asr.recognition.RecognitionParam;
import com.alibaba.dashscope.audio.asr.recognition.RecognitionResult;
import com.alibaba.dashscope.common.ResultCallback;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.utils.ApiKey;
import org.apache.commons.pool2.BasePooledObjectFactory;
import org.apache.commons.pool2.PooledObject;
import org.apache.commons.pool2.impl.DefaultPooledObject;
import org.apache.commons.pool2.impl.GenericObjectPool;
import org.apache.commons.pool2.impl.GenericObjectPoolConfig;

import java.io.FileInputStream;
import java.nio.ByteBuffer;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.Lock;

/**
 * Before making high-concurrency calls to the ASR service,
 * configure the connection pool size through the following environment
 * variables.
 *
 * DASHSCOPE_MAXIMUM_ASYNC_REQUESTS=2000
 * DASHSCOPE_MAXIMUM_ASYNC_REQUESTS_PER_HOST=2000
 * DASHSCOPE_CONNECTION_POOL_SIZE=2000
 *
 * The default is 32. We recommend that you set it to twice the maximum
 * concurrent connections of a single server.
 */
public class Main {
    public static void checkoutEnv(String envName, int defaultSize) {
        if (System.getenv(envName) != null) {
            System.out.println("[ENV CHECK]: " + envName + " "
                    + System.getenv(envName));
        } else {
            System.out.println("[ENV CHECK]: " + envName
                    + " Using Default which is " + defaultSize);
        }
    }

    public static void main(String[] args)
            throws NoApiKeyException, InterruptedException {
        // Check for connection pool env
        checkoutEnv("DASHSCOPE_CONNECTION_POOL_SIZE", 32);
        checkoutEnv("DASHSCOPE_MAXIMUM_ASYNC_REQUESTS", 32);
        checkoutEnv("DASHSCOPE_MAXIMUM_ASYNC_REQUESTS_PER_HOST", 32);
        checkoutEnv(RecognitionObjectPool.RECOGNITION_OBJECTPOOL_SIZE_ENV, RecognitionObjectPool.DEFAULT_OBJECT_POOL_SIZE);

        int threadNums = 3;
        String currentDir = System.getProperty("user.dir");
        // Replace the path with your audio source
        Path[] filePaths = {
                Paths.get(currentDir, "asr_example.wav"),
                Paths.get(currentDir, "asr_example.wav"),
                Paths.get(currentDir, "asr_example.wav"),
        };
        // Use ThreadPool to run recognition tasks
        ExecutorService executorService = Executors.newFixedThreadPool(threadNums);
        for (int i = 0; i < threadNums; i++) {
            executorService.submit(new RealtimeRecognizeTask(filePaths));
        }
        executorService.shutdown();
        // wait for all tasks to complete
        executorService.awaitTermination(10, TimeUnit.MINUTES);
        System.exit(0);
    }
}

class RecognitionObjectFactory extends BasePooledObjectFactory&lt;Recognition&gt; {
    public RecognitionObjectFactory() {
        super();
    }

    @Override
    public Recognition create() throws Exception {
        return new Recognition();
    }

    @Override
    public PooledObject&lt;Recognition&gt; wrap(Recognition obj) {
        return new DefaultPooledObject<>(obj);
    }
}

class RecognitionObjectPool {
    public static GenericObjectPool&lt;Recognition&gt; recognitionGenericObjectPool;
    public static String RECOGNITION_OBJECTPOOL_SIZE_ENV =
            "RECOGNITION_OBJECTPOOL_SIZE";
    public static int DEFAULT_OBJECT_POOL_SIZE = 500;
    private static Lock lock = new java.util.concurrent.locks.ReentrantLock();

    public static int getObjectivePoolSize() {
        try {
            Integer n = Integer.parseInt(System.getenv(RECOGNITION_OBJECTPOOL_SIZE_ENV));
            return n;
        } catch (NumberFormatException e) {
            return DEFAULT_OBJECT_POOL_SIZE;
        }
    }

    public static GenericObjectPool&lt;Recognition&gt; getInstance() {
        lock.lock();
        if (recognitionGenericObjectPool == null) {
            // Set pool size here or via the RECOGNITION_OBJECTPOOL_SIZE
            // environment variable. Set to 1.5–2x your server's maximum
            // concurrent connections.
            int objectPoolSize = getObjectivePoolSize();
            System.out.println("RECOGNITION_OBJECTPOOL_SIZE: "
                    + objectPoolSize);
            RecognitionObjectFactory recognitionObjectFactory =
                    new RecognitionObjectFactory();
            GenericObjectPoolConfig&lt;Recognition&gt; config =
                    new GenericObjectPoolConfig<>();
            config.setMaxTotal(objectPoolSize);
            config.setMaxIdle(objectPoolSize);
            config.setMinIdle(objectPoolSize);
            recognitionGenericObjectPool =
                    new GenericObjectPool<>(recognitionObjectFactory, config);
        }
        lock.unlock();
        return recognitionGenericObjectPool;
    }
}

class RealtimeRecognizeTask implements Runnable {
    private static final Object lock = new Object();
    private Path[] filePaths;

    public RealtimeRecognizeTask(Path[] filePaths) {
        this.filePaths = filePaths;
    }

    /**
     * Set your DashScope API key.
     * If you have set DASHSCOPE_API_KEY in your environment variable, you
     * can ignore this. The SDK automatically gets the API key from the
     * environment variable.
     * */
    private static String getDashScopeApiKey() throws NoApiKeyException {
        String dashScopeApiKey = null;
        try {
            ApiKey apiKey = new ApiKey();
            dashScopeApiKey =
                    ApiKey.getApiKey(null); // Retrieve from environment variable.
        } catch (NoApiKeyException e) {
            System.out.println("No API key found in environment.");
        }
        if (dashScopeApiKey == null) {
            // If you cannot set the API key in your environment variable,
            // you can set it here in the code.
            dashScopeApiKey = "your-dashscope-apikey";
        }
        return dashScopeApiKey;
    }

    public void runCallback() {
        for (Path filePath : filePaths) {
            // Create recognition parameters.
            // You can customize the recognition parameters, such as model, format,
            // and sample_rate.
            RecognitionParam param = null;
            try {
                param =
                        RecognitionParam.builder()
                                .model("paraformer-realtime-v2")
                                .format(
                                        "pcm") // 'pcm', 'wav', 'opus', 'speex', 'aac', or 'amr'.
                                // You can check the documentation for supported formats.
                                .sampleRate(16000) // Supported sample rates: 8000 and 16000.
                                .apiKey(getDashScopeApiKey()) // Use getDashScopeApiKey to get the
                                // API key.
                                .build();
            } catch (Exception e) {
                throw new RuntimeException(e);
            }

            Recognition recognizer = null;
            // if recv onError
            final boolean[] hasError = {false};
            try {
                recognizer = RecognitionObjectPool.getInstance().borrowObject();

                String threadName = Thread.currentThread().getName();

                ResultCallback&lt;RecognitionResult&gt; callback =
                        new ResultCallback&lt;RecognitionResult&gt;() {
                            @Override
                            public void onEvent(RecognitionResult message) {
                                synchronized (lock) {
                                    if (message.isSentenceEnd()) {
                                        System.out.println("[process " + threadName
                                                + "] Fix:" + message.getSentence().getText());
                                    } else {
                                        System.out.println("[process " + threadName
                                                + "] Result: " + message.getSentence().getText());
                                    }
                                }
                            }

                            @Override
                            public void onComplete() {
                                System.out.println("[" + threadName + "] Recognition complete");
                            }

                            @Override
                            public void onError(Exception e) {
                                System.out.println("[" + threadName
                                        + "] RecognitionCallback error: " + e.getMessage());
                                hasError[0] = true;
                            }
                        };
                // Replace the path with your audio file path.
                System.out.println(
                        "[" + threadName + "] Input file_path is: " + filePath);
                FileInputStream fis = null;
                // Read the file and send audio in chunks.
                try {
                    fis = new FileInputStream(filePath.toFile());
                } catch (Exception e) {
                    System.out.println("Error when loading file: " + filePath);
                    e.printStackTrace();
                }
                // Set param and callback.
                recognizer.call(param, callback);

                // Set chunk size to 100 ms for a 16 KHz sample rate.
                byte[] buffer = new byte[3200];
                int bytesRead;
                // Loop to read chunks of the file.
                while ((bytesRead = fis.read(buffer)) != -1) {
                    ByteBuffer byteBuffer;
                    if (bytesRead < buffer.length) {
                        byteBuffer = ByteBuffer.wrap(buffer, 0, bytesRead);
                    } else {
                        byteBuffer = ByteBuffer.wrap(buffer);
                    }
                    // Send the ByteBuffer to the recognition instance.
                    recognizer.sendAudioFrame(byteBuffer);
                    Thread.sleep(100);
                    buffer = new byte[3200];
                }
                System.out.println(
                        "[" + threadName + "] send audio done");
                recognizer.stop();
                System.out.println(
                        "[" + threadName + "] asr task finished");
            } catch (Exception e) {
                e.printStackTrace();
                hasError[0] = true;
            }
            if (recognizer != null) {
                try {
                    if (hasError[0] == true) {
                        // Invalidate the recognition object if an error occurs.
                        recognizer.getDuplexApi().close(1000, "bye");
                        RecognitionObjectPool.getInstance().invalidateObject(recognizer);
                    } else {
                        // Return the recognition object to the pool if no error or exception occurs.
                        RecognitionObjectPool.getInstance().returnObject(recognizer);
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        }
    }

    @Override
    public void run() {
        runCallback();
    }
}

Warm up the connection pool

The DashScope Java SDK uses a global singleton connection pool:

• On-demand creation: WebSocket connections are created during the first call, not at startup.

• 60-second reuse window: Completed connections stay in the pool for up to 60 seconds. New requests within this window reuse them. Connections idle for >60 seconds are closed automatically.

Why warm-up matters

The connection pool has no active connections when:

• The application just started.

• The service has been idle for >60 seconds.

Initial requests in these cases trigger the full WebSocket connection process: TCP handshake, TLS negotiation, and protocol upgrade. This adds latency unrelated to service processing time and skews performance measurements.

Warm-up procedure

Before performance tests or collecting metrics:

1. Make calls at target concurrency for 1--2 minutes to populate the pool.

2. Confirm the pool has enough active connections, then start collecting metrics.

This ensures measurements reflect stable-state performance.

Troubleshooting

TCP connections keep increasing

Cause (Type 1): Without an object pool, Recognition objects are destroyed after each task. The underlying connection enters an unreferenced state and disconnects after a 61-second server-side timeout. During this period, the connection cannot be reused. In high-concurrency scenarios, this leads to:

1. The number of connections continues to increase.

2. Server performance degrades because excessive connections consume server resources.

3. The connection pool becomes full, and new tasks are blocked while they wait for available connections.

Cause (Type 2): MaxIdle is smaller than MaxTotal. Idle objects exceeding MaxIdle are destroyed, causing connection leaks. These connections disconnect after a 61-second timeout, leading to the same growth pattern.

Solution:

• Type 1: Use an object pool.

• Type 2: Set MaxIdle and MaxTotal to match. Disable automatic object pool destruction.

Task takes 60 seconds longer than expected

Connection pool has reached maximum capacity. New tasks wait 61 seconds for an unreferenced connection to time out. Same root cause as the TCP connection growth issue.

Tasks are slow at startup

Cause: Too many WebSocket connections created simultaneously during high-concurrency startup.

Solution: Gradually increase concurrency, or add warm-up tasks after service starts.

"Invalid action('run-task')! Please follow the protocol!" error

Cause: A client-side error left the connection in a task-in-progress state. When this connection is reused, a protocol error occurs.

Solution: After a client-side exception, close the WebSocket connection and invalidate the object.

Abnormal traffic spikes

Cause: Simultaneous creation of many WebSocket connections causes blocking. Incoming traffic queues up, and when blocking resolves, all queued tasks execute at once. potentially exceeding the account concurrency limit and causing failures.

Common triggers:

• Service startup

• Network exceptions causing mass reconnections

• Concurrent server-side errors (such as "Requests rate limit exceeded, please try again later.")

Solution:

1. Check network conditions.

2. Check for server-side errors preceding the spike.

3. Increase your account concurrency limit.

4. Reduce object pool and connection pool sizes. Use the object pool's upper limit to cap concurrency.

5. Upgrade server specifications or add more servers.

All tasks slow down as concurrency increases

Solution:

1. Check whether the network bandwidth limit has been reached.

2. Check whether the concurrency exceeds server capacity.