Batch inference

Process large volumes of requests asynchronously at 50% the cost of real-time inference. Batch inference is OpenAI-compatible and suited for model evaluation, data labeling, and other bulk workloads.

How it works

1. Submit a task: Upload a JSONL file containing your requests.

2. Process asynchronously: The system processes tasks in a background queue. Monitor task progress and status through the console or API.

3. Download results: When the task completes, the system generates a result file with successful responses and an error file with details of any failures.

Scope

International

If you select the International deployment scope, model inference compute resources are dynamically scheduled worldwide, excluding the Chinese mainland. Static data is stored in your selected region. Supported region: Singapore.

Supported models: qwen-max, qwen-plus, qwen-flash, qwen-turbo.

Chinese mainland

If you select the Chinese mainland deployment scope, model inference compute resources are restricted to the Chinese mainland. Static data is stored in your selected region. Supported region: China (Beijing).

Supported models:

• Text generation models : The stable versions and some latest versions of Qwen-Max, Plus, Flash, and Long. The QwQ series (qwq-plus) and some third-party models (deepseek-r1, deepseek-v3.2, deepseek-v3) are also supported.

• Multimodal models : The stable versions and some latest versions of Qwen-VL-Max, Plus, and Flash. The Qwen-OCR model is also supported.

• Text embedding models: The text-embedding-v4 model.

List of supported model names

• Text generation models

  • Qwen-Max: qwen3-max, qwen-max, qwen-max-latest

  • Qwen-Plus: qwen3.6-plus, qwen3.5-plus, qwen-plus, qwen-plus-latest

  • Qwen-Flash: qwen3.5-flash, qwen-flash

  • Recommended models: qwen-long-latest

  • Recommended models: qwq-plus

  • Third-party models: deepseek-r1, deepseek-v3.2, deepseek-v3

• Multimodal models

  • Image and video understanding: qwen3.6-plus, qwen3.5-plus, qwen3.5-flash, qwen3-vl-plus, qwen3-vl-flash, qwen-vl-max, qwen-vl-max-latest, qwen-vl-plus, qwen-vl-plus-latest

  • Text extraction: qwen-vl-ocr

• Text embedding models: text-embedding-v4

Important

• In the batch processing scenario, the maximum number of input tokens per request is 256 K for qwen3.6-plus, qwen3.5-plus, and qwen3.5-flash.

• Some models support thinking mode. Enabling this mode generates thinking tokens and increases costs.

• The qwen3.6-plus and qwen3.5 series models, such as qwen3.5-plus and qwen3.5-flash, have thinking mode enabled by default. If you use a hybrid thinking model, you must explicitly set the enable_thinking parameter. Set this parameter to true to enable the mode or false to disable it.

Run batch inference

Step 1: Prepare the input file

Before creating a task, prepare a JSONL file that meets the following requirements:

• Format: UTF-8 encoded JSONL (one JSON object per line).

• Scale limits: Up to 50,000 requests per file, max 500 MB.

  Split larger datasets into separate tasks.

• Per-request limit: Up to 6 MB per JSON object, within the model context window.

• Consistency: All requests must use the same model .

• Unique identifier : Each request must include a unique custom_id field. This identifier matches requests to their results in the output files.

Each JSON object must follow this schema:

Field	Type	Required	Description
custom_id	string	Yes	Unique identifier for the request within the file
method	string	Yes	HTTP method. Only POST is supported
url	string	Yes	Request endpoint. Only /v1/chat/completions is supported
body	object	Yes	Request body matching the /v1/chat/completions schema

Sample file

HELPCODEESCAPE-json
{"custom_id":"1","method":"POST","url":"/v1/chat/completions","body":{"model":"qwen-max","messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Hello!"}]}​}
{"custom_id":"2","method":"POST","url":"/v1/chat/completions","body":{"model":"qwen-max","messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"What is 2+2?"}]}​}

JSONL batch generation tool

Use this tool to quickly generate JSONL files. JSONL batch generation tool Select a mode: Chinese Mainland (Beijing) International Select a model series: Text generation model Multimodal model General text embedding model Select a specific model: qwen3-max qwen-max qwen-max-latest qwen-flash (thinking mode) qwen-flash (non-thinking mode) qwen-plus (thinking mode) qwen-plus (non-thinking mode) qwen-plus-latest (thinking mode) qwen-plus-latest (non-thinking mode) qwen-turbo (thinking mode) qwen-turbo (non-thinking mode) qwen-turbo-latest (thinking mode) qwen-turbo-latest (non-thinking mode) qwen-long qwen-long-latest qwq-plus qwq-32b-preview deepseek-r1 deepseek-v3 Enter your requests (one request per line): Hello! How can I help you? What is 2+2? Paste your media URLs (one or more per line, separated by commas): Enter your questions about the media: Generate

Select a mode: Chinese Mainland (Beijing) International Select a model series: Text generation model Select a specific model: qwen-max qwen-plus qwen-flash (thinking mode) qwen-flash (non-thinking mode) qwen-turbo Enter your requests (one request per line): Hello! How can I help you? What is 2+2? Generate

Step 2: Create a task

1. On the Batch inference page, click Create Batch.

2. In the dialog box, enter a Task Name and Description , set the Maximum Waiting Time (1 to 14 days), and upload your JSONL file.

   Click Download Sample File to download a template.

3. When ready, click Confirm.

Step 3: Monitor and manage tasks

• View:

  • On the task list page, view the Progress (processed/total requests) and Status for each task.

  • Search by task name or ID, or filter by workspace to locate a specific task.

• Manage:

  • Cancel: Cancel a running task from the Actions column.

  • Troubleshoot: For failed tasks, hover over the status to view an error summary and download the error file for details.

Step 4: Download results

When a task completes, click View Results to download the output files:

• Result file : Contains all successful requests and their response results.

• Error file (if any) : Contains all failed requests and their error details.

Both files include custom_id for matching results with input requests.

Step 5: View usage statistics (optional)

On the Model Monitoring page, filter and view usage statistics for batch inference.

• View data overview : Select a Time (up to 30 days) and set Inference Type to Batches to display:

  • Monitoring data: Summary statistics for all models in the selected period, including total calls and failures.

  • Model list: Detailed metrics for each model, including total calls, failure rate, and average call duration.

  To view inference data older than 30 days, go to the Bills page.

• View model details : In the Models , click Actions for a specific model, then select Monitor to view Call Statistics such as call count and usage volume.

Important

• Call data is recorded when tasks complete. Running tasks show no call data until finished.

• Monitoring data has a 1 to 2 hour delay.

API reference

Use the OpenAI-compatible API to automate batch task creation and management. Core workflow:

1. Upload a file

   Call POST /v1/files to upload your file and record the returned file ID.

2. Create a task Call POST /v1/batches with the file ID from step 1 , and record the returned batch_id.

3. Poll status Use the batch_id to poll GET /v1/batches/{batch_id}. When status becomes completed, record the output_file_id and stop polling.

4. Download results Use the output_file_id to call GET /v1/files/{output_file_id}/content and download the result file.

For complete Batch API definitions and examples, see OpenAI compatible - Batch (file input).

Task lifecycle

• validating: The system is verifying file format (JSONL) and request validity.

• in_progress: The system is processing requests.

• completed: Result and error files are ready for download.

• failed: Validation failed (incorrect format or oversized file). No requests were executed.

• expired: The task exceeded the maximum wait time. Create a new task with a longer timeout to retry.

• cancelled: The task was manually cancelled. Unstarted requests were terminated.

Billing

• Unit price: The input and output tokens for all successful requests are charged at 50% of the real-time inference price for the corresponding model. For more information, see Model list.

• Billing scope:

  • Only requests successfully executed within a task are billed.

  • Requests that fail because of file parsing errors, task execution failures, or row-level errors do not incur charges.

  • For canceled tasks, requests successfully completed before the cancellation are still billed as normal.

Note

• Batch inference is a separate billing item. It supports AI general-purpose savings plan, but not discounts, such as subscription (other savings plans) or free quotas for new users. It also does not support features such as context cache.

• Some models, such as qwen3.5-plus and qwen3.5-flash, have thinking mode enabled by default. This mode generates additional thinking tokens, which are billed at the output token price and increase costs. To control costs, set the `enable_thinking` parameter based on task complexity. For more information, see Deep thinking.

FAQ

1. Do I need to purchase or enable anything extra?

   No. Activate Model Studio and pay as you go.

2. Why does my task fail immediately after submission?

   This typically indicates a file-level error --- no inference requests are executed. Check the following:

   • File format: Verify it uses strict JSONL format with one complete JSON object per line.

   • File scale: Ensure the file size and line count do not exceed limits. For details, see Prepare the input file.

   • Model consistency: Verify that the body.model field is identical across all requests and that the model is available in your region.

3. How long does a batch task take?

   Processing time depends on system load. Tasks may queue during peak hours. Results return within the specified timeout.

Error codes

If a call fails with an error message, see Error messages.