Context cache

Inference requests to a large model often include overlapping input, such as in a multi-turn conversation or a series of questions about the same book. Context Cache caches the common prefix of these requests to reduce redundant computation during inference. This improves response speed and lowers usage costs without affecting response quality. To accommodate various scenarios, the context cache offers two work modes. Choose a mode based on your requirements for convenience, determinism, and cost:

• Explicit cache: A cache mode that you must actively enable. You create a cache for specific content to ensure a deterministic hit within its 5-minute validity period. Tokens used to create the cache are billed at 125% of the standard input token price, while subsequent cache hits are billed at 10% of that price.

• Implicit cache: This automatic mode requires no configuration and cannot be disabled, making it ideal for scenarios that prioritize convenience. The system automatically identifies and caches the common prefix of requests, but cache hits are not guaranteed. The portion of the input served from the cache is billed at 20% of the standard input token price.

Item	Explicit cache	Implicit cache
Affects response quality	No	No
Billing for cache creation tokens	125% of the standard input token price	100% of the standard input token price
Billing for cached input tokens	10% of the standard input token price	20% of the standard input token price
Minimum tokens for caching	1024	256
Cache validity period	5 minutes (resets on hit)	Not guaranteed; the system periodically clears inactive data.

Note

Explicit cache and implicit cache are mutually exclusive. A request can use only one work mode. Note

This topic covers OpenAI Chat Completions, DashScope, and Anthropic-compatible interfaces. Use the session cache with the Responses API to reduce inference latency and cost. For details, see Session cache.

Explicit cache

Compared to implicit cache, explicit cache requires manual creation and incurs an initial overhead. However, it provides a higher cache hit ratio and lower access latency.

How it works

To use explicit cache, add a "cache_control": {"type": "ephemeral"} marker in your messages array. The system then traces back from the position of each cache_control marker, examining up to 20 preceding content blocks to attempt a cache hit.

A single request supports up to four cache markers.

• Cache miss

  If a cache miss occurs, the system creates a new cache block from the content between the start of the messages array and the cache_control marker. The new cache block is valid for 5 minutes.

  Cache creation occurs after the model generates a response. We recommend waiting for the creation request to complete before attempting to hit that cache. A cache block contains at least 1024 tokens.

• Cache hit

  If a cache hit occurs, the system selects the longest matching prefix as the cache block and resets its validity period to 5 minutes.

The following example demonstrates how this works:

1. Send the first request: Send a system message that contains text (A) with more than 1,024 tokens and add a cache marker.

   HELPCODEESCAPE-json
   [{"role": "system", "content": [{"type": "text", "text": A, "cache_control": {"type": "ephemeral"}​}]}]

   The system creates the first cache block, called cache block A.

2. Send the second request: Send a request with the following structure.

   HELPCODEESCAPE-json
   [
       {"role": "system", "content": A},
       <Other messages>
       {"role": "user","content": [{"type": "text", "text": B, "cache_control": {"type": "ephemeral"}​}]}
   ]

   • If there are 20 or fewer "Other messages," the request hits cache block A, and its validity period is reset to 5 minutes. The system also creates a new cache block based on A, the other messages, and B.

   • If there are more than 20 "Other messages," the request misses cache block A. The system still creates a new cache block based on the full context, which includes A, the other messages, and B.

Supported models

International

Qwen Max: qwen3.6-max-preview, qwen3-max

Qwen Plus: qwen3.6-plus, qwen3.5-plus, qwen3.5-plus-2026-04-20, qwen-plus

Qwen Flash: qwen3.6-flash, qwen3.5-flash, qwen-flash

Qwen Coder: qwen3-coder-plus, qwen3-coder-flash

Qwen VL: qwen3-vl-plus, qwen3-vl-flash

DeepSeek: deepseek-v3.2

Global

Qwen Max: qwen3-max

Qwen Plus: qwen3.6-plus, qwen3.5-plus, qwen-plus

Qwen Flash: qwen3.6-flash, qwen3.5-flash, qwen-flash

Qwen VL: qwen3-vl-plus

Qwen Coder: qwen3-coder-plus, qwen3-coder-flash

Kimi: kimi-k2.5

Chinese mainland

Qwen Max: qwen3.6-max-preview, qwen3-max

Qwen Plus: qwen3.6-plus, qwen3.5-plus, qwen3.5-plus-2026-04-20, qwen-plus

Qwen Flash: qwen3.6-flash, qwen3.5-flash, qwen-flash

Qwen Coder: qwen3-coder-plus, qwen3-coder-flash

Qwen VL: qwen3-vl-plus, qwen3-vl-flash

DeepSeek: deepseek-v3.2

Kimi: kimi-k2.6, kimi-k2.5

GLM: glm-5.1

Hong Kong (China)

Qwen Max: qwen3-max

Qwen Plus: qwen-plus

Qwen Flash: qwen3.6-flash, qwen3.5-flash

Qwen VL: qwen3-vl-plus

EU

Qwen Max: qwen3-max

Qwen Plus: qwen-plus

Qwen Flash: qwen3.6-flash, qwen3.5-flash

Qwen VL: qwen3-vl-plus

Quick start

These examples demonstrate how cache blocks are created and hit using OpenAI-compatible, DashScope, and Anthropic-compatible protocols.

OpenAI compatible

HELPCODEESCAPE-python
from openai import OpenAI
import os

client = OpenAI(
    # If the environment variable is not set, replace the following line with: api_key="sk-xxx"
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # If using a model in the China (Beijing) region, replace the base_url with: https://dashscope.aliyuncs.com/compatible-mode/v1
    base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
)


long_text_content = "<Your Code Here>" * 400

# Function to send a request.
def get_completion(user_input):
    messages = [
        {
            "role": "system",
            "content": [
                {
                    "type": "text",
                    "text": long_text_content,
                    # Place the cache_control marker here. This creates a cache block containing all content from the start of the messages array to the current content's position.
                    "cache_control": {"type": "ephemeral"},
                }
            ],
        },
        # The user's question is different for each request.
        {
            "role": "user",
            "content": user_input,
        },
    ]
    completion = client.chat.completions.create(
        # Select a model that supports explicit cache.
        model="qwen3-coder-plus",
        messages=messages,
    )
    return completion

# First request
first_completion = get_completion("What is the content of this code?")
print(f"First request cache creation tokens: {first_completion.usage.prompt_tokens_details.cache_creation_input_tokens}")
print(f"First request cached tokens: {first_completion.usage.prompt_tokens_details.cached_tokens}")
print("=" * 20)
# Second request. The code content is the same, only the question is different.
second_completion = get_completion("How can this code be optimized?")
print(f"Second request cache creation tokens: {second_completion.usage.prompt_tokens_details.cache_creation_input_tokens}")
print(f"Second request cached tokens: {second_completion.usage.prompt_tokens_details.cached_tokens}")

DashScope

Python

HELPCODEESCAPE-python
import os
from dashscope import Generation
# If using a model in the China (Beijing) region, replace the base_url with: https://dashscope.aliyuncs.com/api/v1
dashscope.base_http_api_url = "https://dashscope-intl.aliyuncs.com/api/v1"

# Mock code repository content. The minimum cacheable prompt length is 1,024 tokens.
long_text_content = "<Your Code Here>" * 400

# Function to send a request.
def get_completion(user_input):
    messages = [
        {
            "role": "system",
            "content": [
                {
                    "type": "text",
                    "text": long_text_content,
                    # Place the cache_control marker here. This creates a cache block containing all content from the start of the messages array to the current content's position.
                    "cache_control": {"type": "ephemeral"},
                }
            ],
        },
        # The user's question is different for each request.
        {
            "role": "user",
            "content": user_input,
        },
    ]
    response = Generation.call(
        # If the environment variable is not set, replace the following line with your Model Studio API key: api_key = "sk-xxx",
        api_key=os.getenv("DASHSCOPE_API_KEY"),
        model="qwen3-coder-plus",
        messages=messages,
        result_format="message"
    )
    return response

# First request
first_completion = get_completion("What is the content of this code?")
print(f"First request cache creation tokens: {first_completion.usage.prompt_tokens_details['cache_creation_input_tokens']}")
print(f"First request cached tokens: {first_completion.usage.prompt_tokens_details['cached_tokens']}")
print("=" * 20)
# Second request. The code content is the same, only the question is different.
second_completion = get_completion("How can this code be optimized?")
print(f"Second request cache creation tokens: {second_completion.usage.prompt_tokens_details['cache_creation_input_tokens']}")
print(f"Second request cached tokens: {second_completion.usage.prompt_tokens_details['cached_tokens']}")

Java

HELPCODEESCAPE-java
// Minimum Java SDK version: 2.21.6
import com.alibaba.dashscope.aigc.generation.Generation;
import com.alibaba.dashscope.aigc.generation.GenerationParam;
import com.alibaba.dashscope.aigc.generation.GenerationResult;
import com.alibaba.dashscope.common.Message;
import com.alibaba.dashscope.common.MessageContentText;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.InputRequiredException;
import com.alibaba.dashscope.exception.NoApiKeyException;

import java.util.Arrays;
import java.util.Collections;

public class Main {
    private static final String MODEL = "qwen3-coder-plus";
    // Mock code repository content (repeated 400 times to exceed 1,024 tokens).
    private static final String LONG_TEXT_CONTENT = generateLongText(400);
    private static String generateLongText(int repeatCount) {
        StringBuilder sb = new StringBuilder();
        for (int i = 0; i < repeatCount; i++) {
            sb.append("&lt;Your Code Here&gt;");
        }
        return sb.toString();
    }
    private static GenerationResult getCompletion(String userQuestion)
            throws NoApiKeyException, ApiException, InputRequiredException {
        // If using a model in the China (Beijing) region, replace the base_url with: https://dashscope.aliyuncs.com/api/v1
        Generation gen = new Generation("http", "https://dashscope-intl.aliyuncs.com/api/v1");

        // Build the system message with cache control.
        MessageContentText systemContent = MessageContentText.builder()
                .type("text")
                .text(LONG_TEXT_CONTENT)
                .cacheControl(MessageContentText.CacheControl.builder()
                        .type("ephemeral") // Set the cache type.
                        .build())
                .build();

        Message systemMsg = Message.builder()
                .role(Role.SYSTEM.getValue())
                .contents(Collections.singletonList(systemContent))
                .build();
        Message userMsg = Message.builder()
                .role(Role.USER.getValue())
                .content(userQuestion)
                .build();

        // Build the request parameters.
        GenerationParam param = GenerationParam.builder()
                .model(MODEL)
                .messages(Arrays.asList(systemMsg, userMsg))
                .resultFormat(GenerationParam.ResultFormat.MESSAGE)
                .build();
        return gen.call(param);
    }

    private static void printCacheInfo(GenerationResult result, String requestLabel) {
        System.out.printf("%s cache creation tokens: %d%n", requestLabel, result.getUsage().getPromptTokensDetails().getCacheCreationInputTokens());
        System.out.printf("%s cached tokens: %d%n", requestLabel, result.getUsage().getPromptTokensDetails().getCachedTokens());
    }

    public static void main(String[] args) {
        try {
            // First request
            GenerationResult firstResult = getCompletion("What is the content of this code?");
            printCacheInfo(firstResult, "First request");
            System.out.println(new String(new char[20]).replace('\0', '='));
            // Second request
            GenerationResult secondResult = getCompletion("How can this code be optimized?");
            printCacheInfo(secondResult, "Second request");
        } catch (NoApiKeyException | ApiException | InputRequiredException e) {
            System.err.println("API call failed: " + e.getMessage());
            e.printStackTrace();
        }
    }
}

Anthropic compatible

HELPCODEESCAPE-python
import anthropic
import os

client = anthropic.Anthropic(
    # If the environment variable is not set, replace the following line with: api_key="sk-xxx"
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # If using a model in the China (Beijing) region, replace the base_url with: https://dashscope.aliyuncs.com/apps/anthropic
    base_url="https://dashscope-intl.aliyuncs.com/apps/anthropic",
)

# Mock code repository content. The minimum cacheable prompt length is 1,024 tokens.
long_text_content = "<Your Code Here>" * 400

# Function to send a request.
def get_completion(user_input):
    response = client.messages.create(
        # Select a model that supports explicit cache.
        model="qwen3-coder-plus",
        max_tokens=1024,
        system=[
            {
                "type": "text",
                "text": long_text_content,
                # Place the cache_control marker in 'text' content to create a cache. This can be in a system message (as shown) or in user/assistant/tool messages.
                "cache_control": {"type": "ephemeral"},
            }
        ],
        messages=[
            # The user's question is different for each request.
            {"role": "user", "content": user_input},
        ],
    )
    return response

# First request
first_completion = get_completion("What is the content of this code?")
print(f"First request cache creation tokens: {first_completion.usage.cache_creation_input_tokens}")
print(f"First request cached tokens: {first_completion.usage.cache_read_input_tokens}")
print("=" * 20)
# Second request. The code content is the same, only the question is different.
second_completion = get_completion("How can this code be optimized?")
print(f"Second request cache creation tokens: {second_completion.usage.cache_creation_input_tokens}")
print(f"Second request cached tokens: {second_completion.usage.cache_read_input_tokens}")

Adding the cache_control marker to the mock code repository content enables explicit cache. For subsequent requests about this repository, the system can reuse the cache block to avoid recomputation. This results in faster responses and lower costs compared to requests that do not hit the cache.

HELPCODEESCAPE-plaintext
First request cache creation tokens: 1605
First request cached tokens: 0
====================
Second request cache creation tokens: 0
Second request cached tokens: 1605

Fine-grained cache control

In complex scenarios, prompts often consist of multiple parts with different reuse frequencies. You can use multiple cache markers to achieve fine-grained control.

For example, the prompt for a smart customer service agent typically includes:

• System persona: Highly stable and rarely changes.

• External knowledge: Semi-stable. It is obtained through knowledge base retrieval or tool queries and might remain unchanged within a single conversation.

• Conversation history: Grows dynamically.

• Current question: Different for each request.

If you cache the entire prompt as a single unit, any minor change, such as an update to the external knowledge, can cause a cache miss.

You can add up to four cache markers in a request to create separate cache blocks for different parts of the prompt. This improves the cache hit ratio and enables fine-grained control.

Billing

Explicit cache only affects the billing of input tokens. The rules are as follows:

• Cache creation: Content used to create a new cache is billed at 125% of the standard input token price. If the content for a new cache includes an existing cache as a prefix, only the additional part is billed for cache creation (i.e., new cache tokens minus existing cache tokens).

  For example, if you have an existing 1,200-token cache (Cache A) and a new request needs to cache 1,500 tokens of content (Content AB), the first 1,200 tokens are billed as a cache hit at 10% of the standard price. The new 300 tokens are billed for cache creation at 125% of the standard price.

  You can view the number of tokens used for cache creation in the cache_creation_input_tokens parameter.

• Cache hit: Billed at 10% of the standard input token price.

  You can view the number of cached tokens in the cached_tokens parameter (or cache_read_input_tokens for the Anthropic-compatible protocol).

• Other tokens: Tokens that are neither a cache hit nor used for cache creation are billed at the standard input token price.

Cacheable content

Only the following message types in the messages array support cache markers:

• system message

  Note

  If a request includes the tools parameter for a function calling scenario, the tool definition is included as part of the system message for cache calculation. Tool definitions cannot be cached independently. Cache markers added to tool definitions are ignored, as they can only be added to the content of messages.

• user message

  When you create a cache with the qwen3-vl-plus model, the cache_control marker can be placed after multimodal content or text. Its position does not affect how the entire user message is cached.

• assistant message

• tool message (the result after tool execution)

For a system message, for example, you must change the content field to an array and add the cache_control field:

HELPCODEESCAPE-json
{
  "role": "system",
  "content": [
    {
      "type": "text",
      "text": "<your specified prompt>",
      "cache_control": {
        "type": "ephemeral"
      }
    }
  ]
}

This structure also applies to other message types in the messages array.

Limitations

• The minimum cacheable prompt length is 1,024 tokens.

• The cache uses a prefix matching strategy, searching backward from a cache_control marker through up to 20 preceding content blocks. A cache hit cannot occur if the match is outside this range.

• The type parameter must be set to ephemeral, which has a validity period of 5 minutes.

• A single request supports up to four cache markers.

  If more than four cache markers are provided, only the last four take effect.

Function calling cache optimization

Because a tool definition is serialized into a JSON string for cache calculation, you must ensure that the tool definition is identical across requests to avoid cache invalidation. Pay close attention to the following:

• Consistent tool list order: The order of tools in the tools array must be consistent.

• Consistent field order: The order of JSON fields for the same tool must be consistent.

• Consistent field structure: Do not omit or add fields, even if a field is empty or optional.

Usage examples

Asking different questions about a long text

HELPCODEESCAPE-python
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # If using a model in the China (Beijing) region, replace the base_url with: https://dashscope.aliyuncs.com/compatible-mode/v1
    base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
)

# Mock code repository content.
long_text_content = "<Your Code Here>" * 400

# Function to send a request.
def get_completion(user_input):
    messages = [
        {
            "role": "system",
            "content": [
                {
                    "type": "text",
                    "text": long_text_content,
                    # Place the cache_control marker here to create a cache from the start of the prompt to this content block (the mock code repository content).
                    "cache_control": {"type": "ephemeral"},
                }
            ],
        },
        {
            "role": "user",
            "content": user_input,
        },
    ]
    completion = client.chat.completions.create(
        # Select a model that supports explicit cache.
        model="qwen3-coder-plus",
        messages=messages,
    )
    return completion

# First request
first_completion = get_completion("What is the content of this code?")
created_cache_tokens = first_completion.usage.prompt_tokens_details.cache_creation_input_tokens
print(f"First request cache creation tokens: {created_cache_tokens}")
hit_cached_tokens = first_completion.usage.prompt_tokens_details.cached_tokens
print(f"First request cached tokens: {hit_cached_tokens}")
print(f"First request other tokens: {first_completion.usage.prompt_tokens-created_cache_tokens-hit_cached_tokens}")
print("=" * 20)
# Second request. The code content is the same, only the question is different.
second_completion = get_completion("What are some possible optimizations for this code?")
created_cache_tokens = second_completion.usage.prompt_tokens_details.cache_creation_input_tokens
print(f"Second request cache creation tokens: {created_cache_tokens}")
hit_cached_tokens = second_completion.usage.prompt_tokens_details.cached_tokens
print(f"Second request cached tokens: {hit_cached_tokens}")
print(f"Second request other tokens: {second_completion.usage.prompt_tokens-created_cache_tokens-hit_cached_tokens}")

This example caches the code repository content as a prefix for subsequent requests that ask different questions about the same repository.

HELPCODEESCAPE-plaintext
First request cache creation tokens: 1605
First request cached tokens: 0
First request other tokens: 13
====================
Second request cache creation tokens: 0
Second request cached tokens: 1605
Second request other tokens: 15

To ensure model performance, the system appends a few internal tokens. These tokens are billed at the standard input price. For more information, see the FAQ. Caching the tool list during function calling When caching system messages in a function calling scenario, the tools parameter is included as part of the system message for caching. You must ensure that the tool definition in each request is identical, including the tool order, field order, and field structure. You must add the cache_control marker to the content block that serves as the end of your cacheable prefix.

The following example shows the complete process: the first request creates a cache, and the second request results in a cache hit.

HELPCODEESCAPE-python
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
)

# Mock code repository content, ensuring it exceeds the minimum 1,024-token threshold for explicit cache.
long_text_content = "<Your Code Here>" * 400

# Tool definition: Ensure it is identical for every request (tool order, field order, and field structure).
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather information for a specified city.",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The city name, e.g., Beijing, Shanghai, or New York."
                    },
                    "unit": {
                        "type": "string",
                        "description": "The temperature unit, 'celsius' or 'fahrenheit'. Defaults to 'celsius'.",
                        "enum": ["celsius", "fahrenheit"]
                    }
                },
                "required": ["city"],
                "additionalProperties": False
            },
            "strict": True
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_current_time",
            "description": "Get the current date and time for a specified time zone.",
            "parameters": {
                "type": "object",
                "properties": {
                    "timezone": {
                        "type": "string",
                        "description": "IANA time zone name, e.g., 'Asia/Shanghai' or 'America/New_York'. Defaults to 'Asia/Shanghai'."
                    }
                },
                "required": [],
                "additionalProperties": False
            },
            "strict": True
        }
    },
    {
        "type": "function",
        "function": {
            "name": "convert_currency",
            "description": "Convert currency amounts based on real-time exchange rates.",
            "parameters": {
                "type": "object",
                "properties": {
                    "from_currency": {
                        "type": "string",
                        "description": "The ISO 4217 code of the source currency, e.g., CNY, USD, or EUR."
                    },
                    "to_currency": {
                        "type": "string",
                        "description": "The ISO 4217 code of the target currency."
                    },
                    "amount": {
                        "type": "number",
                        "description": "The amount to be converted."
                    }
                },
                "required": ["from_currency", "to_currency", "amount"],
                "additionalProperties": False
            },
            "strict": True
        }
    }
]

def get_completion(user_input, messages=None):
    if messages is None:
        messages = [
            {
                "role": "system",
                "content": [
                    {
                        "type": "text",
                        "text": long_text_content,
                        # Place the cache_control marker here. This creates a cache block with all content from the start of the messages array to the current content block.
                        # The cache_control marker can only be added to the content of messages, not to tools.
                        "cache_control": {"type": "ephemeral"},
                    }
                ],
            }
        ]

    messages.append({"role": "user", "content": user_input})

    completion = client.chat.completions.create(
        # Select a model that supports explicit cache.
        model="qwen3.6-plus",
        messages=messages,
        tools=tools,
        # Disable thinking mode.
        extra_body={"enable_thinking": False},
    )
    return completion

# First request: Create cache
print("=== First request (Create cache) ===")
first_completion = get_completion("What's the weather like in Beijing now?")
usage = first_completion.usage
print(f"Prompt Tokens: {usage.prompt_tokens}")
print(f"Cache creation tokens: {usage.prompt_tokens_details.cache_creation_input_tokens}")
print(f"Cached tokens: {usage.prompt_tokens_details.cached_tokens}")
print(f"Model selected tool(s): {[t.function.name for t in first_completion.choices[0].message.tool_calls or []]}")
print()

# Second request: Same system message and a different question, resulting in a cache hit.
print("=== Second request (Hit cache) ===")
messages = [
    {
        "role": "system",
        "content": [
            {
                "type": "text",
                "text": long_text_content,
                "cache_control": {"type": "ephemeral"},
            }
        ],
    }
]
second_completion = get_completion("What's the weather like in Shanghai now?", messages=messages)
usage = second_completion.usage
print(f"Prompt Tokens: {usage.prompt_tokens}")
print(f"Cache creation tokens: {usage.prompt_tokens_details.cache_creation_input_tokens}")
print(f"Cached tokens: {usage.prompt_tokens_details.cached_tokens}")
print(f"Model selected tool(s): {[t.function.name for t in second_completion.choices[0].message.tool_calls or []]}")

Running the code produces output similar to the following:

HELPCODEESCAPE-plaintext
=== First request (Create cache) ===
 Prompt Tokens: 2174
 Cache creation tokens: 2156
 Cached tokens: 0
 Model selected tool(s): ['get_weather']

 === Second request (Hit cache) ===
 Prompt Tokens: 2174
 Cache creation tokens: 0
 Cached tokens: 2156
 Model selected tool(s): ['get_weather']

Continuous multi-turn conversation In a typical multi-turn chat scenario, you can add a cache marker to the last content block in the messages array for each request. Starting from the second turn, each request will both hit and refresh the cache block from the previous turn, and create a new cache block for the current turn.

HELPCODEESCAPE-python
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # If using a model in the China (Beijing) region, replace the base_url with: https://dashscope.aliyuncs.com/compatible-mode/v1
    base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
)

system_prompt = "You are a witty person." * 400
messages = [{"role": "system", "content": system_prompt}]

def get_completion(messages):
    completion = client.chat.completions.create(
        model="qwen3-coder-plus",
        messages=messages,
    )
    return completion

while True:
    user_input = input("User: ")
    messages.append({"role": "user", "content": [{"type": "text", "text": user_input, "cache_control": {"type": "ephemeral"}​}]})
    completion = get_completion(messages)
    print(f"[AI Response] {completion.choices[0].message.content}")
    messages.append(completion.choices[0].message)
    created_cache_tokens = completion.usage.prompt_tokens_details.cache_creation_input_tokens
    hit_cached_tokens = completion.usage.prompt_tokens_details.cached_tokens
    uncached_tokens = completion.usage.prompt_tokens - created_cache_tokens - hit_cached_tokens
    print(f"[Cache Info] Cache creation tokens: {created_cache_tokens}")
    print(f"[Cache Info] Cached tokens: {hit_cached_tokens}")
    print(f"[Cache Info] Other tokens: {uncached_tokens}")

Run the code and interact with the large language model. Each question you ask will hit the cache block created in the previous turn.

Implicit cache

Supported models

Global

• Text generation model

  • Qwen Max: qwen3-max

  • Qwen Plus: qwen-plus

  • Qwen Flash: qwen-flash

  • Qwen Coder: qwen3-coder-plus, qwen3-coder-flash

  • Kimi (deployed on Alibaba Cloud Model Studio): kimi-k2.5

• Visual understanding model

  • Qwen VL: qwen3-vl-plus, qwen3-vl-flash

International

• Text generation model

  • Qwen Max: qwen3-max, qwen3-max-preview, qwen-max

  • Qwen Plus: qwen-plus

  • Qwen Flash: qwen-flash

  • Qwen Turbo: qwen-turbo

  • Qwen Coder: qwen3-coder-plus, qwen3-coder-flash

  • DeepSeek: deepseek-v4-pro, deepseek-v4-flash, deepseek-v3.2

• Visual understanding model

  • Qwen VL: qwen3-vl-plus, qwen3-vl-flash, qwen-vl-max, qwen-vl-plus

US

• Text generation model

  • Qwen Plus: qwen-plus-us

  • Qwen Flash: qwen-flash-us

• Visual understanding model

  • Qwen VL: qwen3-vl-flash-us

Chinese mainland

• Text generation model

  • Qwen Max: qwen3-max, qwen3-max-preview, qwen-max

  • Qwen Plus: qwen-plus

  • Qwen Flash: qwen-flash

  • Qwen Turbo: qwen-turbo

  • Qwen Coder: qwen3-coder-plus, qwen3-coder-flash

  • DeepSeek: deepseek-v4-pro, deepseek-v4-flash, deepseek-v3.2, deepseek-v3.1, deepseek-v3, deepseek-r1

  • Kimi: kimi-k2.6, kimi-k2.5, kimi-k2-thinking, Moonshot-Kimi-K2-Instruct

  • GLM: glm-5.1, glm-5, glm-4.7, glm-4.6

  • MiniMax: MiniMax-M2.5

• Visual understanding model

  • Qwen VL: qwen3-vl-plus, qwen3-vl-flash, qwen-vl-max, qwen-vl-plus

Hong Kong (China)

Text generation model

• Qwen Max: qwen3-max

• Qwen Plus: qwen-plus

Visual understanding model

• Qwen VL: qwen3-vl-plus

EU

Text generation model

• Qwen Max: qwen3-max

• Qwen Plus: qwen-plus

Visual understanding model

• Qwen VL: qwen3-vl-plus, qwen3-vl-flash

Note

Snapshot and latest models are not currently supported.

How it works

This feature activates automatically when you send a request to a supported model. The system works as follows:

1. Find : After receiving a request, the system uses prefix matching to check the cache for a common prefix within the request's messages array.

2. Decision:

   • If a cache hit occurs, the system uses the cached result to complete the inference.

   • If a cache miss occurs, the system processes the request normally and may store the prompt's prefix in the cache for subsequent requests.

The system periodically clears inactive cached data. The cache hit ratio is not guaranteed. A cache miss can occur even for identical requests because the system ultimately determines the final hit probability. Note

Generally, content with fewer than 256 tokens is not cached. Specific models may have different thresholds.

Improve the cache hit ratio

The implicit cache works by identifying a common prefix in different requests. To improve the cache hit ratio, place repeating content at the beginning of your prompt and unique content at the end.

• Text generation model: Assume the system has cached "ABCD". A request for "ABE" might hit the "AB" portion, while a request for "BCD" results in a cache miss.

• Visual understanding model:

  • To ask multiple questions about the same image or video: Place the image or video before the text to improve the hit ratio.

  • To ask the same question about different images or videos: Place the text before the image or video to improve the hit ratio.

Billing

There is no extra charge for using the implicit cache.

When a request results in a cache hit, the cached input tokens are billed as cached_token at a discounted rate that varies by model. Input tokens that are not served from the cache are billed at the standard input_token rate. Output tokens are billed at the standard rate.

• For models other than deepseek-v4-pro, the unit price of cached_token is 20% of the standard input_token price.

• For deepseek-v4-pro, the cached_token price is not 20% of the input_token price. See the Model Studio console for details.

For example, consider a request with 10,000 input tokens where 5,000 tokens result in a cache hit. The cost is calculated as follows:

• Uncached tokens (5,000) are billed at 100% of the standard rate.

• Cached tokens (5,000) are billed at 20% of the standard rate.

The total input cost is therefore 60% of what it would be without a cache: (5,000 × 100% + 5,000 × 20%) / 10,000 = 60%.

The number of cached tokens is available in the cached_tokens property of the response.

OpenAI compatible - Batch (file input) requests are not eligible for cache discounts.

Cache hit examples

Text generation models

OpenAI-compatible

When you call a model using an OpenAI-compatible method and an implicit cache hit occurs, the response is similar to the following. The number of cached tokens, reported in usage.prompt_tokens_details.cached_tokens, is included in the total usage.prompt_tokens.

HELPCODEESCAPE-json
{
    "choices": [
        {
            "message": {
                "role": "assistant",
                "content": "I am a large-scale language model developed by Alibaba Cloud. My name is Qwen."
            },
            "finish_reason": "stop",
            "index": 0,
            "logprobs": null
        }
    ],
    "object": "chat.completion",
    "usage": {
        "prompt_tokens": 3019,
        "completion_tokens": 104,
        "total_tokens": 3123,
        "prompt_tokens_details": {
            "cached_tokens": 2048
        }
    },
    "created": 1735120033,
    "system_fingerprint": null,
    "model": "qwen-plus",
    "id": "chatcmpl-6ada9ed2-7f33-9de2-8bb0-78bd4035025a"
}

DashScope

When you call a model using the DashScope Python SDK or an HTTP request and an implicit cache hit occurs, the response is similar to the following. The number of cached tokens, reported in usage.prompt_tokens_details.cached_tokens, is included in the total usage.input_tokens.

HELPCODEESCAPE-json
{
    "status_code": 200,
    "request_id": "f3acaa33-e248-97bb-96d5-cbeed34699e1",
    "code": "",
    "message": "",
    "output": {
        "text": null,
        "finish_reason": null,
        "choices": [
            {
                "finish_reason": "stop",
                "message": {
                    "role": "assistant",
                    "content": "I am a large-scale language model from Alibaba Cloud. My name is Qwen. I can generate various types of text, such as articles, stories, and poems, and can adapt them based on different scenarios and requirements. Additionally, I can answer various questions and provide help and solutions. If you have any questions or need assistance, feel free to ask, and I will do my best to provide support. Please note that repeating the same content may not yield a more detailed response. It is recommended that you provide more specific information or vary your questions so I can better understand your needs."
                }
            }
        ]
    },
    "usage": {
        "input_tokens": 3019,
        "output_tokens": 101,
        "prompt_tokens_details": {
            "cached_tokens": 2048
        },
        "total_tokens": 3120
    }
}

Anthropic-compatible

When you call a model using an Anthropic-compatible method and an implicit cache hit occurs, you can find the number of cached tokens in usage.cache_read_input_tokens. This value is not included in usage.input_tokens but is reported separately.

HELPCODEESCAPE-json
{
    "id": "msg_01XFDUDYJgAACzvnptvVoYEL",
    "type": "message",
    "role": "assistant",
    "content": [
        {
            "type": "text",
            "text": "This content is repeated placeholder text."
        }
    ],
    "model": "qwen3-coder-plus",
    "stop_reason": "end_turn",
    "usage": {
        "input_tokens": 82,
        "cache_creation_input_tokens": 0,
        "cache_read_input_tokens": 1536,
        "output_tokens": 14
    }
}

Visual understanding models

OpenAI-compatible

When you call a model using an OpenAI-compatible method and an implicit cache hit occurs, the response is similar to the following. The number of cached tokens, reported in usage.prompt_tokens_details.cached_tokens, is included in the total usage.prompt_tokens.

HELPCODEESCAPE-json
{
  "id": "chatcmpl-3f3bf7d0-b168-9637-a245-dd0f946c700f",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "message": {
        "content": "This image shows a heartwarming scene of a woman and a dog interacting on a beach. The woman, wearing a plaid shirt, is sitting on the sand and smiling as she interacts with the dog. The dog is a large, light-colored breed wearing a colorful collar, with its front paw raised as if to shake hands or give a high-five to the woman. The background is a vast ocean and sky, with sunlight shining from the right side of the frame, adding a warm and serene atmosphere to the entire scene.",
        "refusal": null,
        "role": "assistant",
        "audio": null,
        "function_call": null,
        "tool_calls": null
      }
    }
  ],
  "created": 1744956927,
  "model": "qwen-vl-max",
  "object": "chat.completion",
  "service_tier": null,
  "system_fingerprint": null,
  "usage": {
    "completion_tokens": 93,
    "prompt_tokens": 1316,
    "total_tokens": 1409,
    "completion_tokens_details": null,
    "prompt_tokens_details": {
      "audio_tokens": null,
      "cached_tokens": 1152
    }
  }
}

DashScope

When you use the DashScope Python SDK or the HTTP method to call a model and trigger the implicit cache, the number of tokens that hit the cache is included in the total input tokens (usage.input_tokens), and the specific viewing location varies by region and model:

• China (Beijing):

  • For qwen-vl-max and qwen-vl-plus, the count is in usage.prompt_tokens_details.cached_tokens.

  • For qwen3-vl-plus and qwen3-vl-flash, the count is in usage.prompt_tokens_details.cached_tokens.

• Asia Pacific SE 1 (Singapore): For all models, the count is in usage.cached_tokens.

Models that currently use usage.cached_tokens will be upgraded to use usage.prompt_tokens_details.cached_tokens in the future.

HELPCODEESCAPE-json
{
  "status_code": 200,
  "request_id": "06a8f3bb-d871-9db4-857d-2c6eeac819bc",
  "code": "",
  "message": "",
  "output": {
    "text": null,
    "finish_reason": null,
    "choices": [
      {
        "finish_reason": "stop",
        "message": {
          "role": "assistant",
          "content": [
            {
              "text": "This image shows a heartwarming scene of a woman and a dog interacting on a beach. The woman, wearing a plaid shirt, is sitting on the sand and smiling as she interacts with the dog. The dog is a large breed wearing a colorful collar, with its front paw raised as if to shake hands or give a high-five to the woman. The background is a vast ocean and sky, with sunlight shining from the right side of the frame, adding a warm and serene atmosphere to the entire scene."
            }
          ]
        }
      }
    ]
  },
  "usage": {
    "input_tokens": 1292,
    "output_tokens": 87,
    "input_tokens_details": {
      "text_tokens": 43,
      "image_tokens": 1249
    },
    "total_tokens": 1379,
    "output_tokens_details": {
      "text_tokens": 87
    },
    "image_tokens": 1249,
    "cached_tokens": 1152
  }
}

Anthropic-compatible

When you call a visual understanding model using an Anthropic-compatible method and an implicit cache hit occurs, the number of cached tokens is reported in the usage.cache_read_input_tokens field, which is consistent with text generation models.

HELPCODEESCAPE-json
{
  "id": "msg_01XFDUDYJgAACzvnptvVoYEL",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "This image shows a heartwarming scene of a woman and a dog interacting on a beach."
    }
  ],
  "model": "qwen-vl-max",
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 369,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 896,
    "output_tokens": 28
  }
}

Use cases

If your requests share a common prefix, context cache can significantly improve inference speed, lower inference cost, and reduce first-packet latency. This feature is particularly useful in the following scenarios:

1. Long-text question and answer

   Use this pattern when you send multiple requests about the same long text, such as a novel, textbook, or legal document.

   Messages array for the first request

   HELPCODEESCAPE-python
   messages = [{"role": "system","content": "You are a language teacher who can help students with reading comprehension."},
             {"role": "user","content": "

   Message array in the subsequent request

   HELPCODEESCAPE-python
   messages = [{"role": "system","content": "You are a language arts teacher. You can help students with reading comprehension."},
             {"role": "user","content": "

Please analyze the third paragraph of this text."}]

Although the questions differ, they are all based on the same article. The identical system prompt and article content create a large amount of repetitive prefix information, making a cache hit highly likely.
2. **Automatic code completion**

In automatic code completion scenarios, the model uses the code in the current context to generate subsequent code. As you continue to write code, the prefix often remains the same, allowing `context cache` to reuse it and accelerate completions.
3. **Multi-turn conversation**

For a `multi-turn conversation`, you append each turn to the `messages` array. This ensures each new request shares a common prefix with the previous turns, increasing the likelihood of a cache hit.

**Messages array for the first turn**

HELPCODEESCAPE-python messages=[{"role": "system","content": "You are a helpful assistant."}, {"role": "user","content": "Who are you?"}]

**Messages array for the second turn**

HELPCODEESCAPE-python messages=[{"role": "system","content": "You are a helpful assistant."}, {"role": "user","content": "Who are you?"}, {"role": "assistant","content": "I am Qwen, developed by Alibaba Cloud."}, {"role": "user","content": "What can you do?"}]

As the conversation grows, the benefits of caching for inference speed and cost become more significant.
4. **Role-playing or few-shot learning**

In role-playing or few-shot learning scenarios, you often include extensive guidance in the `prompt` to control the output format. This creates a large shared prefix across requests.

For example, when instructing the model to act as a marketing expert, the `system prompt` contains extensive text. The following are two example requests:

HELPCODEESCAPE-python system_prompt = """You are an experienced marketing expert. Please provide detailed marketing suggestions for different products in the following format:

1. Target audience: xxx

2. Main selling points: xxx

3. Marketing channels: xxx ...

4. Long-term development strategy: xxx

Please ensure your suggestions are specific, actionable, and highly relevant to the product features."""

User message for the first request, asking about a smartwatch

messages_1=[ {"role": "system", "content": system_prompt}, {"role": "user", "content": "Please provide marketing suggestions for a newly launched smartwatch."} ]

User message for the second request, asking about a laptop. Since the system_prompt is the same, there is a high probability of hitting the cache.

messages_2=[ {"role": "system", "content": system_prompt}, {"role": "user", "content": "Please provide marketing suggestions for a newly launched laptop."} ]

With `context cache`, the system can respond faster because the lengthy `system prompt` is cached, even when users frequently change the product in their request (for example, from a smartwatch to a laptop).
5. **Video understanding**

In video understanding scenarios, if you ask multiple questions about the same video, placing the `video` before the `text` increases the likelihood of a cache hit. If you ask the same question about different videos, placing the `text` before the `video` increases the likelihood of a cache hit. The following are two example requests for the same video:

HELPCODEESCAPE-python

User message for the first request, asking about the content of this video

messages1 = [ {"role":"system","content":[{"text": "You are a helpful assistant."}]}, {"role": "user", "content": [ {"video": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250328/eepdcq/phase_change_480p.mov"}, {"text": "What is the content of this video?"} ] } ]

For the second request about the same video, placing the video before the text increases the likelihood of a cache hit.

messages2 = [ {"role":"system","content":[{"text": "You are a helpful assistant."}]}, {"role": "user", "content": [ {"video": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250328/eepdcq/phase_change_480p.mov"}, {"text": "Please describe the series of events in the video. Output the start time (start_time), end time (end_time), and event (event) in JSON format. Do not output the json code block."} ] } ]

## FAQ
### Q: How do I disable implicit cache?

A: You cannot disable it. The implicit cache is enabled for all applicable model requests as long as it does not affect response quality. When a cache hit occurs, it reduces costs and improves response speed.

### Q: Why did my explicit cache miss?

A: A cache miss occurs for the following reasons:

* The system clears the cache block if it is not hit within its 5-minute validity period.

* A cache miss occurs if there are more than 20 `content` blocks between the last `content` in the prompt and an existing cache block. We recommend creating a new cache block.

### Q: Does a cache hit reset its validity period?

A: Yes. Each hit resets the cache block's validity period to 5 minutes.

### Q: Is explicit cache shared between accounts?

A: No. Both implicit and explicit cache data are isolated at the account level.

### Q:Is explicit cache shared across models?

A: No. Cache data is isolated between models.

### Q: Why isn'tinput_tokensinusagethe sum ofcache_creation_input_tokensandcached_tokens?

A: To ensure the quality of the model output, the backend service appends a small number of tokens (typically fewer than 10) to the prompt that you provide. These tokens are placed after the `cache_control` marker. Therefore, they are not counted toward cache creation or reading, but they are included in the total `input_tokens`.