목차
Large language model applications often reuse the same system prompts, project instructions, or document context across many requests. Processing that repeated input at the full rate can raise API costs without adding new information. Prompt caching reduces this overhead by applying a lower cache read price to eligible repeated input. On SiliconFlow, supported models separate standard input, cached input, and output pricing, giving developers a clearer way to estimate prompt caching cost and compare models before deployment.
What Is Prompt Caching?
Prompt caching allows previously processed input to be reused when the same stable context appears in later requests.
Consider a customer support chatbot with a 10,000-token system prompt containing:
Response rules
Product policies
Tool definitions
Safety instructions
Formatting examples
A new user message may contain only 100 tokens, but the application still sends the complete system prompt with every request.
Without caching, the repeated 10,000-token section may be processed at the standard input rate each time. When that section qualifies for a cache read, it can use the model’s lower cached-input rate.
Prompt caching does not reuse a previous answer. The model still generates a new response for the current request. Only eligible repeated input processing is reused.
Standard Input, Cached Input, and Output Tokens
A reliable prompt caching pricing estimate separates the request into three categories.
Standard Input Tokens
Standard input tokens are processed at the model’s normal input rate. They may include:
New user messages
Uncached system instructions
Conversation history
Retrieved documents
Source code
Tool results
Changed or newly added context
Repeated text does not automatically receive cached-input pricing. Billing depends on whether the repeated section qualifies for and uses the available cache.
Cached Input Tokens
Cached input tokens are eligible input tokens processed through a prompt cache. SiliconFlow model pages may label the corresponding rate as Cache Read, while the pricing page may use Cached Input.
Cache read pricing is model-specific. Some models offer a large difference between standard and cached input rates, while others provide a smaller discount.
As of July 15, 2026, the GLM-5.2 Serverless model page shows:
Token Category | Price per 1M Tokens |
|---|---|
Standard input | $1.302 |
Cached input | $0.26 |
Output | $4.092 |
Prices may change. Check the current GLM-5.2 model page before finalizing a production budget.
Output Tokens
Output tokens are generated by the model and remain subject to the normal output price.
Prompt caching can reduce the cost of eligible repeated input, but it does not reduce the output rate. Applications that produce long responses may therefore see a smaller total bill reduction, even when much of their input is cached.
Why Cache Read Pricing Matters
Repeated context can become expensive at scale.
Suppose an application includes 20,000 reusable tokens in 1,000 requests. That creates 20 million repeated input tokens. Even a modest difference between standard input and cache read pricing can have a meaningful effect on total spending.
However, a cache read discount is not the same as a total bill reduction.
The complete bill may still include:
The first processing of the prompt
New or changed input
Cache misses
Generated output
Retries
Additional Agent steps
Tool-call responses
For example, a cache read rate that is 80% below the standard input rate does not mean the entire API bill will fall by 80%. The discount only applies to eligible input that actually uses the cache.
Why Cache Hit Rate Determines Real Savings
Published cache-read pricing shows the potential discount, but cache hit rate determines how often that lower rate is realized in practice. In an OpenRouter Effective Pricing snapshot captured on August 25, 2026, SiliconFlow ranked first among the listed GLM 5.2 providers by one-day token share, at 42.5%, while recording a 91.8% cache hit rate. The high cache hit rate indicates that cached input was being used extensively during the measured period, helping narrow the gap between listed input pricing and effective input cost. These figures can change with traffic, request structure, and the measurement window, so developers should treat them as a current market snapshot and validate savings with representative production workloads. This is why prompt caching cost should be evaluated at the workflow level, not only by comparing two token prices.
OpenRouter Effective Pricing data for GLM 5.2, captured on August 25, 2026. SiliconFlow ranked first by one-day token share at 42.5% and recorded a 91.8% cache hit rate. Provider metrics are dynamic and may change over time.
This is why prompt caching cost should be evaluated at the workflow level, not only by comparing two token prices.
Where Prompt Caching Saves the Most
Prompt caching provides the most value when a large, stable block of context appears across many requests.
Chatbots With Repeated System Prompts
Production chatbots often use system prompts that are much longer than individual user messages. These prompts may contain:
Brand and tone instructions
Escalation policies
Product information
Tool descriptions
Safety rules
Response examples
If this content stays consistent, it can represent a large reusable portion of every request.
For example, a chatbot may send:
8,000 tokens of stable instructions
2,000 tokens of recent conversation history
100 tokens from the latest user message
The system instructions offer a stronger caching opportunity than the short user message. As the conversation changes, the stable prefix may remain reusable while recent messages continue to use the standard input rate.
Caching should not replace prompt cleanup. Removing duplicated or unnecessary instructions can reduce costs further.
Coding Agents With Repeated Project Context
Coding agents frequently reuse substantial project information during planning, code generation, debugging, and review.
Repeated context may include:
Repository instructions
Architecture notes
Coding standards
Dependency information
Tool definitions
File summaries
Testing requirements
The Agent can keep this stable project context at the beginning of the prompt and append the current task, selected files, terminal output, or error messages afterward.
SiliconFlow provides OpenAI-compatible APIs that can reduce integration work for tools already using common Chat Completions patterns. Developers can also connect supported models to coding environments such as Roo Code.
The interface makes model testing easier, but model parameters, context limits, tool behavior, and output quality still require validation before production deployment.
A larger context window does not automatically produce a more efficient Agent. Sending an entire repository on every step may add cost and irrelevant information. Excluding dependencies, generated files, duplicated code, and unrelated documents improves both token efficiency and task focus.
Long-Document and Session-Based RAG Workflows
Prompt caching can also help when a RAG workflow repeatedly uses the same document or stable reference set.
Relevant use cases include:
Asking several questions about one contract
Reviewing sections of a technical manual
Analyzing a fixed research report
Running repeated checks against the same policy collection
Maintaining a document-based assistant session
Savings may be limited when every query retrieves a different set of passages. If most of the prompt changes on every request, less content remains eligible for reuse.
A practical RAG structure separates:
Stable system instructions
Persistent document context
Newly retrieved passages
The current user question
This makes the reusable and dynamic parts easier to measure independently.
How to Structure Prompts for Better Cache Reuse
Prompt stability is important. Frequent changes to the beginning of a request can reduce the amount of reusable context.
A practical structure places stable content before frequently changing information:
System instructions
Tool definitions
Fixed examples
Persistent project or document context
Recent conversation history
Current user message
Live data and tool output
Keep the stable portion consistent where possible. Avoid repeatedly changing:
Message order
Tool definitions
Instruction wording
Example formatting
Document ordering
Whitespace and generated metadata
Dynamic information such as timestamps, session variables, user IDs, search results, and tool responses usually belongs later in the request.
Cache behavior can vary by model and workload. Test representative production requests instead of assuming that repeated text will always use the cache read rate.
How to Estimate Prompt Caching Savings
A cost estimate should separate reusable input, changing input, and generated output.
Use the following variables:
N: Number of requests
R: Reusable input tokens per request
U: New or changing input tokens per request
O: Output tokens per request
Pi: Standard input price per million tokens
Pc: Cached-input price per million tokens
Po: Output price per million tokens
Without prompt caching:
Total cost =
N × [(R + U) × Pi + O × Po] ÷ 1,000,000
The following formula provides an idealized cache estimate. It assumes that the first request uses standard input pricing and that the reusable portion of every later request qualifies for the cache read rate.
Total cost =[(R + U) × Pi+ (N - 1) × (R × Pc + U × Pi)+ N × O × Po]÷ 1,000,000
Estimated savings:
Savings =(N - 1) × R × (Pi - Pc) ÷ 1,000,000
Example Using GLM-5.2 Pricing
Assume a workflow has:
1,000 requests
20,000 reusable input tokens per request
1,000 new input tokens per request
500 output tokens per request
Using the July 15, 2026 GLM-5.2 prices:
Scenario | Estimated Cost |
|---|---|
All input charged at the standard rate | $29.39 |
Idealized estimate with later reusable input cached | $8.57 |
Estimated savings | $20.82 |
Estimated total cost reduction | 70.8% |
The cached-input price is about 80% below the standard input price in this example. The total bill falls by about 70.8%, not 80%, because new input and output still use their normal rates.
This is an idealized estimate. Actual costs may differ because of cache eligibility, prompt changes, cache misses, retries, and real token usage.
How to Compare Prompt Caching Pricing Across Models
Do not select a model from its cache read rate alone. Compare the complete cost and performance profile.
Factor | Why It Matters |
|---|---|
Standard input price | Applies to new input and cache misses |
Cached-input price | Applies to eligible reused input |
Output price | May dominate generation-heavy workloads |
Context capacity | Limits how much material can fit in one request |
Model capability | Affects accuracy and task completion |
Token efficiency | Influences input and output volume |
Agent behavior | More steps or tool calls increase total cost |
SiliconFlow provides access to a broad Serverless model catalog through unified APIs. This makes it easier to compare models with different capability, context, and pricing profiles without operating separate inference infrastructure for every option.
For applications already using common OpenAI-compatible request formats, changing the Base URL, API key, and model ID can reduce initial migration work. Each model still needs to be tested for supported parameters, tool use, context handling, latency, and output quality.
The most useful production metric is usually cost per successful task, rather than cost per request.
A lower-priced model may become more expensive if it requires repeated attempts. A model with a higher output rate may still be cost-effective if it completes the task with fewer tokens or fewer Agent steps.
When Prompt Caching May Not Save Much
Prompt caching offers limited value when there is little stable input to reuse.
Common examples include:
Short system prompts
One-time requests
Low request volume
Prompts that change substantially on every call
RAG results that use different documents each time
Frequently modified tool definitions
Low cache hit rates
Workflows dominated by output-token costs
Caching also cannot fix inefficient context selection. Sending irrelevant documents at a lower cached-input rate still consumes tokens and may reduce response quality.
Start by removing unnecessary context. Then use prompt caching for the content that genuinely needs to remain available across requests.
Prompt Caching vs Fine-Tuning
Prompt caching and fine-tuning address different problems.
Prompt Caching | Fine-Tuning |
|---|---|
Reuses repeated input processing | |
Reduces eligible inference input cost | Changes task-specific model behavior |
Does not change model weights | Produces an adapted model |
Helps with long repeated context | Helps with stable task patterns |
Context remains in the request | Some behavior is learned from examples |
Value depends on repeated use | Value depends on data quality and task fit |
Use prompt caching when an application must repeatedly provide long instructions, documents, project context, or tool definitions.
Use fine-tuning when the goal is to improve stable response behavior, terminology, formatting, classification, or task execution through training examples.
The two approaches can also work together. A fine-tuned model may still receive repeated system instructions, tools, or documents that benefit from cached-input pricing.
SiliconFlow supports fine-tuned chat models through the Chat Completions workflow, allowing teams to manage model adaptation and inference within the same platform.
Reduce Repeated Input Without Guessing
Prompt caching can materially reduce API spending when large, stable prompts appear across many requests. The strongest opportunities often come from chatbots, coding agents, and document workflows with substantial reusable context.
SiliconFlow separates standard input, cached input, and output pricing for supported models, making it easier to understand where each part of the API bill comes from. OpenRouter’s Effective Pricing data also provides an external signal that prompt caching is delivering meaningful savings in real workloads, although actual results vary by model and request pattern. With cached-input pricing available across a growing model catalog, developers can compare and switch models through one unified, OpenAI-compatible API instead of maintaining separate integrations. Review current model pricing, test representative prompts, and measure effective cost per completed task before scaling.
Common Questions About Prompt Caching on SiliconFlow
Q1. What Is Prompt Caching Pricing?
Prompt caching pricing is the rate applied to eligible input reused through a prompt cache. It is usually lower than the standard input rate, but the exact price varies by model.
Q2. Does SiliconFlow Cache Every Repeated Prompt?
No. Repeated text does not guarantee a cache hit. Results depend on the model, prompt structure, prefix stability, and cache eligibility. However, SiliconFlow has demonstrated strong cache performance in third-party data. In an OpenRouter snapshot captured on July 30, 2026, it recorded a 93.2% cache hit rate for Kimi K2.7 Code, the highest among the third-party providers shown at that time. Since cache hit rates are dynamic, developers should check current data and measure their own workloads.
Q3. Are Output Tokens Included in Cache Read Pricing?
No. Cache read pricing applies only to eligible input. Generated output continues to use the model’s standard output rate.
Q4. How Do You Calculate Prompt Caching Cost?
Separate reusable input, changing input, and output. Apply the cached-input rate only to the portion expected to use the cache, then add standard input and output costs.
Q5. Why Are Actual Savings Lower Than the Cache Read Discount?
The discount only covers cached input. New input, output tokens, initial processing, cache misses, retries, and additional Agent calls remain billable.
