同样的 API 调用有人多花 10 倍?Token 交易市场缓存命中机制,才是 2026 年真正的降本密码

Model prices have fallen, open-source alternatives have multiplied, and price-per-million-Token charts keep moving downward.

Bilingual Article | English First, Chinese Version Below

The Same API Call Can Cost 10 Times More: Cache Hit Rate Is the Overlooked Cost Lever of 2026

Over the past two years, AI developers have repeatedly heard the same claim:

“Tokens are becoming worthless.”

Model prices have fallen, open-source alternatives have multiplied, and price-per-million-Token charts keep moving downward.

Yet many teams open their monthly bills and discover that their total AI spending has barely changed. Some applications can repeat a request at almost no inference cost, while others continue paying the full price every time the same context is processed.

The difference is not always the model.

It is often the cache-hit rate.

By 2026, cost optimization is no longer simply about finding the provider with the lowest listed Token price. It is about determining which information should be calculated once, which results can be reused, and how requests can remain attached to the provider that already holds the warm cache.

Three mechanisms are becoming especially important:

  • Response Caching for identical requests;

  • Prompt Caching for reusable context;

  • Sticky Routing for keeping follow-up requests on the same provider.

Used together, they can create a dramatic difference in cost and latency—even when two applications are calling exactly the same model.


Response Caching: Identical Requests Can Skip Inference

On April 30, OpenRouter introduced Response Caching.

When enabled, the platform generates a cache key from elements such as the request body, selected model, API credential context, and streaming configuration.

The first request is processed normally by the model provider and billed according to standard usage.

When a later request is completely identical and the cached result remains valid, OpenRouter can return the stored response directly from its edge cache.

That means the platform does not need to send the request to the underlying model provider again. No new inference is performed, and normal Token-generation charges can be avoided for that cache hit.

In simple terms:

The first request pays for the answer. Identical requests can reuse it.

The default cache time-to-live is five minutes, while developers can adjust the retention period from a few seconds to as long as 24 hours, depending on the workflow.

The latency improvement can be as important as the financial saving.

A normal model request may require several seconds to complete. A cached response can return within roughly 80 to 300 milliseconds, while the cache lookup itself may take only a few milliseconds.

Compared with uncached model responses that may take one, five, or even nine seconds, a cache hit can improve response time by an order of magnitude.


Where Response Caching Creates the Most Value

Response Caching works best when the entire request is exactly the same.

Typical use cases include:

Agent Workflow Retries

A multi-step agent may fail during its fifth operation and restart the workflow.

Without caching, the first four completed steps may be generated and billed again.

With response caching, the repeated steps can return almost immediately, allowing the user to pay primarily for the new work.

Repeated Test-Suite Execution

Development teams often run the same LLM evaluation or regression suite several times.

The first execution can populate the cache. Subsequent identical runs can reuse previous responses, reducing the cost of confirming whether surrounding application code still behaves correctly.

Repeated Batch Requests

Some batch-processing systems encounter duplicate inputs across files, customers, or data partitions.

Response Caching prevents the same content from being inferred repeatedly.

Static or Semi-Static Answers

Frequently asked questions, standard policy explanations, fixed classifications, template outputs, and deterministic transformations may all benefit from cached responses.

Text, images, audio, documents, and tool-related requests may be cacheable when the complete request is reproducible.

Extremely large multimodal payloads may be excluded because their processing is handled through different internal infrastructure.


Response Caching Is Powerful, but the Match Must Be Exact

The main limitation is also straightforward: the request must match.

A minor difference in the prompt, model, parameter, tool definition, or conversation context may generate a different cache key.

For example, these two prompts may appear equivalent to a human:

Summarize this report in three points.

Summarize this report using three bullet points.

But they are not identical requests and may not share the same response cache.

This is why production teams often normalize requests before sending them.

They standardize:

  • Whitespace;

  • Parameter order;

  • Default values;

  • Tool definitions;

  • System prompts;

  • Message structure;

  • Model identifiers.

The more consistent the request format, the more likely repeated work will produce a cache hit.


Prompt Caching: Reuse the Shared Prefix at a Discount

Response Caching solves the problem of completely identical requests.

Most AI applications, however, do not send the same request every time.

The system instructions remain unchanged, the knowledge-base content is similar, and the conversation history may be reused—but the latest user question is different.

That is where Prompt Caching becomes more valuable.

Prompt Caching allows the provider to store the stable portion of a prompt and reuse it in future requests. Instead of charging the full input price every time the model reads the same long context, cached reads are billed at a lower rate.

The reusable content might include:

  • A long system prompt;

  • A company handbook;

  • Product documentation;

  • A code repository summary;

  • Contract templates;

  • Compliance policies;

  • Tool instructions;

  • Long conversation history.

The new portion of the request is still processed normally, while the shared prefix receives the cached-read rate.


Cache-Read Discounts Vary by Provider

Different providers apply different cache-read discounts.

Representative pricing structures include:

Provider or model familyTypical cached-read relationship
Anthropic Claude Sonnet seriesAround 0.1× the standard input price
DeepSeek and Alibaba QwenAround 0.1× in supported scenarios
OpenAICommonly around 0.25×–0.5×, depending on the model and policy
Gemini and GrokOften around 0.25× for supported caching

For example, if a model’s normal input price is $3 per million Tokens and its cached-read price is $0.30, reusing the stored context costs only one-tenth of reading it from scratch.

This becomes financially significant when a long prompt is reused thousands of times.

Consider an enterprise assistant that includes a 100,000-Token policy handbook in every request.

Without caching, the company repeatedly pays the full price to process that handbook.

With Prompt Caching, the handbook is written once and then read at a steep discount, while only the user’s new question and the model output are billed at normal rates.


Cache Writes Are Not Always Free

The common mistake is to focus only on the discounted read price.

Creating the cache may cost more than a standard input request.

For Anthropic, a five-minute cache write may be priced at approximately 1.25 times the normal input cost. A longer one-hour cache duration may increase the write cost to around twice the standard input price.

For newer OpenAI models, cache creation may also be billed above the regular input rate, such as approximately 1.25 times the uncached input price.

Some Gemini and Grok caching implementations may not charge a separate cache-write premium, although storage rules and model policies can vary.

This creates an important economic rule:

Caching only saves money when the cached content is reused enough times.

Suppose a prompt costs $1 to process normally, while writing it to cache costs $1.25 and each cached read costs $0.10.

  • No reuse: caching costs $1.25 instead of $1.

  • One reuse: total caching cost is $1.35 instead of $2.

  • Ten reuses: total caching cost is $2.25 instead of $11.

The cache becomes more valuable as reuse increases.

If the content is written once and never read again, enabling caching can make the request more expensive rather than cheaper.

That is why “turning on caching” is not yet a cost strategy. The strategy must also determine what is worth caching and how frequently it will be reused.


Why Do Some Teams Spend 10 Times More?

Two applications can use the same model for the same business task and still produce dramatically different bills.

There are several common causes.

1. The Team Does Not Enable Caching

Response Caching may require only a cache-related request header or platform setting.

Yet many developers never notice the option and continue sending every repeated request to the model provider at the full price.

A small configuration change can therefore separate an application that pays once from one that pays repeatedly.

2. The Prompt Is Too Short

Prompt Caching often requires a minimum prompt length.

Examples may include:

  • Claude Sonnet models: approximately 1,024 Tokens;

  • OpenAI models: often at least 1,024 Tokens;

  • Gemini Pro models: potentially several thousand Tokens, depending on the model and caching implementation.

If the stable prefix is below the required threshold, the provider may not create a cache entry.

The developer believes caching is active, but the request continues to be billed normally.

3. The Prompt Changes Too Often

A timestamp, random request ID, reordered tool list, changing metadata field, or dynamically generated system message can break the shared prefix.

Even when most of the prompt appears unchanged, the provider may treat it as new content.

4. The Cache Is Created but Never Reused

A high cache-write cost cannot be recovered if the content expires before another matching request arrives.

An inappropriate TTL can make this worse.

A five-minute cache is not useful if the next related request arrives an hour later. A one-hour cache may be wasteful if the content is needed only twice within one minute.

5. The Request Moves to Another Provider

This is one of the most easily overlooked problems in multi-provider routing.

A cache created by Provider A may not be available at Provider B.

If the next request is sent to a different provider, the previously warm cache becomes irrelevant, and the same context may need to be processed again at the full price.

This problem is known as provider drift.


Provider Drift Can Destroy a Good Cache Strategy

OpenRouter connects applications to a large network of model and infrastructure providers.

That diversity improves availability and price competition, but it also creates a challenge.

Imagine that the first request is routed to Provider A, where a long system prompt is written into cache.

The second request is routed to Provider B because its current price or latency appears slightly better.

Provider B does not possess Provider A’s cache. The long context must be processed again, generating another cache write or full-price input charge.

The platform may have selected a cheaper provider for the individual request but created a more expensive workflow overall.

This is why routing decisions must account for cache locality.

The cheapest cold node is not always cheaper than the slightly more expensive warm node.


Sticky Routing Keeps Requests Close to the Warm Cache

OpenRouter addresses provider drift through Sticky Routing.

When the routing system detects that a provider offers discounted cached reads, follow-up requests can be directed back to the same provider instead of being redistributed freely.

A more deliberate approach is to include an explicit session identifier.

The session allows the routing policy to associate related requests from the beginning, rather than waiting for a successful cache read to reveal where the warm state exists.

Sticky Routing helps preserve:

  • Prompt-cache locality;

  • Conversation continuity;

  • Provider-specific state;

  • Consistent latency;

  • Predictable cost.

This changes routing from a simple price comparison into a workflow-level optimization problem.

The platform should not ask only:

Which provider is cheapest for this request?

It should ask:

Which provider can complete the entire session at the lowest total cost?


A High Cache-Hit Rate Changes the Economics of the Application

According to platform data, well-optimized applications may achieve cache-hit rates in the range of approximately 40% to 60%.

That means close to half of all calls may receive either a deeply discounted cached-input price or a fully reused response.

Actual savings depend on the model, cache type, write price, output volume, and routing stability.

For some Claude workloads, cache-hit calls may cost around 30% to 50% of a comparable uncached request after all components are considered.

This creates a large gap between teams.

One application may spend $1 on a reusable workflow.

Another may spend $3, $5, or more for the same result because it repeatedly processes the same context, misses the cache, or drifts between providers.

The model is identical.

The business task is identical.

The difference is infrastructure design.


The Real Metric Is Not Token Price, but Paid Reprocessing

Many teams obsess over the price per million Tokens while ignoring how often they pay for the same Tokens again.

A model that charges $1 per million Tokens but repeatedly processes identical context may be more expensive than a $3 model with effective caching.

A more useful cost equation is:

Total cost = New work + Repeated work + Failed work + Routing inefficiency

Caching primarily attacks repeated work.

Sticky Routing reduces routing inefficiency.

Retry protection lowers the cost of failed work.

Once these components are optimized, the listed Token price becomes only one part of the total bill.


A Practical Three-Layer Caching Strategy

A mature AI application can treat caching as three separate layers.

Layer One: Response Caching

Use it for requests that are fully identical and safe to reuse.

Examples include standard answers, test cases, repeat retries, and fixed transformations.

Layer Two: Prompt Caching

Use it for long, stable prefixes that support many different user requests.

Examples include documentation, code context, policies, contracts, and system instructions.

Layer Three: Sticky Routing

Keep related requests attached to the provider that holds the useful cache.

A platform may also track:

  • Cache-hit rate;

  • Cache-write frequency;

  • Cached-read savings;

  • Provider switches;

  • TTL expiration;

  • Savings per workflow;

  • Cost per successful task.

Caching should be measured as an operating system, not treated as a one-time API option.


Unexhub: Moving Beyond Caching Toward a Token Marketplace

Caching answers one question:

How can repeated model usage become cheaper?

Unexhub is addressing a broader question:

How can Token capacity be aggregated, competitively allocated, and settled like a market resource?

Unexhub is developing a Token aggregation and distribution marketplace designed to connect major model ecosystems, smaller AI suppliers, enterprise users, and developers.

The platform covers or plans to aggregate services from model ecosystems such as Gemini, ChatGPT, Claude, DeepSeek, and GLM.

It also creates a route to market for independent developers, specialized providers, and small or medium-sized AI companies with quality model capacity.

Its model can be understood through three layers.


Layer One: Faster Usage-Based Settlement

Traditional distribution may require smaller suppliers to wait weeks before receiving payment for completed API calls.

During that period, GPU, cloud, bandwidth, storage, and operating expenses continue accumulating.

Unexhub aims to shorten the interval between successful usage and supplier settlement.

Once consumption has been measured and the request has been completed, the corresponding revenue share can be calculated and settled more quickly.

This can help suppliers:

  • Recover infrastructure costs sooner;

  • Improve cash-flow visibility;

  • Reduce working-capital pressure;

  • Reinvest more quickly in capacity;

  • Release idle inference resources to the market.

It would be more accurate to say that faster settlement can materially relieve financial pressure rather than eliminate it entirely.


Layer Two: Early Supplier Incentives

Early suppliers may receive promotional platform rates as low as 0% commission.

This lowers the initial cost of joining the Token distribution network.

A smaller supplier does not need to build a complete international sales, billing, routing, and clearing system independently. It can connect its capacity to the platform and receive traffic according to price, capability, and availability.

The benefit is not merely avoiding a platform fee.

It gives smaller providers a lower-risk way to test global demand and monetize underused capacity.


Layer Three: API Bidding and Dynamic Pricing

Unexhub is also developing an API bidding mechanism in which prices can respond to market conditions.

The logic resembles an electricity spot market.

During weekday business hours, enterprise inference demand may rise and available capacity may become more valuable.

During late nights, weekends, or low-utilization periods, suppliers may lower their offers to attract additional requests.

For developers, this creates the possibility of:

  • Scheduling non-urgent workloads during cheaper periods;

  • Automatically selecting lower-cost suppliers;

  • Paying according to actual Token usage;

  • Avoiding unused prepaid packages;

  • Accessing selected services at prices as low as approximately 30% of typical market rates.

For suppliers, dynamic pricing can help them:

  • Earn more during peak demand;

  • Monetize idle capacity during off-peak periods;

  • Receive traffic based on real market needs;

  • Compete through price, speed, and service quality.

Combined with millisecond-level routing, the platform can compare model capability, current price, latency, availability, and historical success rate before selecting a supplier.

The developer sees one API.

Behind it, the market continuously evaluates where the request should go.


Caching and Market Pricing Optimize Different Parts of the Bill

Caching and Token-market mechanisms are complementary.

Caching reduces the amount of work that needs to be purchased.

Dynamic routing and bidding reduce the price of the work that remains.

The cost equation therefore becomes:

Final cost = Unavoidable new work × Best available market price

Response Caching can remove identical repeated work.

Prompt Caching can discount reusable context.

Sticky Routing can preserve the warm cache.

Unexhub-style bidding can locate more economical capacity for the remaining inference.

Together, these mechanisms move cost optimization beyond a simple model-price comparison.

The question is no longer only:

Which model has the lowest price?

It becomes:

Which work can be reused, when should the remaining work run, and which supplier should complete it?


Final Takeaway

The same API call can cost one team several times more than another—not necessarily because they selected the wrong model, but because they designed the workflow differently.

Response Caching allows identical requests to reuse previous results without repeating normal inference.

Prompt Caching reduces the cost of repeatedly reading the same long context.

Sticky Routing prevents a warm cache from being lost when traffic moves between providers.

Together, these mechanisms form one of the most important cost strategies of 2026.

Unexhub points toward the next layer of optimization.

When Token capacity can be aggregated across suppliers, routed in milliseconds, competitively priced, and settled according to actual usage, the cost structure of AI applications becomes increasingly dynamic.

Developers can benefit from:

  • Unified access to major domestic and international models;

  • Pay-as-you-go Token billing;

  • Millisecond-level supplier routing;

  • Lower prices for high-volume and off-peak workloads;

  • Selected services available at approximately 30% of typical market rates;

  • Less engineering and financial overhead from multiple providers.

Suppliers can benefit from:

  • One-step access to a broader distribution network;

  • Faster usage-based settlement;

  • Early-stage commission incentives as low as 0%;

  • Automated traffic allocation;

  • A way to monetize idle or underused inference capacity.

The future of AI cost optimization will not be decided only by whose model is cheapest.

It will be decided by three questions:

How much work can be reused?
When should new work be executed?
Which supplier can deliver it at the best total cost?

Tokens may be becoming cheaper.

But the teams that understand caching, routing, and market timing will still pay far less than those that do not.


中文版|English Version Above

本文为英中双语文章。以上为英文全文,以下为完整中文版本。

同样的 API 调用,有人为什么多花 10 倍?缓存命中率才是 2026 年被低估的降本密码

过去两年,AI 应用开发者听到最频繁的一句话,大概是:

“Token 已经不值钱了。”

模型单价不断下调,开源模型越来越多,各类价格表上的“每百万 Token 成本”也在持续下降。

但很多团队打开月底账单,却发现总支出并没有明显减少。

有些应用可以让重复请求以接近零推理成本返回,有些应用却仍在为相同的系统提示词、文档上下文和重试步骤反复支付全价。

差距未必来自模型选得对不对。

更多时候,差距来自缓存命中率。

到了 2026 年,AI 降本已经不只是寻找目录价格最低的模型,而是判断哪些信息只需要计算一次、哪些结果可以直接复用,以及后续请求如何继续路由到保存着热缓存的供应商。

其中,三种机制尤其重要:

  • 响应缓存:复用完全相同的请求;

  • 提示缓存:复用稳定不变的上下文;

  • 粘性路由:让后续请求继续命中同一家供应商的缓存。

三层机制叠加后,即使两个应用调用的是同一个模型,最终成本和响应速度也可能相差数倍。


响应缓存:相同请求无需重复推理

4 月 30 日,OpenRouter 推出了 Response Caching,也就是响应缓存。

开启后,平台会结合请求体、模型、API 鉴权环境和流式模式等信息生成缓存键。

第一次请求仍会正常发送给模型供应商,并按照实际用量计费。

在缓存有效期内,如果后续请求与第一次完全一致,OpenRouter 可以直接从边缘缓存返回已经生成的完整响应。

这意味着平台不必再次调用底层模型,也不会发生新的常规 Token 推理费用。

简单来说就是:

第一次请求购买答案,后续相同请求直接复用。

响应缓存的默认 TTL,也就是有效期,为 5 分钟。开发者可以根据业务特点,将其调整到几秒或最长 24 小时。

除了成本之外,速度提升也十分明显。

正常模型推理可能需要数秒,而缓存命中的响应通常可以在约 80—300 毫秒内返回,缓存查找本身甚至可能只需要几毫秒。

与耗时一秒、五秒甚至九秒的未缓存请求相比,缓存命中的响应速度可以提高一个数量级。


哪些场景最适合响应缓存?

响应缓存最适合整个请求完全相同的工作流。

智能体失败后的重复执行

一个多步骤智能体可能在第五步发生错误,需要重新启动。

如果没有缓存,前四个已经成功完成的步骤仍可能重新执行并再次计费。

使用响应缓存后,重复步骤可以直接返回,用户主要为尚未完成的新工作付费。

LLM 测试套件重复运行

开发团队经常需要多次执行相同的模型评测或回归测试。

第一次运行负责填充缓存,后续相同请求可以复用已有结果,用更低成本确认外围代码是否仍然正常。

批量任务中的重复输入

在批量处理文件、客户数据或不同数据分区时,相同输入可能重复出现。

响应缓存能够避免模型反复处理完全一致的内容。

固定或半固定回答

常见问题、标准政策说明、固定分类、模板化输出以及确定性较强的内容转换,都可能从响应缓存中获益。

当完整请求能够稳定复现时,文本、图像、音频、文档和部分工具调用都可以进入缓存。

但对于体积特别大的多模态负载,平台可能因为内部处理机制不同而不支持缓存。


响应缓存的前提,是请求必须完全一致

响应缓存最明显的限制,也正是“完全相同”。

提示词、模型名称、参数、工具定义或对话上下文中的微小变化,都可能生成不同的缓存键。

例如下面两句话,对人类来说意思非常接近:

请把这份报告总结成三个要点。

请用三个要点总结这份报告。

但它们并不是完全相同的请求,未必能够共享同一份响应缓存。

因此,成熟的生产系统通常会在发送请求之前进行标准化处理,包括:

  • 统一空格和换行;

  • 固定参数顺序;

  • 明确默认参数;

  • 规范工具定义;

  • 保持系统提示词一致;

  • 固定消息结构;

  • 统一模型名称。

请求格式越稳定,重复任务命中缓存的概率就越高。


提示缓存:共享上下文,读取价最低可至 1 折

响应缓存解决的是“整个请求完全相同”的问题。

但大多数 AI 应用每次发送的内容都不完全一样。

系统提示词保持不变,企业知识库大致相同,对话历史也会被重复使用,但用户提出的新问题不同。

这时,Prompt Caching,也就是提示缓存,更具价值。

提示缓存允许供应商保存提示词中稳定不变的部分。后续请求无需再按照完整输入价格重新读取这些内容,而是按照更低的缓存读取价格结算。

适合缓存的内容包括:

  • 长篇系统提示词;

  • 企业员工手册;

  • 产品说明文档;

  • 代码仓库摘要;

  • 合同模板;

  • 合规政策;

  • 工具使用说明;

  • 较长的对话历史。

用户新提交的问题仍然正常处理,而稳定的共享前缀则享受缓存读取价格。


不同供应商的缓存读取折扣

各家模型供应商的缓存政策并不完全一致。

具有代表性的价格关系包括:

厂商或模型体系常见缓存读取比例
Anthropic Claude Sonnet 系列约为标准输入价的 0.1 倍
DeepSeek、阿里通义千问支持场景下约为 0.1 倍
OpenAI根据模型与规则,常见约为 0.25—0.5 倍
Gemini、Grok部分缓存场景约为 0.25 倍

假设某模型的标准输入价格是每百万 Token 3 美元,缓存读取价格为 0.30 美元,那么重复读取相同上下文的成本仅为原来的十分之一。

当长提示词需要被重复使用数千次时,这项差异会非常明显。

例如,一个企业智能助手每次请求都需要附带一份 10 万 Token 的政策手册。

没有缓存时,每一次调用都需要按全价重新处理整份手册。

开启提示缓存后,手册只需要进行一次缓存写入,后续以大幅折扣后的价格重复读取;只有用户的新问题和模型输出按照正常价格结算。


缓存写入并不一定免费

很多开发者只看到了低廉的缓存读取价格,却忽略了缓存写入成本。

创建缓存可能比普通输入更贵。

例如,Anthropic 的 5 分钟缓存写入价格可能约为标准输入成本的 1.25 倍;如果选择 1 小时的缓存期限,写入价格可能升至标准输入成本的约 2 倍。

部分新一代 OpenAI 模型的缓存写入价格,也可能达到普通输入价格的约 1.25 倍。

Gemini 和 Grok 的部分缓存方案可能不额外收取写入费用,但具体储存规则仍取决于模型和服务政策。

这带来一个非常重要的经济原则:

缓存只有被重复使用,才真正能够省钱。

假设一份提示词正常处理需要 1 元,写入缓存需要 1.25 元,每次缓存读取需要 0.1 元。

  • 从未复用:缓存成本为 1.25 元,高于普通调用的 1 元;

  • 复用一次:总成本为 1.35 元,低于两次普通调用的 2 元;

  • 复用十次:总成本为 2.25 元,而普通调用需要 11 元。

缓存被复用得越多,价值越明显。

如果一份内容写入缓存后再也没有被读取,开启缓存反而可能更贵。

所以,“打开缓存开关”还不是完整的降本策略。团队还需要判断哪些内容值得缓存,以及未来能被复用多少次。


为什么同样调用,有人会多花 10 倍?

两个应用使用同一个模型、执行相似业务,账单却可能相差数倍。

常见原因主要有以下几类。

第一,没有启用缓存

响应缓存可能只需要增加一项请求头或打开平台设置。

但很多开发者根本没有注意到这个功能,仍然把每一次相同请求发送给供应商并支付全价。

一个很小的配置差异,就能决定应用是在“支付一次”还是“重复支付”。

第二,提示词没有达到最低长度

提示缓存通常存在最低 Token 阈值。

例如:

  • Claude Sonnet 系列可能要求至少约 1,024 Token;

  • OpenAI 的部分模型也可能要求不少于 1,024 Token;

  • Gemini Pro 系列根据模型与缓存机制,门槛可能达到数千 Token。

如果共享前缀没有达到最低长度,供应商不会真正创建缓存。

开发者以为缓存已经开启,但请求仍然按照正常输入价格结算。

第三,提示词变化过于频繁

时间戳、随机请求 ID、重新排序的工具列表、变化的元数据字段,以及动态生成的系统提示词,都可能破坏共享前缀。

即使提示内容大部分没有改变,供应商也可能将它视为一份新输入。

第四,缓存写入后没有及时复用

如果缓存还没被再次读取就已经过期,较高的写入费用无法被摊薄。

TTL 设置不合理,也会加剧这个问题。

如果下一次相关请求一小时后才出现,5 分钟缓存没有价值;如果内容只会在一分钟内使用两次,购买一小时缓存又可能造成浪费。

第五,请求被调度到另一家供应商

这是多供应商路由中最容易被忽略的问题之一。

供应商 A 创建的缓存,通常无法被供应商 B 直接读取。

如果下一轮请求被路由到另一家供应商,原本的热缓存就失去了作用,同样的上下文需要重新处理并再次计费。

这就是“供应商漂移”。


供应商漂移,会摧毁原本有效的缓存策略

OpenRouter 连接了大量模型和基础设施供应商。

供应商数量越多,价格比较与服务可用性越强,但缓存管理也会更复杂。

假设第一次请求被发送到供应商 A,一段很长的系统提示词已经成功写入缓存。

第二次请求由于供应商 B 当前价格更低或延迟更小,被调度到供应商 B。

但供应商 B 并没有供应商 A 的缓存。

长上下文必须重新处理,可能再次产生缓存写入费或正常输入费。

平台看似为单次请求选择了更便宜的节点,却让整条工作流付出了更高成本。

因此,路由策略不能只考虑当前单次请求的价格,还必须考虑缓存位于哪里。

最便宜的冷节点,未必比价格略高的热节点更划算。


粘性路由:让请求继续回到热缓存

OpenRouter 通过 Sticky Routing,也就是粘性路由,解决供应商漂移问题。

当路由系统检测到某家供应商提供缓存读取折扣后,可以将后续相关请求继续发送到该供应商,而不是在不同节点之间自由切换。

更主动的做法,是显式传入会话 ID。

会话标识可以让路由系统从第一次成功请求开始,就识别哪些调用属于同一工作流,而无需等待缓存命中后再确定热节点的位置。

粘性路由有助于保持:

  • 提示缓存的本地性;

  • 对话上下文连续性;

  • 供应商特有状态;

  • 更稳定的响应延迟;

  • 更可预测的成本。

这使路由从单纯的供应商价格比较,升级为整条工作流的成本优化。

系统不应只问:

当前哪家供应商最便宜?

而应该问:

哪家供应商能够以最低综合成本完成整个会话?


高缓存命中率,会改变应用的成本结构

根据平台公布的数据,优化良好的应用,缓存命中率可能达到约 40%—60%。

这意味着,接近一半的 API 请求可能享受深度折扣后的缓存读取价格,或者直接复用已有响应。

实际节省比例仍取决于模型、缓存类型、写入价格、输出长度与路由稳定性。

在部分 Claude 工作流中,综合计算写入、读取和输出费用后,缓存命中请求的成本可能仅为普通调用的约 30%—50%。

团队之间的差距由此被迅速拉开。

一个应用完成可复用任务可能只需要 1 元。

另一个应用却可能花费 3 元、5 元甚至更多,因为它不断重复处理相同上下文、频繁错过缓存,或者在不同供应商之间漂移。

模型没有变化。

业务任务没有变化。

真正变化的是基础设施设计。


真正应该关注的,不是 Token 单价,而是重复付费

很多团队长期关注每百万 Token 价格,却忽略了自己为相同的 Token 重复支付了多少次。

一个每百万 Token 只收 1 美元、却不断重复处理相同上下文的模型,实际可能比一个定价 3 美元、但缓存利用率极高的模型更贵。

更完整的成本公式应该是:

总成本 = 新增工作 + 重复工作 + 失败工作 + 路由损耗

缓存主要减少重复工作。

粘性路由减少路由损耗。

重试保护则可以降低失败工作带来的浪费。

当这些环节得到优化后,模型目录中的 Token 单价,只是整张账单中的一个组成部分。


一套成熟的三层缓存策略

成熟的 AI 应用,可以把缓存分为三个层次。

第一层:响应缓存

适用于完全相同、可以安全复用的请求。

例如标准回答、固定测试、失败重试和确定性转换。

第二层:提示缓存

适用于会被大量请求共同使用的长篇稳定前缀。

例如企业文档、代码上下文、政策、合同和系统指令。

第三层:粘性路由

将相关请求持续发送到保存着有效缓存的供应商。

平台还应持续监测:

  • 缓存命中率;

  • 缓存写入频率;

  • 缓存读取节省金额;

  • 供应商切换次数;

  • TTL 过期情况;

  • 每条工作流节省的成本;

  • 每项成功任务的综合成本。

缓存不应只被看成一个 API 参数,而应被当成一套持续运营的成本系统。


Unexhub:从缓存降本,走向 Token 交易市场

缓存解决的是一个问题:

怎样让重复调用更便宜?

Unexhub 试图解决更进一步的问题:

怎样让 Token 算力被统一聚合、参与竞价,并像市场资源一样完成分发和结算?

Unexhub 正在建设 Token 聚合与分发的算力交易中心,用于连接主流模型生态、中小型 AI 供应商、企业用户与开发者。

平台覆盖或计划聚合 Gemini、ChatGPT、Claude、DeepSeek、GLM 等海内外主流模型服务,同时也为个人开发者、垂直模型服务商和中小型 AI 企业提供算力分发渠道。

其商业与基础设施逻辑可以分为三个层面。


第一层:按实际调用更快结算

传统 API 分销模式下,中小型供应商完成服务后,可能需要等待数周才能收到款项。

但 GPU、云资源、带宽、存储与运维成本仍然需要持续支付。

Unexhub 希望缩短成功调用与供应商收益结算之间的时间。

当请求完成、实际用量得到确认后,平台可以更快计算相应收益,并推进结算。

这能够帮助供应商:

  • 更快回收基础设施成本;

  • 提高现金流可见性;

  • 缓解营运资金压力;

  • 更快将收入投入算力扩容;

  • 提高释放闲置推理能力的意愿。

更准确地说,快速结算可以明显缓解资金压力,而不是让所有财务压力瞬间归零。


第二层:早期供应商激励

早期入驻的供应商,有机会享受最低 0 佣金的阶段性政策。

这一权益可以降低供应商进入 Token 分销网络的初始成本。

中小型供应商无需独立搭建完整的国际销售、计费、路由与清算体系,就能将自身算力接入平台,根据价格、能力和可用性获得相应流量。

其价值不只是少交一笔平台费。

它还让中小供应商能够以更低风险测试全球市场需求,并把闲置能力转化为实际收入。


第三层:API 竞价与动态定价

Unexhub 还在推进 API 竞价机制,让价格可以随着市场供需变化。

这套逻辑类似电力现货市场。

工作日白天,企业推理需求集中,可用算力价值上升。

深夜、周末或低利用率时段,大量算力可能处于闲置状态,供应商可以通过更低报价吸引请求。

对开发者来说,这意味着:

  • 将非紧急任务安排在更低价的时段;

  • 自动匹配成本更低的供应商;

  • 根据实际 Token 用量支付费用;

  • 减少预付套餐中的闲置资金;

  • 部分服务最低可至市场常规价格的约 3 折。

对供应商来说,则意味着:

  • 在调用高峰期获得更高收益;

  • 在低谷期通过优惠报价变现闲置能力;

  • 根据真实市场需求获得流量;

  • 凭借价格、速度与服务质量参与竞争。

配合毫秒级路由,平台可以在请求发生时,综合比较模型能力、当前报价、延迟、可用性与历史成功率,再选择更合适的供应商。

开发者看到的仍然只是一个 API。

在它背后,市场正在持续判断每一次请求应该发往哪里。


缓存与市场定价,优化的是账单的不同部分

缓存机制与 Token 交易市场并不冲突,而是互相补充。

缓存减少的是必须购买的工作量。

动态路由和竞价降低的,则是剩余工作量的采购价格。

最终成本可以被简化为:

最终成本 = 无法复用的新增工作 × 当前最优市场价格

响应缓存删除完全重复的推理。

提示缓存降低共享上下文的读取成本。

粘性路由确保热缓存不因供应商切换而丢失。

Unexhub 式竞价则为剩余推理寻找更具性价比的算力节点。

当这些机制叠加后,降本不再只是比较哪个模型的目录价格更低。

真正的问题变成:

哪些工作可以复用?剩余工作应该在什么时间运行?又应该由哪家供应商完成?


说到底

同样的 API 调用,有的团队可能比另一个团队多花数倍成本。

未必是因为模型选错了,而是因为工作流设计不同。

响应缓存让完全相同的请求直接复用结果,避免重复进行常规模型推理。

提示缓存让稳定不变的长上下文按照折扣价格反复读取。

粘性路由则防止请求在供应商之间漂移,导致原本的热缓存失效。

这三层机制结合起来,构成了 2026 年最重要的 AI 降本策略之一。

Unexhub 则指向下一层优化。

当 Token 算力可以从多家供应商统一聚合,在毫秒级完成路由,根据市场供需进行竞价,并按照实际使用量结算,AI 应用的成本结构将变得更加动态。

开发者与企业用户可以获得:

  • 统一接入海内外主流模型;

  • 根据实际 Token 消耗按量付费;

  • 在多家供应商之间毫秒级路由;

  • 为高频与闲时任务匹配更低价格;

  • 部分服务最低可至市场常规价格的约 3 折;

  • 降低多供应商带来的工程和财务管理成本。

供应商则可以获得:

  • 一键接入更广泛的分发网络;

  • 按真实调用量更快结算收益;

  • 早期最低 0 佣金的阶段性激励;

  • 根据能力自动获得流量;

  • 将闲置或未充分利用的算力转化为收入。

未来 AI 降本的胜负,不会只取决于谁的模型目录价格最低。

它将取决于三个问题:

有多少工作可以直接复用?
新的工作应该在什么时间执行?
哪家供应商能以最低综合成本完成它?

Token 的确正在变得越来越便宜。

但真正理解缓存、路由与市场时机的团队,仍然会比没有这些策略的团队少花得多。