
From Budget Guardrails to Dynamic Token Trading: How AI Cost Control Is Moving into the Runtime
Bilingual Article | English First, Chinese Version Below
For AI application developers, opening the monthly API bill can feel strangely similar to opening a mystery box.
The application may have executed the same workflow only a few times, yet the final cost can vary by an order of magnitude.
The problem is not always excessive usage.
More often, the application is using the wrong cost strategy: an oversized model, an expensive provider, an ineffective cache configuration, or an agent loop without clear spending limits.
Each mistake adds another layer to the bill.
Over the past six months, OpenRouter has introduced three important cost-control mechanisms:
Budget limits, intelligent routing, and two-layer caching.
Together, they move AI spending management away from the traditional model of “discovering the cost at the end of the month.”
The new model is more proactive:
Know the limit before execution, control spending during execution, and trace every cost after completion.
Budget Limits: Draw the Red Line Before Optimizing Efficiency
The first question in cost control should not be:
How can we make this request cheaper?
It should be:
How much are we willing to spend?
A model can be inexpensive per million Tokens and still create an expensive workflow if an agent continues running, repeatedly calls tools, or enters an unnecessary retry loop.
OpenRouter’s budgeting structure addresses this risk through several levels of control.
Workspace Budgets: The Top-Level Spending Boundary
Workspace Budgets establish the overall spending ceiling.
Teams can define limits based on:
Daily usage;
Weekly usage;
Monthly usage;
Lifetime usage.
Once the relevant limit is reached, OpenRouter can block additional requests and return a 403 response rather than allowing the application to continue spending.
A workspace can maintain several budget windows at the same time, provided they follow a logical hierarchy:
Lifetime budget > monthly budget > weekly budget > daily budget
This creates a hard financial boundary for the entire organization.
Even if an individual application, agent, or user behaves unexpectedly, the workspace cannot continue consuming unlimited credits.
Guardrails: Control Spending by Member or API Key
The second layer is Guardrails.
Instead of controlling only the total workspace budget, teams can apply limits to individual members, applications, or API keys.
For example, three developers on the same team may each receive a daily allowance of $50.
One member’s experimental agent cannot consume the budget reserved for the rest of the team.
API Key Budgets can add another layer of control.
A team member may have a personal spending limit, while the API key connected to a specific project has an even lower cap.
This helps separate:
Production workloads;
Development environments;
Evaluation jobs;
Internal tools;
Customer-facing applications.
The result is a layered budget structure:
Organization limit → member limit → application or key limit
If an agent begins looping, one project may stop—but the entire team budget remains protected.
Request-Level Price Ceilings: Refuse an Expensive Route
Budget controls can also be applied at the request level.
A parameter such as max_price can define the highest acceptable model or provider price for a request.
If no available supplier satisfies the requirement, the request fails instead of silently using a more expensive route.
This is particularly useful for workloads where cost is more important than immediate completion, such as:
Offline document processing;
Large-scale classification;
Batch summarization;
Model evaluation;
Data labeling;
Background research tasks.
For long-running agents, price ceilings alone are not enough.
A safer configuration combines:
A maximum number of steps;
A maximum Token allowance;
A maximum total cost;
A timeout;
A retry limit.
This prevents a simple task from turning into an expensive multi-hour execution loop.
Intelligent Routing: The Same Model Can Have a 10× Price Difference
Once spending limits are established, the next task is to reduce the cost of every permitted request.
OpenRouter aggregates more than 400 models across over 70 providers.
The same model can be offered by several suppliers at very different prices.
For example, the input price of Llama 3.3 70B may range from approximately $0.10 per million Tokens to more than $1, depending on the hosting provider.
The model name is identical.
The underlying infrastructure, regional cost, hardware efficiency, available capacity, and service quality are not.
This creates a substantial routing opportunity.
Default Routing Already Considers Price
OpenRouter’s default routing logic can evaluate several provider factors, including:
Price;
Availability;
Latency;
Throughput;
Context support;
Feature compatibility;
Historical reliability.
Where multiple providers satisfy the request, the platform can favor more economical routes.
Developers therefore do not need to manually integrate and compare dozens of provider APIs.
The routing layer performs that comparison behind one compatible interface.
:floor: Prioritize the Lowest-Priced Provider
For workloads in which price is the dominant priority, developers can use a routing option such as :floor.
For example:
meta-llama/llama-3.3-70b-instruct:floor
This instructs the platform to prioritize the lowest-priced provider currently available for that model.
The approach is particularly suitable for:
Batch jobs;
Offline processing;
Large-scale testing;
Non-urgent internal tasks;
Workloads that can tolerate occasional retries.
For tool-calling requests, OpenRouter may first group providers according to tool-use quality. Price ordering is then applied within the same quality tier.
Using :floor can restore a stronger price preference.
However, the cheapest endpoint is not always the most reliable endpoint.
A strict floor strategy may reduce load balancing, concentrate traffic on one supplier, or select a node with limited capacity during busy periods.
For real-time customer requests, teams still need to balance:
Price, latency, quality, and availability.
The lowest listed price is useful only when the request can still be completed successfully.
Response Caching: Identical Requests Can Avoid New Inference
On April 30, OpenRouter introduced Response Caching.
Once enabled, the platform creates a cache key based on elements such as:
Request body;
Selected model;
API credential context;
Streaming configuration.
The first request is processed normally by the provider and billed according to actual usage.
If a later request is completely identical and the cache remains valid, OpenRouter can return the stored response directly from its edge cache.
The underlying provider does not need to execute the request again.
No new model inference is required, and normal Token-generation costs can be avoided for that cache hit.
In practical terms:
Pay for the first answer. Reuse identical answers while the cache remains valid.
Cache Hits Are Not Only Cheaper, but Faster
A standard model request may require several seconds.
A cached response can return in approximately 80–300 milliseconds, while the cache lookup itself may take only a few milliseconds.
For comparison, uncached responses from different models may take:
Around 1.3 seconds for a fast Flash-class model;
Several seconds for a larger model;
Close to ten seconds for a deeper reasoning model.
A cache hit can therefore improve response time by an order of magnitude.
The default cache TTL is approximately five minutes, while developers can adjust it from seconds to as long as 24 hours, depending on the workflow.
A cache-control header can also be used to force a refresh, while the response can indicate whether the request produced a cache HIT or MISS.
Where Response Caching Delivers the Most Value
Agent Workflow Retries
Suppose an agent completes four steps and fails at the fifth.
Without caching, restarting the workflow may repeat and rebill the first four steps.
With response caching, identical completed steps can return immediately. The team pays primarily for the new work.
Repeated LLM Test Suites
The first evaluation run fills the cache.
Subsequent identical runs can reuse the same outputs while developers test changes in surrounding code, interface logic, or evaluation tooling.
Duplicate Batch Inputs
Large datasets often contain repeated or near-duplicate tasks.
When the complete request matches, response caching prevents unnecessary inference.
Standardized Answers and Transformations
Frequently asked questions, standard policy responses, fixed classifications, and deterministic transformations can often be reused safely.
Text, images, audio, documents, and tool-related requests may all benefit when the complete request is reproducible.
Very large multimodal payloads may remain outside the caching system because of different internal processing requirements.
Prompt Caching: Shared Context at a Deep Discount
Response Caching requires an identical request.
Most AI applications, however, reuse only part of the prompt.
The system instructions remain stable. The company handbook remains stable. The codebase context remains stable. Only the latest user message changes.
Prompt Caching is designed for this situation.
It stores the stable prefix of a request so the provider does not need to process that same long context at the full input price every time.
Reusable prompt content may include:
System instructions;
Product documentation;
Enterprise policies;
Codebase summaries;
Contract templates;
Tool specifications;
Long conversation history;
Knowledge-base context.
The new user input is still processed normally, but the shared prefix can receive a deeply discounted cache-read rate.
Cache-Read Discounts Vary by Provider
Representative cache-read pricing may look like this:
| Provider or model family | Typical cached-read rate |
|---|---|
| Anthropic Claude Sonnet series | Approximately 0.1× the standard input price |
| DeepSeek and Alibaba Qwen | Approximately 0.1× in supported scenarios |
| OpenAI | Commonly around 0.25×–0.5×, depending on the model |
For example, if standard input costs $3 per million Tokens and cached reads cost $0.30, the stable context is read at one-tenth of the normal price.
For an application that repeatedly sends a 100,000-Token company manual, this difference can become substantial.
The first request creates or populates the cache.
Thousands of later requests can reuse the document at a discounted rate while paying full price only for the new question and output.
Sticky Routing: Keep Requests on the Provider with the Warm Cache
Prompt caches are usually provider-local.
A cache created by Provider A may not exist at Provider B.
If a multi-turn conversation is routed to a different supplier on the next request, the application can lose the warm cache and pay the full context-processing price again.
This is known as provider drift.
OpenRouter uses Provider Sticky Routing to reduce that problem.
When the platform detects that a provider offers a discounted cached-read rate, later related requests can be routed back to the same provider.
A more deliberate method is to send an explicit session identifier.
This allows the routing system to associate requests with the same workflow from the first successful call.
Sticky Routing can help preserve:
Cache locality;
Conversation continuity;
More predictable latency;
Provider-specific state;
Lower total workflow cost.
The important question is no longer:
Which provider is cheapest for this one request?
It becomes:
Which provider can complete the entire session at the lowest total cost?
A 40%–60% Cache-Hit Rate Can Reshape the Bill
According to OpenRouter data, a well-optimized application may achieve cache-hit rates of approximately 40%–60%.
That means a significant portion of requests may receive:
A deeply discounted cached-input rate;
Or a fully reused response without new model inference.
After accounting for output costs and cache-write expenses, the final savings depend on the model and workflow.
But in suitable applications, cache-hit requests may cost 60%–80% less than repeatedly processing the same input at the standard rate.
This explains why two teams using the same model can see very different bills.
One team pays full price for every request.
Another team:
Blocks uncontrolled spending;
Selects a lower-cost provider;
Reuses identical responses;
Discounts shared context;
Keeps requests attached to the warm supplier.
For one team, a workflow costs $3.
For another, the effective cost may fall below $0.30.
The difference is not necessarily model intelligence.
It is infrastructure discipline.
OpenRouter Creates a Cost-Control Loop
OpenRouter’s three mechanisms form a relatively complete operating loop.
Before Execution: Set the Boundary
Workspace Budgets, member Guardrails, API Key Budgets, and request-level price ceilings define how much the system is allowed to spend.
During Execution: Select the Route
Provider comparison and routing options help direct requests toward a more economical supplier without requiring separate integrations.
During Reuse: Reduce Repeated Work
Response Caching eliminates repeated inference for identical requests. Prompt Caching discounts stable context. Sticky Routing protects the warm cache.
After Execution: Trace the Cost
Usage data can be associated with the relevant API key, team member, project, model, and provider.
This moves cost control from retrospective accounting into runtime operations.
Unexhub: When Token Becomes a Market Object
OpenRouter focuses on buying and using existing model capacity more efficiently.
Budget controls, routing, and caching can push the cost of an application significantly lower.
However, the underlying Token price is still usually determined by the supplier’s published rate.
Unexhub is pursuing another layer of the market.
It is developing a Token aggregation and distribution marketplace that connects major model ecosystems, specialized AI suppliers, enterprise customers, and developers.
The platform covers or plans to aggregate services from ecosystems such as Gemini, ChatGPT, Claude, DeepSeek, and GLM.
It also creates distribution opportunities for individual developers and small or medium-sized AI companies with quality models or available inference capacity.
Its structure can be understood through three mechanisms.
Faster Settlement and Clearing
Under traditional distribution models, a smaller supplier may complete substantial API traffic but wait weeks before receiving payment.
Meanwhile, GPU, cloud, bandwidth, storage, engineering, and operational expenses continue accumulating.
Unexhub aims to shorten the interval between successful usage and supplier settlement.
Once a request has been completed and its Token consumption verified, the supplier’s revenue share can be calculated and settled more quickly.
This may help suppliers:
Recover infrastructure costs sooner;
Improve cash-flow visibility;
Reduce working-capital pressure;
Reinvest more quickly in additional capacity;
Release idle inference resources to the market.
Faster settlement does not make every financial burden disappear.
Its practical value is improving capital turnover and shortening the path from completed inference to realized revenue.
Early Supplier Incentives
Early suppliers may qualify for promotional commission rates as low as 0%.
This can reduce the initial cost of entering a global Token distribution network.
Instead of independently building customer acquisition, account management, billing, routing, and settlement systems, smaller providers can connect their capacity to the platform.
They may then receive suitable traffic according to:
Price;
Model capability;
Latency;
Availability;
Historical service quality.
The value extends beyond saving a platform fee.
It creates a lower-risk path for suppliers to test market demand and monetize idle or specialized capacity.
API Bidding: Let Prices Respond to the Market
Unexhub is also developing an API bidding model in which prices can move according to supply and demand.
The logic resembles an electricity spot market.
During weekday business hours, enterprise AI demand may rise and available inference capacity may become more valuable.
During nights, weekends, and low-utilization periods, suppliers may lower their offers to attract additional workloads.
For developers, this can provide several practical benefits:
Move non-urgent jobs to lower-priced periods;
Automatically select more economical suppliers;
Pay according to actual Token consumption;
Reduce unused prepaid balances;
Balance price, latency, quality, and reliability;
Access selected services at prices as low as approximately 30% of typical market rates.
For suppliers, dynamic pricing may allow them to:
Earn more during peak demand;
Monetize idle capacity during off-peak periods;
Receive traffic based on current market needs;
Compete through price, speed, specialization, and service quality.
With millisecond-level routing, the platform can evaluate available suppliers and select an appropriate path without requiring developers to compare quotations manually.
The user sees one API request.
Behind it, the market continuously decides where that request should run.
Caching and Market Pricing Optimize Different Parts of the Cost
OpenRouter-style caching and Unexhub-style market mechanisms are complementary.
Caching reduces the amount of work that must be purchased.
Dynamic routing and bidding reduce the price of the work that remains.
The cost equation can therefore be reframed as:
Final cost = Necessary new work × Best available execution price
Response Caching removes identical repeated work.
Prompt Caching reduces the cost of reusable context.
Sticky Routing protects the warm cache.
Budget Guardrails prevent runaway execution.
Provider routing selects a more economical endpoint.
Market-based bidding can further reduce the cost of the remaining inference.
Together, these mechanisms shift the question from:
Which model is cheapest?
to:
Which work can be reused, which work must be performed, when should it run, and which supplier should complete it?
Final Takeaway
AI cost control is no longer a month-end accounting exercise.
A mature system should know the spending boundary before execution, manage the route during execution, reuse completed work whenever possible, and record the cost afterward.
OpenRouter’s three major mechanisms create that operating loop:
Budget limits draw the red line.
Intelligent routing selects a more economical supplier.
Response and Prompt Caching reduce the price of repeated work.
When these tools are combined, the same API workflow may cost one team $3 and another less than $0.30.
The gap does not necessarily come from the model.
It comes from whether cost controls are built into the application.
Unexhub points toward the next stage.
When Token capacity can be aggregated across suppliers, routed in milliseconds, priced according to market conditions, and settled according to actual usage, cost optimization becomes a runtime market mechanism rather than a manual afterthought.
For developers and enterprise users, this can mean:
Unified access to major domestic and international models;
Pay-as-you-go Token billing;
Dynamic supplier selection;
Lower costs for batch and off-peak workloads;
Selected services available at approximately 30% of typical market rates;
Less engineering and financial overhead from multi-provider management.
For suppliers, it can mean:
One-step access to a wider distribution network;
Faster usage-based settlement;
Early-stage commission rates as low as 0%;
Automated traffic allocation;
A way to monetize idle or specialized inference capacity.
The future of AI budgeting will not be defined by who discovers the cheapest model after receiving the bill.
It will be defined by who can automatically make four decisions before and during every request:
How much can be spent?
Which provider should execute it?
Which work can be reused?
What is the best available market price right now?
That is how Token cost control moves from a financial report into the runtime itself.
中文版|English Version Above
本文为英中双语文章。以上为英文全文,以下为完整中文版本。
从预算红线到 Token 动态竞价:AI 成本控制正在被写进运行时
做 AI 应用开发的人,大概都体验过一种微妙的“账单眩晕”。
月底打开 API 后台,明明只运行了几次看起来完全相同的流程,最终费用却可能相差数倍,甚至直接差出一个数量级。
问题未必是用量太大。
更多时候,是成本策略没有设计好:模型能力过剩、供应商报价偏高、缓存没有正确命中,或者智能体缺少明确的步数与预算限制。
每一个小问题,都会在账单上继续叠加。
过去半年,OpenRouter 围绕成本管理连续补齐了三类重要能力:
预算上限、智能路由与双层缓存。
三套机制结合后,AI 成本控制不再是“月底打开账单,才知道花了多少”。
它开始变成一套更加主动的流程:
运行前知道红线,运行中持续控制,运行后能够追溯。
预算上限:先画红线,再讨论怎么省
成本管理的第一个问题,不应该是:
怎样让这次请求更便宜?
而应该是:
这项任务最多允许花多少钱?
一个模型的每百万 Token 单价可能很低,但如果智能体不断调用工具、重复重试,或者进入没有终止条件的循环,最终仍然会形成一笔高额账单。
OpenRouter 的预算体系,正是用多层控制降低这类风险。
Workspace Budgets:工作区的总预算边界
Workspace Budgets 用于设置整个工作区的最高支出。
团队可以按照不同周期制定美元限额:
每日预算;
每周预算;
每月预算;
终身预算。
当对应预算达到上限后,OpenRouter 可以直接拦截后续请求,并返回 403,而不是让应用继续消耗余额。
一个工作区可以同时配置多个预算周期,但需要保持合理的层级关系:
终身预算 > 月度预算 > 周度预算 > 日度预算
这一层相当于组织级别的财务红线。
即使某个应用、智能体或成员出现异常行为,也无法无限消耗整个团队的 credits。
Guardrails:把预算细分到成员与 API Key
第二层是 Guardrails。
除了控制整个工作区的总支出,团队还可以为不同成员、项目和 API Key 配置单独限制。
例如,同一团队中的三名开发人员,每人每天可以拥有 50 美元额度。
其中一名成员运行实验性智能体时,即使任务出现异常,也不会直接占用其他成员的预算。
API Key Budgets 还能在成员限额基础上继续细分。
同一个成员可能拥有较高的个人额度,但连接某个测试项目的 API Key,可以设置更低的预算上限。
这样便可以将不同工作负载隔离开来:
生产环境;
开发环境;
模型评测;
内部工具;
面向客户的应用。
最终形成的是一套层层收紧的预算结构:
组织预算 → 成员预算 → 项目或 API Key 预算
即使某个智能体开始循环,也只会让对应项目停下来,而不会拖累整个团队。
请求级价格上限:没有合适报价,就直接失败
预算还可以被控制到单次请求层面。
通过类似 max_price 的参数,开发者可以为请求设置可接受的最高价格。
如果当前没有任何供应商满足条件,请求可以直接失败,而不是静默切换到一个更昂贵的节点。
这种机制特别适合成本优先、并不要求立刻完成的工作:
离线文档处理;
大规模分类;
批量摘要;
模型评测;
数据标注;
后台研究任务。
对于长时间运行的智能体,仅设置价格上限仍然不够。
更安全的方案,是同时配置:
最大执行步数;
最大 Token 用量;
最大总费用;
超时时间;
重试次数。
这样可以防止一个简单任务逐渐演变成长达数小时的高成本循环。
智能路由:同一个模型,供应商报价可相差 10 倍
预算红线画好之后,下一步才是让每一次被允许的请求花得更少。
OpenRouter 已经聚合 400 多个模型和 70 多家供应商。
同一款模型在不同托管商处,价格可能存在非常明显的差异。
以 Llama 3.3 70B 为例,不同供应商的输入价格可能从每百万 Token 约 0.10 美元,到超过 1 美元不等。
模型名称完全相同。
但底层硬件效率、区域成本、可用算力、节点负载和服务质量并不相同。
这就为智能路由留下了较大的优化空间。
默认路由已经会考虑价格
OpenRouter 的默认路由可以综合比较多个供应商维度,例如:
调用价格;
服务可用性;
响应延迟;
处理吞吐;
上下文能力;
功能兼容性;
历史稳定性。
当多家供应商都能满足请求时,平台可以优先选择更具成本优势的路径。
开发者不需要分别接入和维护几十家供应商接口。
这些比较和切换,都可以被隐藏在一套兼容 API 背后。
:floor:把最低价放到最高优先级
对于价格优先的任务,可以使用类似 :floor 的路由方式。
例如:
meta-llama/llama-3.3-70b-instruct:floor
这会让平台优先选择当前提供该模型的最低价供应商。
它更加适合:
批量任务;
离线处理;
大规模测试;
非紧急内部工作;
可以容忍少量重试的场景。
对于工具调用请求,OpenRouter 可能会先根据工具调用质量对供应商进行分层,再在同一质量层级中按照价格排序。
加入 :floor,可以进一步提高价格权重。
但最低价并不永远等于最优路线。
严格锁定最低价供应商,可能削弱负载均衡,也可能在高峰时段选中容量有限、稳定性较低的节点。
对于批处理任务,这通常可以接受。
对于实时面向用户的业务,则仍然需要在以下指标之间权衡:
价格、延迟、质量和可用性。
只有请求能够稳定完成,低价才真正有价值。
响应缓存:相同请求无需重复推理
4 月 30 日,OpenRouter 推出了 Response Caching,也就是响应缓存。
开启后,平台会根据多项内容生成缓存键,例如:
请求体;
所选模型;
API 鉴权环境;
流式模式。
第一次请求仍然会正常调用供应商,并根据真实用量计费。
在缓存有效期内,如果后续请求与第一次完全相同,OpenRouter 可以直接从边缘缓存返回已经生成的完整响应。
底层模型无需再次执行。
本次缓存命中也不需要产生新的常规模型推理成本。
简单来说:
第一次为答案付费,后续相同请求直接复用。
缓存命中不仅更便宜,也更快
普通模型请求可能需要数秒才能返回。
缓存命中的响应,通常可以在约 80—300 毫秒内返回,而缓存查找本身可能只需要几毫秒。
相比之下,不同模型的未缓存请求可能需要:
快速 Flash 级模型约 1 秒;
较大型模型数秒;
深度推理模型接近 10 秒。
命中缓存后,响应速度可以提高一个数量级。
缓存默认 TTL 约为 5 分钟,开发者可以根据业务特点,将其调整为几秒到最长 24 小时。
还可以通过相应请求头强制刷新缓存,并从响应头中查看此次请求是缓存 HIT 还是 MISS。
响应缓存最适合哪些任务?
智能体失败后的重试
假设一个智能体已经完成前四步,在第五步失败。
没有缓存时,重新启动任务可能意味着前四步全部重新执行并再次计费。
使用响应缓存后,相同步骤可以快速返回,团队主要为尚未完成的新工作支付费用。
LLM 测试套件重复运行
第一次评测负责填充缓存。
后续相同请求可以复用原有输出,开发者只需要验证外围代码、接口或评测工具是否仍然正常。
批量任务中的重复输入
大规模数据集中经常包含重复任务。
当整个请求完全一致时,响应缓存可以避免重复推理。
标准回答与固定转换
常见问题、标准政策说明、固定分类、模板化输出和确定性较强的转换,都适合使用响应缓存。
只要完整请求可以稳定复现,文本、图像、音频、文档和部分工具调用都可能被缓存。
体积特别大的多模态负载,则可能由于内部处理机制不同而暂不支持。
提示缓存:共享上下文,读取价最低可至 1 折
响应缓存要求整个请求完全一致。
但现实中的多数 AI 应用,只会重复使用部分内容。
系统提示词不变,企业手册不变,代码库上下文不变,只有用户提出的新问题发生变化。
这正是 Prompt Caching,也就是提示缓存发挥作用的场景。
提示缓存会保存请求中稳定不变的前缀,使供应商无需每次都按照完整输入价格重新处理同一段长上下文。
可复用的内容包括:
系统指令;
产品说明;
企业政策;
代码库摘要;
合同模板;
工具规范;
长对话历史;
知识库资料。
用户的新问题仍然正常处理,而稳定的共享前缀则可以使用更低的缓存读取价格。
不同厂商的缓存读取折扣
具有代表性的缓存读取价格关系包括:
| 厂商或模型体系 | 常见缓存读取价格 |
|---|---|
| Anthropic Claude Sonnet 系列 | 约为标准输入价格的 0.1 倍 |
| DeepSeek、阿里通义千问 | 支持场景下约为 0.1 倍 |
| OpenAI | 根据模型和规则,常见约为 0.25—0.5 倍 |
假设标准输入价格为每百万 Token 3 美元,缓存读取价格为 0.30 美元,那么重复读取稳定上下文的费用只有原来的十分之一。
对于每次请求都需要携带 10 万 Token 企业手册的应用,这项折扣会非常明显。
第一次请求完成缓存写入。
之后成千上万次请求,都可以低价复用这份文档,只为新问题和模型输出支付正常费用。
粘性路由:别让热缓存因为换供应商而失效
提示缓存通常保存在供应商本地。
供应商 A 创建的缓存,未必能够被供应商 B 读取。
如果多轮对话的下一次请求被调度到另一家供应商,应用就可能丢失已有热缓存,再次按全价处理长上下文。
这就是供应商漂移。
OpenRouter 通过 Provider Sticky Routing,也就是供应商粘性路由,降低这一问题。
当平台检测到某个供应商提供较低的缓存读取价格后,后续相关请求可以继续被路由回该节点。
更主动的做法,是显式传入会话 ID。
这样,路由系统从第一次成功请求开始,就能识别哪些调用属于同一条工作流。
粘性路由有助于保持:
缓存本地性;
对话上下文连续性;
更稳定的响应延迟;
供应商特有状态;
更低的工作流综合成本。
真正应该比较的,不是:
当前这一轮哪家供应商最便宜?
而是:
哪家供应商能够以最低总成本完成整个会话?
40%—60% 的缓存命中率,足以重构账单
根据 OpenRouter 数据,优化良好的应用,缓存命中率可以达到约 40%—60%。
这意味着相当一部分请求能够获得:
深度折扣后的缓存输入价格;
或者直接复用已有响应,无需再次执行模型推理。
最终能够节省多少,仍然取决于模型、缓存写入价格、输出长度和工作流设计。
但在适合的场景下,缓存命中请求的实际费用可能比反复按标准价格处理相同输入低 60%—80%。
这也解释了,为什么两个团队使用同一款模型,账单却完全不同。
一个团队每次都支付全价。
另一个团队则会:
先限制最大支出;
再选择价格更低的供应商;
复用完全相同的结果;
低价读取共享上下文;
让请求持续回到拥有热缓存的节点。
同一套流程,一个团队可能花 3 元。
另一个团队的实际成本,则可能低于 3 毛。
差距不一定来自模型能力。
更可能来自基础设施纪律。
OpenRouter 构成了一套完整的成本控制闭环
OpenRouter 的三类能力,可以形成一套相对完整的运营闭环。
运行之前:先确定红线
Workspace Budgets、成员 Guardrails、API Key Budgets 和请求级价格上限,共同决定系统最多可以花多少钱。
运行过程中:选择合适路线
供应商比较与智能路由,在无需维护多套接口的情况下,为请求匹配更具经济性的节点。
重复调用时:减少重复工作
响应缓存消除相同请求的重复推理,提示缓存降低共享上下文费用,粘性路由负责保护热缓存。
运行结束后:追踪费用来源
用量可以继续关联到对应的 API Key、成员、项目、模型与供应商。
AI 成本控制因此从月底核算,进入了应用运行时。
Unexhub:当 Token 从计价单位变成交易对象
OpenRouter 解决的是如何更经济地购买和使用现有模型算力。
预算、路由和缓存结合后,可以显著降低应用成本。
但底层 Token 价格通常仍然由供应商提前公布。
Unexhub 正在推进另一层市场机制。
它希望建设一个 Token 聚合与分发的算力交易中心,连接主流模型生态、专业 AI 供应商、企业用户和开发者。
平台覆盖或计划聚合 Gemini、ChatGPT、Claude、DeepSeek、GLM 等海内外主流模型,同时也为个人开发者和中小型 AI 企业提供优质模型与推理能力的分发渠道。
其核心设计可以分为三个部分。
更快完成收益结算
传统分销模式下,中小型供应商即使已经完成大量 API 调用,也可能需要等待数周才能收到款项。
与此同时,GPU、云资源、带宽、存储、工程和运营支出仍在持续发生。
Unexhub 希望缩短成功调用与供应商收益到账之间的时间。
当请求完成、Token 用量得到确认后,平台可以更快计算供应商应得收益,并推进结算。
这能够帮助供应商:
更快回收基础设施成本;
提高现金流可见性;
缓解营运资金压力;
更快将收入用于算力扩容;
提高释放闲置推理资源的意愿。
即时或更快结算并不意味着所有财务压力都会凭空消失。
它真正改善的,是从完成服务到获得收入之间的资金周转效率。
早期供应商激励
早期接入平台的供应商,有机会享受最低 0 佣金的阶段性权益。
这可以降低进入全球 Token 分销网络的初始成本。
中小型供应商无需自行建设完整的获客、账号、账单、路由与结算体系,就可以将自身模型能力接入平台。
平台可以根据以下指标为其匹配流量:
报价;
模型能力;
响应延迟;
服务可用性;
历史质量表现。
这项权益的价值,并不只是减少一笔平台费用。
它为中小供应商提供了一条低风险验证市场需求,并将闲置或专业算力转化为收入的路径。
API 竞价:让价格跟随市场供需变化
Unexhub 还在推进 API 竞价机制,使模型调用价格可以根据供需状态进行变化。
其逻辑类似电力现货市场。
工作日白天,企业 AI 调用需求集中,可用推理能力的价值可能上升。
深夜、周末和低利用率时段,供应商则可以通过更低报价,吸引额外工作负载。
对开发者而言,这可以带来以下具体权益:
将非紧急任务安排在更低价时段;
自动选择更具成本优势的供应商;
按实际 Token 用量付费;
减少闲置预充值;
在价格、速度、质量和可靠性之间灵活平衡;
部分服务最低可至市场常规价格的约 3 折。
对供应商而言,则可以:
在需求高峰期获得更高收益;
在低谷期变现闲置算力;
根据真实市场需求获得调用流量;
依靠价格、速度、专业能力和服务质量参与竞争。
结合毫秒级路由,平台可以自动比较当前可用供应商,并完成请求匹配,不需要开发者手动查看每一家报价。
用户看到的是一次 API 请求。
平台背后则在持续判断,这次请求应该运行在哪里。
缓存与市场定价,优化的是成本的不同部分
OpenRouter 式缓存与 Unexhub 式市场机制并不冲突,而是互相补充。
缓存减少的是必须购买的工作量。
动态路由和竞价降低的,则是剩余工作量的执行价格。
成本公式可以被重新表达为:
最终成本 = 必须完成的新增工作 × 当前最优执行价格
响应缓存去掉完全重复的工作。
提示缓存降低共享上下文的成本。
粘性路由负责保护热缓存。
预算限制阻止任务失控。
供应商路由选择更经济的节点。
市场竞价则进一步压低剩余推理任务的采购价格。
降本问题也因此从:
哪个模型最便宜?
变成:
哪些工作可以复用、哪些工作必须执行、应该在什么时间执行,又应该交给哪家供应商?
说到底
AI 成本控制已经不再是一项月底对账工作。
成熟的系统应该在任务开始前明确支出红线,在运行过程中管理供应商路线,尽可能复用已经完成的工作,并在结束后清楚追踪费用来源。
OpenRouter 的三张牌,构成了这套运行闭环:
预算上限先画红线;
智能路由选择更具性价比的供应商;
响应缓存和提示缓存降低重复工作的成本。
三套机制结合后,同一套 API 流程,一个团队可能花 3 元,另一个团队却可以把有效成本压到 3 毛以内。
差距不一定来自模型。
更可能来自成本控制是否已经被写进应用。
Unexhub 则指向下一阶段。
当 Token 算力可以被多家供应商统一聚合,在毫秒级时间内完成路由,根据市场供需变化进行定价,并按照真实用量完成结算,成本优化就不再是月底补救,而会成为运行时自动执行的市场机制。
对开发者和企业用户而言,这意味着:
统一接入海内外主流模型;
按实际 Token 消耗付费;
在不同供应商之间动态选择;
为批量任务和闲时任务匹配更低成本;
部分服务最低可至市场常规价格的约 3 折;
减少多供应商带来的工程与财务管理负担。
对供应商而言,则意味着:
一键进入更广泛的分发网络;
根据真实调用量更快结算;
早期最低 0 佣金的阶段性激励;
根据能力自动获得流量;
将闲置或专业推理能力转化为收入。
未来 AI 预算管理的竞争,不会取决于谁在收到账单后找到最便宜的模型。
而会取决于谁能在每一次请求发出之前和执行过程中,自动做出四个判断:
最多允许花多少钱?
应该交给哪家供应商?
哪些工作可以直接复用?
当前市场上的最优价格是多少?
这才是 Token 成本控制从财务报表走向运行时的真正意义。