NVIDIA 开源路由库让 AI 成本暴降 74%!Token 交易市场迎来"智能调度"新基建

让 AI 算力更容易流动,让 Token 更容易交易,让价格更透明,也让每一份算力都能够流向真正需要它的地方。


I Finally Did the Math on the "Overkill Model" Problem

On August 11, 2026, NVIDIA did something that looked contradictory at first glance.

It released Nemotron 3.5 Lightning, a 30-billion-parameter MoE model capable of generating output at up to four times the speed of comparable models. At the same time, NVIDIA open-sourced NeMo Switchyard, a model-routing library.

Put the two together, and the message is surprisingly clear:

The next stage of AI competition is moving from "who has the biggest model?" to "who can route models most efficiently?"

NVIDIA isn't trying to steal the model championship from OpenAI. It is addressing a much more practical enterprise problem:

The more agents you run, the faster your AI bill grows.


7% of Frontier Calls Consumed 68% of the Budget

LangChain tested 145 multi-turn deep-agent tasks using its Deep Agents evaluation suite.

When routing between Nemotron 3.5 Lightning and Claude Opus 4.8, only 7% of calls were assigned to the frontier model.

Yet those 7% of calls accounted for 68% of the total cost.

The remaining 93%—code reviews, tool calls, security-alert monitoring, billing questions, and similar workloads—were handled by the lightweight 30B model.

The result was striking:

  • 74% lower total cost

  • Accuracy declined from 86% to 80%

  • The system retained roughly 93% of the original accuracy level

Ramp reached a similar conclusion in its own SWE-Bench testing: costs fell by 58% while execution time dropped by 33%.

After Cognition integrated Switchyard's stage router into Devin Desktop, average costs were reportedly 28% lower than using a single frontier model.

The 74% figure isn't magic.

It's simple arithmetic:

Most tasks don't require the smartest model—but most teams have been assigning every task to the smartest model anyway.


So, What Exactly Does Switchyard Do?

NeMo Switchyard is essentially a Rust-based proxy positioned between an inference-server API and the models themselves.

It isn't another AI model.

It's the system deciding which model should handle each task.

Switchyard provides several routing strategies.

1. LLM Classifier Mode

A smaller model evaluates the request and helps determine which model is appropriate.

2. Escalation Mode

Start with a cheaper model. If it repeatedly fails to perform adequately, automatically escalate the task to a more capable model.

3. Stage Router

Route requests according to different stages of an agent workflow without introducing additional model calls.

The critical idea is that the system optimizes for total task cost, not simply the price per token.

A model may cost 10 times less per token, but if it consumes 10 times more tokens to complete the same task, it isn't necessarily cheaper.

Switchyard looks at request characteristics, model capabilities, and system conditions before deciding:

Does this task actually deserve the most expensive model?


The Ripple Effect of an Open-Source Routing Layer

The most interesting part of Switchyard isn't necessarily the routing technology itself.

It's the ecosystem position it occupies.

Existing AI gateways and routing platforms such as Kong, OpenRouter, and LiteLLM are already moving toward multi-model routing architectures.

NVIDIA's senior director Joey Conway summarized the direction clearly: multi-model collaborative systems are becoming an important part of the future of AI.

And NVIDIA doesn't necessarily need Nemotron to beat GPT-5.

It needs more companies to deploy AI.

Any AI.

Because every additional AI deployment ultimately consumes computing infrastructure.

Open models lower the entry barrier, expand the number of AI applications, and therefore expand the market for NVIDIA's core business: chips and infrastructure.

Nemotron + Switchyard is essentially a bet on the "second phase" of AI.

Not a model-centric ecosystem.

An infrastructure-centric ecosystem.

Whoever can efficiently orchestrate multiple models, route tasks intelligently, and reduce costs without sacrificing too much capability may ultimately gain a powerful position in the next generation of AI economics.


Unexhub's Calculation: Making Tokens Trade Like Electricity

This is where Unexhub's approach becomes particularly interesting.

Unexhub is building a token aggregation, distribution, and computing-power trading center, with the goal of turning tokens from an internal technical resource into a tradable market commodity.

Its model focuses on three areas.

Real-Time Settlement

Small and medium-sized AI suppliers can receive settlement based on the tokens they actually provide and the traffic they generate.

Instead of waiting through long payment cycles, suppliers can recover funds in near real time, significantly reducing cash-flow pressure.

Zero-Commission Incentives

Early suppliers can benefit from 0% platform commission, giving them a lower-cost way to access additional distribution channels while helping build marketplace liquidity.

The point isn't simply to offer a discount.

It is to encourage more suppliers to participate, increasing the diversity and availability of AI capacity in the marketplace.

Dynamic API Bidding

Instead of relying entirely on fixed API pricing, Unexhub introduces a market-oriented pricing mechanism in which API prices can respond dynamically to supply and demand.

When demand is high, suppliers can earn more.

When capacity is abundant, lower prices can make AI services more accessible to developers.

The result is a more flexible relationship between computing supply, token consumption, and price.


Switchyard Answers "Which Model?"

Unexhub Answers "How Much Should It Cost?"

This distinction matters.

Switchyard solves the model-selection problem.

Unexhub aims to solve the market-pricing problem.

One is technical orchestration.

The other is economic coordination.

Put together, they point toward a new architecture for AI inference:

Intelligently route the task. Dynamically price the capacity. Pay for what you actually use.

For developers and enterprises, that can translate into a more flexible way to control inference costs instead of locking every workload to the same expensive model.


From "More Models" to "Better Cost Accounting"

The winning strategy for enterprise AI is undergoing a fundamental change.

It is no longer:

"Use the best model for everything."

It is becoming:

"Use the right model for every step—and switch intelligently when the task changes."

Among LangChain's 145 tasks, only 7% of calls required a frontier model, yet those calls consumed 68% of the budget.

The underlying lesson is simple:

A large portion of enterprise AI spending may come from using expensive intelligence where cheaper intelligence would have been sufficient.

Over the next 12 to 24 months, competitive advantage may come less from simply having access to a better model and more from using multiple models more intelligently.

And making multiple models work efficiently requires two layers of infrastructure:

Routing + Trading.

NVIDIA is opening up the routing layer.

Unexhub is building toward the trading layer.

Together, they point toward an AI inference market where model selection is no longer based purely on intuition:

Tasks are routed according to requirements, while computing resources are priced according to market conditions.

That is the beginning of AI's "broadband era."

And 74% isn't the finish line.

It's only the beginning.


Unexhub: Turning AI Capacity Into a More Flexible Marketplace

Unexhub is positioning itself as a global token aggregation and distribution platform and computing-power trading center, bringing together AI suppliers of different sizes.

Its ecosystem is designed to include major models such as Gemini, ChatGPT, Claude, DeepSeek, and GLM, alongside AI services provided by individual developers and smaller companies.

The idea is straightforward: aggregate more supply, introduce competition among providers, and allow enterprises and developers to access AI capacity through a more flexible marketplace.

For developers, Unexhub promotes token-based actual usage billing, with eligible services potentially reaching as low as 30% of prevailing market prices, depending on the model and market conditions.

That means the potential benefit isn't simply a lower headline price. Developers can gain access to:

  • Multi-model aggregation instead of maintaining separate integrations

  • Millisecond-level routing for faster model selection and traffic distribution

  • Competitive pricing through supplier-side competition

  • Pay-by-token usage rather than unnecessary fixed capacity

  • A broader selection of domestic and international AI models

  • Real-time settlement mechanisms for participating suppliers

For suppliers, the proposition is equally direct: one integration can open access to additional distribution channels, while usage-based settlement helps accelerate revenue realization.

In other words, the goal isn't simply to make AI cheaper.

It's to make AI capacity more liquid, more transparent, and easier to allocate according to actual demand.


中文版|Chinese Version

我终于算清了"杀鸡用牛刀"这笔账

2026 年 8 月 11 日,英伟达做了一件乍看之下有些矛盾的事。

它发布了 Nemotron 3.5 Lightning——一个拥有 300 亿参数的 MoE 模型,最高输出速度可达到同类模型的 4 倍;与此同时,又开源了 NeMo Switchyard 模型路由库。

把这两个动作放在一起看,信号其实非常明确:

AI 的下一阶段竞争,正在从"谁的模型最大",转向"谁能把模型用得最聪明"。

英伟达并不是想靠 Nemotron 去抢 OpenAI 的模型冠军。

它真正瞄准的是企业更现实的一个痛点:

智能体跑得越多,AI 账单涨得越快。


7% 的前沿模型调用,吃掉了 68% 的账单

LangChain 使用自己的 Deep Agents 评测套件,对 145 个多轮深度智能体任务进行了测试。

在 Nemotron 3.5 Lightning 与 Claude Opus 4.8 之间进行智能路由时,只有 7% 的调用被分配给前沿模型。

但这 7% 的调用,却吃掉了 68% 的总账单。

剩下 93% 的任务——代码审查、工具调用、安全告警监控、账单问答等——都可以交给这个 300 亿参数的轻量模型处理。

最终结果非常直观:

  • 总成本下降 74%

  • 准确率从 86% 降至 80%

  • 仍然保留约 93% 的原始准确率水平

Ramp 在自己的 SWE-Bench 测试中也得出了类似结论:成本下降 58%,运行时间缩短 33%。

Cognition 将 Switchyard 的阶段路由器集成到 Devin Desktop 后,平均成本相比单独使用前沿模型降低了 28%。

74% 并不是什么魔法数字。

它其实只是一个简单的算术结果:

绝大多数任务根本不需要最聪明的模型,但过去很多团队却习惯把所有任务都交给最聪明、也最昂贵的模型。


Switchyard 到底在做什么?

NeMo Switchyard 本质上是一个使用 Rust 编写的代理层,位于推理服务器 API 与模型之间。

它不是另一个模型。

它负责决定:这项任务究竟应该交给哪个模型。

Switchyard 提供了多种路由策略。

1. LLM 分类器模式

通过一个更轻量的模型分析请求,然后辅助判断应该使用哪一个模型。

2. 自动升级模式

先使用便宜模型。

如果连续几轮表现不佳,再自动把任务升级到能力更强的模型。

3. 阶段路由器

根据 Agent 工作流所处的不同阶段选择模型,而且不会额外增加模型调用。

真正关键的是:

它计算的是任务总成本,而不是单个 Token 的价格。

一个模型即使每 Token 便宜 10 倍,如果为了完成同一个任务需要消耗 10 倍 Token,它也未必真的更便宜。

Switchyard 所做的事情,就是综合考虑请求特征、模型能力以及系统状态,然后判断:

这项任务到底值不值得调用最贵的模型?


开源路由层正在产生连锁反应

Switchyard 最值得关注的地方,也许并不是路由技术本身。

真正重要的是它所处的生态位置。

Kong、OpenRouter、LiteLLM 等 AI 网关和路由平台,都正在向多模型协同和智能路由方向发展。

英伟达高级总监 Joey Conway 对这一趋势的判断也很直接:多模型协同系统将成为 AI 的重要发展方向。

英伟达其实并不需要 Nemotron 击败 GPT-5。

它需要的是更多企业开始部署 AI。

什么 AI 都可以。

因为每增加一个 AI 应用,就意味着更多推理请求,也意味着更多算力基础设施需求。

开源模型降低了 AI 应用的进入门槛,同时扩大整个 AI 应用市场,而这最终会反过来扩大英伟达真正的核心市场:

芯片与基础设施。

Nemotron + Switchyard,本质上是在押注 AI 的"第二阶段"。

不是模型中心。

而是基础设施中心。

谁能够高效编排多个模型、智能路由不同任务,同时在保证能力的情况下持续压低成本,谁就可能掌握下一代 AI 的重要定价权。


Unexhub 的算盘:让 Token 像电力一样流动

也正是在"智能路由"逐渐成为刚需的节点上,再来看 Unexhub 正在做的事情,就更有意思了。

Unexhub 正在搭建一个 Token 聚合、分发与算力交易中心,核心思路是:

把 Token 从模型厂商内部的技术资源,逐渐转变为可以交易、流通和定价的市场商品。

它主要解决三个问题。

第一,实时结算

对于中小 AI 供应商来说,最现实的压力之一是资金周转。

Unexhub 的模式强调按照实际提供的 Token 和调用量进行实时结算,让供应商能够更快回笼资金,从而降低传统模式下长期垫资带来的压力。

第二,早期入驻零佣金

对于早期供应商,Unexhub 提供 0 佣金入驻权益。

它的意义并不只是"少收一点钱"。

更重要的是降低供应商进入新分销渠道的成本,让更多 AI 服务加入市场,从而提高整个 Token 市场的流动性。

换句话说,这是在用更低的进入成本换取更大的市场机会。

第三,API 动态竞价

传统 API 定价往往是固定价格。

高峰期算力拥堵一个价,凌晨资源闲置还是一个价。

Unexhub 希望让 API 价格能够根据市场供需动态变化:

忙时需求高,价格上升,供应商获得更高收益;闲时供给充足,价格下降,开发者获得更低成本。

这实际上是在给 AI 算力市场装上一套"价格信号灯"。


Switchyard 解决"用哪个模型",Unexhub 解决"到底多少钱"

两者看似是两个完全不同的项目,但放在一起看,逻辑非常清晰。

Switchyard 解决的是模型选择问题。

Unexhub 试图解决的是市场定价问题。

一个负责技术调度。

一个负责经济协调。

两者结合之后,AI 推理的成本结构正在发生变化:

任务由最合适的模型处理,算力按照市场供需定价,用户按照实际使用量付费。

对于企业和开发者而言,这意味着未来不一定需要把所有任务绑定到一个昂贵模型上,而是可以根据任务复杂度、实时价格和性能需求进行更灵活的资源配置。


从"堆模型"到"算细账"

企业 AI 的制胜策略正在发生根本变化。

过去的思路是:

"所有任务都用最好的模型。"

未来更可能变成:

"每一步都用合适的模型,并且根据任务变化智能切换。"

LangChain 的 145 个任务已经给出了一个非常值得关注的数字:

7% 的前沿模型调用,占用了 68% 的账单。

Switchyard 将成本降低 74% 的本质,也说明了一件事:

企业 AI 的大量支出,很可能来自把昂贵的智能浪费在了并不需要那么高智能的任务上。

未来 12~24 个月,企业之间的 AI 竞争优势,可能不再只是"谁能用到更强的模型",而是:

谁能把多个模型用得更聪明。

而要做到这一点,需要两层基础设施:

路由层 + 交易层。

英伟达正在打开路由层。

Unexhub 正在搭建交易层。

两者最终指向的是同一个趋势:

AI 推理正在从"凭感觉选模型",进入"按需求调度、按价格交易"的宽带时代。

74% 不是终点。

它只是开始。


Unexhub:让 AI 算力进入更灵活的市场

Unexhub 正在构建面向全球的 Token 聚合与分发平台及算力交易中心,汇聚不同规模的 AI 供应商。

平台生态覆盖包括 Gemini、ChatGPT、Claude、DeepSeek、GLM 等大型 AI 模型,同时也吸纳个人开发者和中小企业提供的 AI 服务。

其核心逻辑很简单:

聚合更多供应,让供应商之间形成竞争,再通过市场化机制帮助企业和开发者获得更灵活的 AI 算力选择。

对于开发者而言,Unexhub 提供的是一套更强调"按需使用"的权益组合:

  • 多模型聚合:减少分别对接不同模型 API 的开发成本

  • 毫秒级路由:根据任务与资源情况进行快速模型选择和流量分配

  • 市场化竞争价格:通过供应商竞争形成更有弹性的价格

  • 按 Token 实际付费:减少不必要的固定算力成本

  • 国内外模型资源覆盖:提供更丰富的模型选择

  • 实时结算机制:让供应商能够更快获得调用分成

在符合具体市场条件的情况下,开发者使用部分服务的成本最低可达到市场价格的 3 折。

这个权益真正有价值的地方,并不只是"价格更低"。

而是开发者可以在一个平台上比较不同模型、不同供应商和不同价格,在满足任务效果的前提下,尽可能把每一个 Token 都花在真正需要的地方。

对于供应商而言,则可以通过一次接入获得更多分发渠道,并通过调用量进行实时结算,从而降低渠道拓展和资金周转压力。

最终,Unexhub 想解决的并不仅仅是"让 AI 更便宜"。

它真正试图推动的是:

让 AI 算力更容易流动,让 Token 更容易交易,让价格更透明,也让每一份算力都能够流向真正需要它的地方。