The AI bill is now a routing decision, not a model decision

The router that reads only the question

H2O Super Router is in early tech preview for select customers, and its design is deliberately minimal. The router reads only the text of the question and needs nothing from inside the models. It runs as a single container that speaks the OpenAI chat API, and it is built on LiteLLM. That minimalism lets it route across vendors without access to model internals. m:…1546

H2O reports striking results for that minimal design. The router raised accuracy from 65.8% to 74.2% at a quarter of the cost of the best model, and H2O says it matched the best single model, Gemini 2.5 Pro, at 1/48th of its cost. The company also says its text-only ensemble reaches 40x cost savings without touching model internals. These are vendor-reported tech preview figures, so they deserve independent replication before anyone budgets against them. m:…1546

The most interesting result is the ensemble finding. Combining answers from three different models scored 58.7% on deep-research tasks, compared with 54.0% for three answers from the best model. Diversity across models beat repetition of the best model. That cuts against the instinct to find one winner and use it for everything. For research-style work, the second and third opinions are not redundancy. They are the source of the gain. m:…1546

H2O also reports a pricing insight that should interest anyone paying per token. Estimating price per question cut the bill to reach Gemini 2.5 Pro accuracy by 3.5x against the published rule. In other words, knowing what a question will cost before routing it is itself a saving, because it lets the router spend heavily only where the question warrants it. The router also supports agent monitoring and certification against drift and quality bars, which is the instrumentation a team needs to verify any of this on its own traffic. m:…1546

There is a limit, and H2O states it plainly. The router lost up to 13.8 points of accuracy on unseen benchmarks. Routing learned on one set of tasks does not automatically transfer to another. Any team adopting a router should therefore test it on its own unseen workload first, because a router tuned to public benchmarks may be tuned to exactly the wrong distribution. m:…1546

Why pricing now forces the question

Routing matters more now because the way AI is billed has changed underneath it. SAS reports that AI providers are shifting from flat-rate subscriptions to usage-based billing tied to token consumption, and that GitHub Copilot recently shifted from flat-rate subscriptions to usage-based billing. A flat subscription hides the cost of a bad routing decision. A token bill exposes it, line by line. m:…2559

The numbers SAS cites show how fast the exposure grows. The Claude Opus 4.7 multiplier in GitHub Copilot increased from 7.5x to 27x, and SAS reports inference costs in engineering departments are reaching 10% of total headcount costs. Against that background, the price spread is enormous: GPT-4 launch pricing was $30 per million input tokens and $60 per million output tokens, while GPT-5.4 Nano input pricing is $0.20 per million tokens and Gemini 3.1 Flash-Lite input pricing is $0.25 per million tokens. m:…2559

SAS also describes two technical levers that make cheaper routing viable. Model quantization reduces memory requirements by compressing weights from 16 or 32 bits to 8 or 4 bits, and Multi-Token Prediction reduces latency and compute costs by predicting multiple tokens per forward pass. SAS argues local inference provides cost predictability, data sovereignty and reduced latency, and recommends a hybrid strategy using cloud APIs for frontier models and local inference for specialized tasks. m:…2559

Caps and outages make routing a continuity tool

Cost is only half the routing story. The other half is continuity when a provider says no. OpenRouter explains that Claude Code plans have a five-hour limit that resets five hours after the first prompt and a weekly limit that resets every seven days, and that Anthropic does not publish the exact number of prompts allowed under their usage limits. OpenRouter says Claude Code internal cost estimates are often inaccurate, and that Claude Code consumes more tokens than a standard chat session. m:…2706

OpenRouter positions its own keys as the workaround: OpenRouter API keys do not have five-hour or weekly usage limits, and OpenRouter applies account-level budget guardrails to Ori sessions, with users able to set a credit limit on keys to control spending. Its session cost examples make the routing arithmetic concrete: a typical short Claude Code session costs approximately $0.16 on Sonnet 5.5, approximately $0.31 on Opus 5.5, and less than one cent on Haiku 5.5. The gap between those three numbers is the entire routing business case in miniature. m:…2706

Outages push the same way. A 529 overloaded error indicates that Anthropic's API is temporarily out of capacity for a specific model, and exponential backoff with randomness is recommended to handle transient 529 errors. OpenRouter serves Claude Sonnet, Opus and Haiku 5.5 models through five different providers, and its default routing automatically fails over to alternative providers when a primary provider fails. Claude Code on OpenRouter enables provider failover for Claude sessions, and Claude Code retries transient failures up to 10 times by default before returning an error. m:…4922

The practical conclusion is that model choice is becoming a policy, not a preference. Teams should be able to say, for each kind of request, which model earns it, what it may cost, and what happens when that model is busy or capped. Teams that write that policy themselves, and measure it on their own traffic, will keep the savings. m:…1546 m:…2706