The AI bill is coming due, and teams are building a control plane to pay it

The bill arrives

Databricks went all in on AI coding agents, and the numbers show both the payoff and the problem. The company says engineering output is its main bottleneck, and that AI tools have nearly doubled output with the same team size — while token spend per head is now flat despite increasing usage. As Patrick put it in the post: "Engineering output is the main bottleneck in our business." m:…5433

The headline number: Databricks says it reduced unit costs for AI coding agents by as much as 90%. The biggest lever was moving traffic to newer, cheaper models, accounting for over 50% of the savings. But the subtler finding is that Patrick is right when he says, "A model's token price is a bad predictor of what it costs" — how the model is routed, harnessed, and cached matters more than the price list. m:…5433

Route, don't cap

Hard spend caps, Databricks found, are counterproductive: they disrupt developer flow and push people toward shadow usage. Instead, the company shows developers their spend breakdowns — which saves about 10% on its own — and lets budgets trigger alerts or capacity reduction rather than abrupt shutoffs. Its Unity Gateway is used by thousands of customers. m:…5433 m:…5354

The routing logic is the interesting part. Databricks' Smart Router assesses task complexity in real time and cuts average task cost by more than 30% — and the company's analysis is that approximately 5 to 10% of AI tasks actually require frontier models, with about 90% solvable by cheaper non-frontier models. As Kevin Stumpf put it, "The industry is moving from tokenmaxxing to valuemaxxing." m:…5433 m:…5354

The harness decides the price

If routing decides which model runs, the harness decides what each run costs. In a case study published on AI21's blog, Martian says it cut the cost of GitLab Duo Code Review — which runs four model stages: Exploration, File fetching, Analysis, and Review — by 98%. The review stage alone accounted for 65% of the pipeline's total cost. Context compaction cut the median review prompt from 59K to 32K tokens, reducing review-stage cost by 19.4% — and more importantly, it let larger prompts fit inside the open-weight model's deployment window. m:…3843

Model choice was stage-specific, not one-size-fits-all: Claude Haiku 4.5 maintained quality on the mechanical stages but failed on the reasoning-heavy review stage, while DeepSeek V4 Flash — which Martian says costs 14x less per input token and 36x less per output token than Claude Sonnet 5 — handled it. The final setup cost $0.0135 per pull request with review quality measured at 47.2% F1. Stephane Benayoun's summary line: "The model sets the price. The harness decides whether you get to use it." m:…3843

Replace the call, retime the bill

The most aggressive lever skips the LLM call entirely. In a preprint posted this week, the authors of TabAgent describe distilling LLM agent decision components into cheap tabular classifiers. They report inference latency down about 95% and costs down 85-91% on the AppWorld benchmark, with task-level success comparable to the generative agents — but only in closed-set selection tasks, where the decision is a choice among known options rather than open-ended generation. m:…8858

And the billing model itself is getting a control plane. Modal's new open-source, Tinker-compatible training server, Spindle, lets teams pay for GPU time instead of per-token pricing, and uses multi-LoRA so many clients can share one trainer — Modal says trainer utilization reaches about 88% with four clients. The company claims an 80% cost reduction versus Tinker for DAPO-math, 46% for SWE-Gym, and 25% for search agents, all with 8 clients. The implication is that the cost curve now has three layers to manage: routing for inference, the harness for agents, and GPU-time scheduling for training. m:…1867