Serving AI Is Now a Cost-Control Problem, and Vendors Are Selling the Controls

Route each task to the cheapest model that clears the bar

Clay has introduced auto-mode as a new default setting for Claygent, its AI research agent, the company says. Auto-mode selects the best model for each task while optimizing for cost, and it enforces a quality floor for every task category, so simple queries go to efficient models while frontier models are reserved for complex research. m:…9992

On production traffic, auto-mode cost 47% less than routing every task to the most capable model, the company says. Each routing decision takes less than a second, and the router handles hundreds of millions of queries a month, the company says, which means the routing layer itself has to be cheap and fast or it eats the saving it creates. m:…9992

Clay post-trained the auto-mode model router in collaboration with Applied Compute, the company says, and the router is named AC-001 and runs on Applied Compute's platform, AC2. Users should test auto-mode on a sample of 5 to 10 rows to estimate costs for large tables, according to Clay's guidance — a reminder that the 47% figure is a production-traffic result, not a promise for any particular workload. m:…9992

Seal the data, stream the start

Baseten released BDN Sealed Volumes to provide immutable, versioned storage for model data, the company says. The volumes use content-addressable digests to identify immutable versions of data, which prevents data drift between staging and production: the version tested is the version served, identified by its digest rather than by a mutable path. m:…2186

Sealed Volumes eliminate the need for pre-deployment downloads by streaming data into containers, the company says, and they optimize uploads by splitting files into chunks and only transferring changed data. They support syncing data from Hugging Face, S3, GCS, Azure Blob, Cloudflare R2, and CoreWeave, the company says, and deleted versions remain restorable for 7 days. m:…2186

The limits are stated in the release. Sealed Volumes are not available on T4 or L4 instances, Baseten CLI version 1.1.0 or newer is required, and Truss version 0.18.30 or newer is required for users deploying with Truss. Sealed Volumes also allow deployments to serve two versions of a model simultaneously after a tag update, the company says, which supports gradual rollouts rather than cutover deployments. m:…2186

Pay for seconds, scale to zero

fal adds runners as requests arrive and removes them when traffic stops, down to zero by default, the company says. Billing is per second at the machine type's rate for setup, idle, running, draining, and terminating states, while pending, Docker pull, and terminated states are not billed. Requests that fail with a 5xx error are not charged, the company says. m:…1782

Users can configure scaling behavior through parameters such as minimum and maximum concurrency and keep-alive, balancing latency against spend, and settings can be adjusted in real time via the CLI or dashboard without a full redeployment, according to fal's documentation. The default maximum concurrency limit is 2 runners, and multi-GPU runners are billed as GPU count multiplied by duration. m:…1782

The three releases point to the same budgeting shift. When model calls were rare, the bill was dominated by the model itself. As agents call models hundreds of millions of times a month, routing, data versioning, and idle capacity become line items — and vendors are now selling controls for each of them. m:…9992

The line item to watch

The saving that compounds is the one tied to volume. A router handling hundreds of millions of queries a month turns a small per-query difference into a large monthly figure, and per-second billing turns idle capacity from a fixed cost into a controllable one. That is why the controls in these releases — routing, versioning, and scaling — are sold as cost features rather than performance features. m:…9992 m:…1782