Model prices fell 13-fold in a year. The bill now depends on the serving stack
Model prices fell about 13-fold in a year, yet inference now takes about two thirds of AI compute. The money is moving to serving systems, memory and verification.
- Unusual Capital reports model prices fell about 13-fold last year, and the cost of a fixed level of AI performance has fallen about 47 percent per quarter since 2023. m:…0013
- Inference grew from about a third of AI compute in 2023 to an estimated two thirds in 2026. m:…0013
- Fal says the NVIDIA B300 supports about twice as many 32K sessions as the B200 for a 671B model on an 8-GPU node, and is cheaper per concurrent user even at list price. Self-reported m:…9594
- MLPerf Endpoints v1.0 measures live API endpoints under production-like loads, testing the whole stack from hardware to batching and networking. m:…6960
- Parasail warns that a model ID in an API response does not identify the weights or precision used, and only hardware-backed attestation can prove which model handled a request. m:…6768
The price fall
Unusual Capital reports that model prices fell about 13-fold last year, and that the cost of a fixed level of AI performance has fallen about 47 percent per quarter since 2023. The firm divides the cost curve of intelligence into three layers, the model, the serving system and the environment, and argues that the last two hold most of the savings still available. Inference, meanwhile, grew from about a third of AI compute in 2023 to an estimated two thirds in 2026, so the serving layer is now where most of the money goes. m:…0013
The firm gives a concrete comparison. OpenAI's o3 model scored 75 percent on GPQA Diamond at an estimated 30 cents per question in January 2025, while GPT-5.6 Luna matched that performance at 0.0004 dollars per question. The firm also argues that open weights now cap what any provider can charge for a model. This suggests the model itself is becoming the cheap part, and the advantage moves to whoever runs it most efficiently. m:…0013
Memory, not speed, decides the hardware
Fal's comparison of NVIDIA's B200 and B300 GPUs makes the same point in hardware terms. The company says the B300 does not offer faster FP8 or BF16 compute than the B200, but offers 1.5 times the FP4 compute and up to two times faster attention. For a 671B mixture-of-experts model on an 8-GPU node, the B300 supports approximately twice the number of 32K sessions as the B200, because it has more memory to hold them. m:…9594
Fal says the B300 node is cheaper per concurrent user than the B200 node, even at list price, although on Fal the B300 costs 1.63 times the list price of a B200 and 1.09 times at as low as rates. The trade is real, though. The company says the B300 gives up almost all of its INT8 and FP64 compute compared with the B200, and its INT8 throughput is approximately 4 percent of the B200's. A buyer chasing raw speed on the wrong precision would pay more and get less. m:…9594
Benchmarks move to the endpoint
MLCommons has introduced MLPerf Endpoints v1.0, a benchmark that measures live API endpoints under production-like loads instead of testing silicon in isolation. It evaluates the entire AI stack, including hardware, model, serving software, batching behaviour and networking, and uses workloads that customers actually use, including multi-turn AI agents. The suite produces a Pareto frontier of system behaviour across varying workloads rather than a single peak score. m:…6960
MLCommons says the benchmark does not use estimated or interpolated data, and submissions run in rolling publication cohorts every few weeks. That matters because the techniques Unusual Capital points to for serving efficiency, speculative decoding, megakernels, weight streaming and new number formats, only show their value under real load. A benchmark that tests the endpoint is the only way to see whether those techniques actually lower the bill. m:…6960 m:…0013
Trust what you are served
Parasail warns that the model name on an API does not guarantee what is running behind it. The company notes that DeepSeek began serving DeepSeek-V4.1-Flash-1017 under the deepseek-flash API name on October 17, 2026, and redirected requests for retired names to the same model. Model IDs in API responses are set by the provider and do not identify the weights or numerical precision used, so a price comparison between two providers can be comparing two different models. m:…6768
Parasail says changed behaviour usually traces to announced model updates, serving misconfigurations or undisclosed model changes, and that hardware-backed attestation is required to prove which model handled a specific request. In a preprint, the TideCache authors report up to 2.04 times the agent turn capacity of SGLang with HiCache and a 20 to 59 percent cut in median per-token latency on GB300 and GH200 platforms. The implication is that the cheapest provider on paper may not be the cheapest in practice, and the buyer who verifies both the model and the serving stack is the one who actually saves money. m:…6768 m:…0688
What is still unresolved
- How much of the 13-fold price fall reaches buyers who cannot switch providers easily
- Whether B300 pricing holds once it becomes a standard machine type on Fal rather than arranged through the team
- How widely providers will adopt hardware-backed attestation for model identity
- Whether TideCache results hold outside the preprint's own tests, since the work is not peer-reviewed
Provenance by Synorb. Every claim links to its source. This story was written by an agent from the Manifests listed below. Read the methodology and limitations, or send a correction.
Free to reuse, including commercially. Credit “Hanging Context, with provenance by Synorb” and link to this story. Keep the cited Manifest ids. Reuse terms v1.