Inference Is Becoming the Market: A 10 Trillion Forecast Meets Flash Pricing

Rao's inference decade

Naveen Rao, speaking on the Twenty Minute VC podcast, projected that the inference market will reach 10 trillion dollars within 10 years, a figure that reframes the AI economy from a training story to a serving story, since inference is where models earn their keep after training ends. m:…6264

Rao's split forecast adds structure to that projection: the market for AI models will likely see a 50-50 split in spend between frontier models and open models, despite open models handling 80 percent of token traffic, which means the models serving most tokens capture only half the revenue. m:…6264

He tied the shift to a paradigm change: the current AI development paradigm is moving from pre-training focus to post-training reinforcement learning and specialization, and the transformer architecture is not the final paradigm, with a move toward more efficient, specialized reasoning architectures ahead. m:…6264

Flash pricing makes the shift concrete

SambaNova's MiniMax M3.1 Flash shows what inference competition looks like in practice: the model is available on SambaCloud delivering over 200 tokens per second, runs 3 times faster than its predecessor, and reduces usage costs by 70 percent, according to the company. m:…2627

Its benchmark and price points let builders do the arithmetic: scores of 73.7 percent on DeepSWE version 1.1, 82 percent on CyberGym, 61.7 on OSWorld version 2, 76.2 on BankerToolBench, and 29.6 on Agents' Last Exam, with cached input at 0.03 dollars per million tokens, input at 0.30 dollars, and output at 1.20 dollars. m:…2627

The combination of speed and price is the point: if inference is the market Rao describes, then tokens per second and dollars per million tokens are the competitive metrics, not just benchmark peaks. m:…2627 m:…6264

The chip money follows

Mayfield's roundup shows capital following the same logic: Tenstorrent raised 1.5 billion dollars at a 6.2 billion dollar valuation, a bet on alternative chip supply for the inference era, and Google released TorchTPU to reduce developer dependence on NVIDIA GPUs, a tooling move toward the same diversification. m:…4783

Rao's energy warning bounds the optimism: energy supply is the primary bottleneck for AI scaling, with current infrastructure growth rates insufficient to meet demand, and building one gigawatt of energy infrastructure costs approximately 50 billion dollars, which means the inference market's growth depends on power as much as on chips. m:…6264

For builders, the three sources converge on a planning question: which models to serve, at what token price, on whose chips, and with what power, since the answers to those questions now determine more of the business than the next training run. m:…6264 m:…2627 m:…4783

Serving is the strategy

The inference story's three sources point the same way: a forecast that serving becomes a ten-trillion-dollar market, a flash model competing on tokens per second and dollars per million tokens, and chip funding plus tooling that diversify supply, which together make inference strategy a board-level question rather than an engineering detail. m:…6264 m:…2627 m:…4783

The split between traffic and revenue is the detail to watch: open models handling 80 percent of tokens while capturing about half the spend suggests frontier pricing power persists even as open usage dominates, and builders should plan pricing and partnerships around that gap rather than assuming traffic converts to revenue. m:…6264