The energy bill for inference is now measured in joules, and agents pay 13x

A law for inference energy

Researchers have introduced the SIMPLE Law, a four-parameter analytical model that predicts LLM inference energy consumption without requiring per-model calibration, in a preprint. The model separates prefill from decode, and decode into KV-cache cost and next-token computation, and it attained an R-squared of 0.969 on 13 dense LLMs with 11.97% mean absolute percentage error. The authors say it is 3.1 to 3.5 times more accurate than state-of-the-art models. A law that predicts energy without calibrating per model is useful precisely because the model zoo keeps growing faster than anyone can measure it. m:…7600

The law's findings are practical. Prefill energy grows quadratically with the hidden dimension of a model, and tokens enter the KV-cache and raise the cost of subsequent tokens, so energy grows non-linearly along a sequence. Grouped query attention reduces per-token energy by up to 3 times relative to standard multi-head attention. The implication for a team choosing an architecture is that the attention mechanism is now an energy decision, not just an accuracy decision — and the difference is a factor of three. m:…7600

The most expensive finding concerns agents. Agentic serving raises energy per request by 13.0 times relative to non-agentic requests, because agents make many more calls with long contexts. Prefix caching removes 93% of prefill cost in agentic serving, though the KV-cache term is unaffected by prefix caching. And reasoning has a sweet spot: intelligence per joule peaks at an intermediate reasoning budget and falls by up to 3.14 times across a 4.0 times reasoning-budget range. Letting a model think four times as long does not buy four times the intelligence per joule — it buys less. m:…7600

Local models catch up

Researchers from Stanford and Hazy Research introduced Intelligence per Watt, a metric that divides task accuracy by power draw to compare models and hardware, in a preprint. Local language models saw a 3 times improvement in accuracy between 2023 and 2025, intelligence per watt improved 5.3 times over two years, and intelligence per joule improved 18 times over 16 months, driven primarily by better hardware. Local inference is suitable for about 77% of practical, information-seeking tasks, according to the study. The researchers expect the world to become local by default but hybrid by design. m:…8665

The routing finding is the one to act on. A routing system that is 80% accurate can reduce energy consumption by 80%, compute consumption by 77%, and cost by 73% — by sending each task to the smallest model that can handle it. Local accelerators currently deliver 2 to 3 times less intelligence per watt and per joule than an Apple M4 Max compared with a B200, and local accelerators often serve a single user, resulting in lower batch sizes compared with cloud deployments. The gap is a hardware and batching problem, not a capability problem, and it is closing. m:…8665

The companion Joules-to-GDP study measures the dollar value and expert time AI systems produce per joule of energy, and its headline is that peak performance is not peak efficiency. GLM 5.2 delivered the most net value per task among the five models tested, but its net value per unit of energy fell 70% when judged against AI-assisted experts. Gemma 4 31B delivered 56% of GLM 5.2's value using 23% of the energy. GPT-OSS 120B used 68% less energy than GLM 5.2 despite similar cost and speed. The biggest model wins the task; the smaller model wins the joule. m:…8036

What it means for buyers

Hardware choice matters as much as model choice. Running Qwen3.6-27B on an NVIDIA B200 returned $535.5 of net value per megajoule, compared with $124.1 on an AMD MI350X — more than a fourfold difference for the same model on the same task. Moving the same model from one H100 to eight GPUs increased speed by 22% while using 3.24 times the energy. Architecture and engineering tasks produced 27% more net value per joule than software development tasks. The study used Gemini 3.1 Flash-Lite as an automated judge to compare AI outputs with expert deliverables, with expert data from Snorkel AI for 220 tasks completed by experts with and without AI assistance. m:…8036

The practical programme that falls out of these three preprints is short: cache aggressively, because agentic serving multiplies energy 13-fold and prefix caching removes most of the prefill cost; route each task to the smallest model that can do it, because an 80%-accurate router cuts energy by 80%; and stop letting models reason without a budget, because intelligence per joule peaks in the middle. The energy bill for inference is no longer a footnote to the token bill. For agent workloads, it is the bill. m:…7600 m:…8665