Open agentic models trade size for routing: hybrid experts, distilled teachers and reused depth

A hybrid model built for agent work

In a preprint, NVIDIA authors describe Nemotron 3.5 Super and Lightning, open mixture-of-experts models that combine Mamba and Transformer layers in a hybrid architecture designed for high-reasoning agentic tasks. The design choice matters because agent work is not a single forward pass: it involves long contexts, repeated tool calls and extended reasoning, and an architecture that handles long sequences cheaply while routing each token to a subset of experts can serve that pattern at lower cost than a dense model of similar capability. The paper also reports that agentic mid-training improves software-engineering capabilities and robustness across agent harnesses, which is a claim about training for the deployment pattern, not just about a benchmark score. m:…0833

Two efficiency results in the same preprint show how the models are meant to be served. Multi-teacher on-policy distillation is reported to improve speculative decoding acceptance length by up to 21.8%, which means a draft model proposes longer stretches that the main model accepts, reducing wasted work. NVFP4 quantisation is reported to achieve 99.62% median accuracy recovery relative to BF16, which means the model can run in a compact numeric format while keeping almost all of its accuracy in the authors' tests. For a founder choosing a model to deploy, those two numbers matter more than a headline capability score: they describe what the model costs to run, and whether a cheaper serving format preserves what the model can do. m:…0833

Distillation and decoding do the same job elsewhere

A second preprint, on Turing LLM post-training, applies a similar idea to dynamically routed mixture-of-experts models: calibrated multi-teacher on-policy distillation improves specialist integration compared with uncalibrated methods, and asynchronous rollout-training overlap combined with bounded-width dispatch reduces consolidation step time by 2.86 times. Its Turing-20B-A2B model, with 20 billion total parameters and 2 billion active, is reported to achieve an unweighted mean score of 85.92 on reasoning-enabled mathematics and code benchmarks, against 80.33 for Qwen3-8B. The comparison is the authors' own, in a preprint, but the shape of the result is consistent with the Nemotron paper: routing and calibrated distillation let a model with few active parameters compete on reasoning tasks that once seemed to require a much larger dense model. m:…0927

A third preprint, CopyLoop, shows the same reuse logic inside decoding for looped Transformers, which reuse model blocks to gain depth without adding parameters. By combining context-copy proposals with shallow screening, which filters incorrect branches early, and concurrent block processing, CopyLoop is reported to speed decoding by 2.98 to 10.70 times over full-depth autoregressive decoding on base models and 2.74 to 3.52 times on thinking models. A fourth preprint, Express, converts non-causal attention approximations into causal ones with matching approximation guarantees, using an I/O-aware Triton implementation reported to give substantial speedups over FlashAttention 2 for long-context prefill and key-value cache compression. The common thread is not a single architecture. It is that the expensive part of a model, its depth, its attention or its expert routing, can increasingly be approximated, reused or screened, rather than paid for in full on every token. m:…5715 m:…8941

The buying decision moves to serving cost

For a founder or engineering lead, these preprints move the model decision from capability alone to capability per serving dollar. A hybrid routed model with strong quantisation recovery, a distilled specialist integration, and a decoding method that reuses depth may serve agent workloads more cheaply than a larger dense model with a higher headline score, and the gap grows with context length and reasoning length, which is exactly where agent costs accumulate. The caveat applies to every number in this story: all of it comes from preprints, reported by the authors, on their own benchmarks and hardware. The right next step is not to adopt a model on the basis of a preprint. It is to test the routed, quantised, distilled version on the team's own agent workloads, and to measure acceptance length, accuracy recovery and decoding speed there, because those are the numbers that will set the bill. m:…0833 m:…0927 m:…5715