Inference Is Getting Cheaper From Three Directions at Once

One diffusion model, two jobs

In a preprint, researchers introduce Nemotron-Labs-Diffusion-Uni, a unified discrete diffusion large language model designed to integrate visual understanding and generation. It uses a cross-paradigm on-policy distillation framework to combine capabilities from autoregressive vision-language models and continuous flow models, transferring visual understanding ability from the former and visual generation ability from the latter into a single unified model. The model also features a progressive self-refining diffusion process that corrupts tokens across multiple steps during training and allows intermediate token commitments at inference, improving visual quality by letting the model commit gradually rather than all at once. m:…3082

The preprint reports that NLD-Uni achieved state-of-the-art performance among unified diffusion language models on visual question-answering, multi-modal reasoning, object grounding, text-to-image generation, and image editing. That breadth is the point of a unified model: one system that can both understand an image and produce or edit one, rather than two specialised systems that must be maintained separately. The claim is scoped to unified diffusion models, not to all models, and it rests on the distillation design that lets the diffusion model borrow strengths from two different model families without inheriting both of their serving stacks. m:…3082

A tiny drafter for diffusion

In a second preprint, researchers introduce SPOT, a 0.62-million-parameter drafter model designed to accelerate diffusion language models by predicting optimal token generation order. SPOT accelerates diffusion language models by predicting token generation order and verifying candidates in parallel, and it reduces target model passes by up to 6.8 times while maintaining exact decoding. Coupled-noise match keeps verification exact at any temperature by sharing Gumbel noise between the drafter and target, and diffusion language model predictions from the previous pass already provide the correct token at over 90 percent of positions, which is why a small drafter can do so much of the work. m:…5877

The throughput numbers reported in the preprint are substantial. SPOT reached up to 3.4 times the throughput of standard decoding on SDAR-8B, and SPOT-Sets, which drafts whole commit sets together with their tokens for dynamic threshold decoding, reached 7.6 times the throughput of standard decoding on SDAR-8B and improved throughput by 59 percent on average over the threshold decoder alone. The learned drafter also transferred across six different diffusion language models, which matters because a speedup that only works for one model is a curiosity, while a transferable drafter is a reusable piece of serving infrastructure. m:…5877

Routing without calibration

In a third preprint, researchers introduce Prox, which optimises mixture-of-experts inference by replacing calibrated thresholds with a top-k channel selection mechanism and a weight-derived reader. Calibrated thresholds for channel selection are inefficient because the number of channels passing the threshold varies across inputs, and calibrating a threshold provides neither a known read budget nor a consistent accuracy gain. Prox with thresholds calibrated for an 18 percent read budget read only 12.9 percent on average across benchmarks, a concrete illustration of how unpredictable threshold-based routing can be in practice. m:…2169

Top-k channel selection matched or beat threshold-based versions in 40 of 43 comparisons at 13 percent and 18 percent budgets, and it avoided the repetition-loop collapse that calibrated thresholds caused on Gemma 4, according to the preprint. The Prox reader retained 98.1 percent of full-model accuracy at 17.8 percent of routed-expert reads, screening channels with a ternary sketch and recovering skipped ones with a low-rank map. Its Triton kernels ran mixture-of-experts blocks 1.60 to 1.90 times faster than optimised vLLM at batch one and up to 2.69 times faster at larger batches, which turns a routing improvement into a serving improvement that a deployment team can measure directly. m:…2169

Why this combination matters

The three preprints attack different layers of the same cost problem. NLD-Uni reduces the number of models a team must serve by unifying understanding and generation. SPOT reduces the number of passes a diffusion model must make by predicting generation order with a tiny drafter. Prox reduces the number of experts a mixture-of-experts model must read by replacing an unpredictable threshold with a fixed top-k budget. None of them is a new flagship model announcement, and none of them requires a new hardware generation. They are the kind of work that lowers the cost of running models that already exist, which is often where a product margin is actually won or lost. m:…3082 m:…5877 m:…2169