The decoder wars: diffusion and parallel generation take on autoregressive LLMs

Fixing diffusion decoding

For years, language-model decoding has meant one thing: emitting tokens left to right. Four papers published on October 20 argue that monopoly is ending, and they arrive with both better quality and better speed. Two fix diffusion-language-model decoding quality, one turns diffusion models into speculative-decoding drafters, and one generates tokens in parallel without giving up autoregressive quality. m:…4232

The first two targets are diffusion language models, which generate by denoising rather than autoregressing. In a preprint, researchers diagnose a specific failure in sequential Monte Carlo decoding: multinomial resampling resets the effective sample size to its maximum while simultaneously discarding distinct lineages, masking the loss of diversity. Their fix, lineage-preserving SMC (LP-SMC), improves oracle coverage by 1.8 to 25.6 points across mathematical and code-generation benchmarks, and increases end-to-end accuracy by up to 7.9 points on LLaDA-1.5 and 18.3 points on Dream-7B. Because this is a preprint, the numbers are author-reported and unrefereed. m:…4232

Joint modeling for the latency frontier

A second preprint, Codpula, attacks the same quality-latency problem from the modeling side. It enables a context-dependent joint distribution in diffusion language models while retaining exact conditional inference, and it improves the accuracy-latency Pareto frontier in mathematical reasoning and code generation. The paper's complaint about prior work is specific: existing joint-modeling methods for diffusion language models suffer from rigid coupling parameters and lack of backbone adaptation. Like LP-SMC, these are preprint results awaiting independent replication. m:…9728

Speculative decoding from diffusion drafters

The third paper repurposes diffusion models as speculative-decoding drafters. Speculative decoding accelerates large-language-model inference through parallel verification of tokens proposed by a drafter, and the DScout authors flag the counterintuitive catch: diffusion drafters parallelize proposal generation, but greater proposal capacity or higher mean accepted tokens (MAT) need not yield higher throughput. Part of the reason is structural: verification accepts only a consecutive prefix, so early rejections can render later correct predictions ineffective. m:…1797

Measured against plain autoregressive decoding, DScout achieves 3.84x mean speedup on Qwen3-4B and 4.14x on Qwen3-8B. That is a 9.1% mean-speedup improvement over the state-of-the-art baseline on Qwen3-4B and 7.1% on Qwen3-8B, and the paper's inference configuration policy adds up to 9.75% additional mean speedup over fixed configurations. These numbers land on mainstream Qwen3 models, which makes them the most immediately testable of the four papers. m:…1797

Generating tokens in one pass

The fourth paper, another preprint, goes furthest. Parallel Language Models generate mutually consistent tokens in one forward pass by routing dependencies through latent auxiliary variables. On a 1.3B model, the authors report 2.3-5.7x inference speedup compared to autoregressive baselines, while matching or exceeding identically trained autoregressive models on math, coding, and instruction-following benchmarks. If the result holds at larger scales, parallel generation would remove the token-by-token tax entirely rather than just optimizing around it. m:…0378

What developers should try

As reasoning rather than measurement: decoding is now a design space, not a given. For developers running inference, the interesting experiments are no longer only about model choice — a diffusion drafter or a parallel decoder may beat a move to a bigger model on latency alone. The caveat is provenance: three of these four are preprints, so their figures are unrefereed author reports; DScout's Qwen3 numbers are the ones to replicate first. Watch for reproduction on larger models before rebuilding serving stacks around parallel decoding, and track whether the parallel methods hold up on reasoning-heavy workloads where autoregressive models still dominate. m:…1797