Diffusion language models get their own training stack — RL, correction, and faster decoding

Training: RL and distillation catch up

Diffusion language models generate text by denoising a block of tokens in parallel, which makes them fast but hard to train with the usual tools. The exact likelihood of a completed block is intractable, because it marginalizes over the intermediate states and reveal orders of the denoising process, which has left diffusion models without a clean path to reinforcement learning. A preprint this week proposes Prefix-Block Policy Optimization, or PBPO, which builds a prefix-block state and uses a factorized one-step surrogate so RL training stays stable while preserving block-level conditioning. m:…9638

A second preprint merges two training stages that used to run in sequence. Sequential approaches to diffusion model training require separate stages for capability consolidation and step distillation. StepX-OPD trains text-to-image diffusion models for capabilities and few-step sampling in a single stage, supervising students on their own rollouts with frozen specialists, and focusing velocity regression on the low-noise part of the rollout. It trained SD3.5-Medium with 37% fewer GPU-hours than sequential capability consolidation followed by step distillation, and outperformed the forty-step Flow-OPD model. A variant also reduces the tendency to write unrequested text into images. m:…7411

A third preprint asks whether one-step generation was overcomplicated all along. Priming Models use a two-stage strategy: train first on a reconstruction objective, then do distributional post-training. The authors report a state-of-the-art FID of 1.19 on ImageNet at 256x256 for one-step generators, using approximately one-tenth the training compute of prior one-step methods with comparable quality. Their claim is that a reconstruction objective is sufficient to remove most of the difficulty of distributional training. m:…9627

Decoding: fix mistakes, skip redundancy

The decoding side of the week is about what diffusion models do wrong when they generate in parallel. Standard masked diffusion models cannot reliably revise incorrect tokens, because they lack discriminative confidence about which tokens are wrong. A preprint on Corrective Diffusion Language Models proposes a correction-oriented post-training objective that supervises visible corrupted tokens, teaching the model to assign lower confidence to incorrect tokens so it can refine them in place. Models trained this way outperformed standard masked-diffusion fine-tuning on HumanEval(+), MATH, and ParallelBench. m:…9628

A second decoding preprint tackles visual redundancy in diffusion-based multimodal models. When a model commits several high-confidence tokens in the same parallel step, those tokens can rely on overlapping visual grounding, which wastes the parallelism. The authors introduce a Visual Redundancy Index that quantifies that overlap, and a decoding method, VRCD, that prioritizes visually complementary tokens. In a preprint, VRCD achieved relative accuracy gains of up to 18.8% on M^3CoT and 6.9% on MMBench over confidence-based decoding, with no additional training. m:…9649

Why this week matters

None of these papers claims diffusion models have caught up with autoregressive ones. What they show is the stack filling in: PBPO gives diffusion models an RL method, StepX-OPD and Priming Models cut the training cost, corrective post-training gives them a way to fix their own errors, and VRCD gives them a way to decode in parallel without tripping over their own redundancy. The through-line is that diffusion models are no longer a curiosity waiting for a training recipe. They are getting one, piece by piece, in public, in a single week. m:…9638 m:…7411 m:…9628 m:…9649