Reasoning leaves the token stream: four preprints on non-autoregressive intelligence

Latent reasoning, in parallel

One preprint proposes parallelizable probabilistic latent reasoning, or PLRM. The move is to replace autoregressive latent prediction with a JEPA-style objective, which lets latent reasoning models train in parallel rather than one step at a time. Mixture-based priors keep the model's diverse reasoning trajectories from collapsing into a single Gaussian distribution, preserving the spread of different ways to reach an answer. On multi-trajectory reasoning datasets, PLRM outperforms existing stochastic baselines. m:…5030

The move is philosophical as much as technical: reasoning does not have to be a token-by-token monologue. If latent trajectories can be trained in parallel and kept diverse, inference gets both faster and richer. That is a direct challenge to the assumption that chain-of-thought style sequential generation is the only substrate for reasoning. m:…5030

Diffusion models that fix their mistakes

Another preprint brings self-correction to masked diffusion models. ProSeCo lets these models change already-decoded tokens during generation through corrective refinement steps: an early bad token is no longer a commitment. The payoff is speed, with ProSeCo sampling up to 4 times faster than the best accuracy baseline. Scaling inference-time compute buys up to 1.2x improvement on benchmarks. m:…6367

This is the diffusion answer to a problem autoregressive models punt on: revision. Being able to rewrite mid-generation trades a little compute for noticeably better tokens, and it does so without the left-to-right straitjacket. For developers, the practical variable is now how much refinement budget to spend at inference time, not just how long a context to fill. m:…6367

Temperature that respects the model

A third preprint tackles temperature in flow language models. Their Bayes-consistent tempering, or BCT, improves GSM8K performance more reliably than standard post-hoc temperature scaling. They report that simply lowering temperature in flow LMs is less reliable than doing so in discrete diffusion LMs, unless BCT is used. The learned BCT realization stays more robust at very low temperatures than training-free variants. m:…7835

Temperature is usually treated as a dial you turn after training. This work suggests the dial behaves differently depending on the generative machinery underneath, and that flow models need tempering designed for their own dynamics. Anyone tuning sampling parameters on a flow-based model should check whether their temperature intuitions were built on a different architecture. m:…7835

What a diffusion transformer is actually doing

The fourth preprint steps back to mechanism. The fourth preprint offers an aggregator-predictor view of diffusion transformers: DiTs function as input-adaptive local denoisers, with attention acting as the aggregator and MLPs acting as patch predictors. For Gaussian Markov random field priors, the global Bayes denoiser can be approximated using only a subset of context patches, with error decaying exponentially in context radius. And shared-memory patch predictors can replace DiT MLPs while delivering competitive denoising and generative performance. m:…6247

If MLPs in a DiT are really patch predictors, there may be cheaper ways to do their job, which is exactly what the shared-memory variant tests. The thread across all four papers is the same: the token stream is a choice, not a law, and the alternatives are getting fast, correctable, and theoretically grounded. Mechanistic clarity of this kind is where performance engineering starts: once you know what each part does, you can replace the expensive parts. m:…6247