The compute-savings papers: sparser attention, looped models, cheaper distillation

Attention gets sparser

Long-context decoding is dominated by KV-cache traffic, and in a preprint posted this week, the authors of JSSA argue practical sparse attention methods mostly operate at coarse granularity while finer token-level selection has been held back by two problems: scoring can lose the query-key structure needed for accurate ranking, and gathering scattered tokens adds latency. Their answer is a shared low-dimensional query-key subspace learned by minimizing token-score preservation error, executed through a gather-free token-level indexed-attention kernel that works under the standard grouped-query attention KV-cache layout. m:…1622

The reported numbers are strong: up to 6.43x the decoding throughput of full-KV attention at 128K context on an NVIDIA B200 GPU, 96-99% of model-native DSA throughput even at 700B scale, and reasoning accuracy comparable to model-native DSA without any model training. The authors also claim JSSA avoids the generation-length inflation and architecture-dependent failures seen in competing sparse attention methods, and say it is compatible with standard GQA, hybrid sliding-window, and MLA models. The authors say direct refinement of the exact objective provided no measurable improvement in the tested settings, which keeps the method simple to implement. m:…1622

Looped models spend unevenly

A second preprint, LatentLoop, starts from an observation about looped mixture-of-experts models: different loop visits can play different roles even when they share parameters, and shrinking the first visit preserves or even improves performance while shrinking the second hurts it. The method compresses the first visit's sequence, keeps full resolution in the second, and reallocates the saved computation to increase the hidden size, using separate attention parameters to handle the different resolutions. The paper notes that sharing MoE weights across visits while using separate attention parameters is what lets the two visits operate at different sequence resolutions. m:…6224

Across 15 configurations spanning 1.3B to 7B parameters, LatentLoop consistently lowered validation NLL and delivered compute savings of 19.0% relative to vanilla MoE and 7.9% relative to the looped baseline at a fitted NLL of 1.988, while improving average scores on nine benchmarks in both 0-shot and 5-shot settings. This suggests a design principle worth watching: in parameter-shared architectures, not every pass deserves equal compute. m:…6224

Distillation and editing go lean

The third preprint, Glance Forcing, attacks video generation distillation, where existing pipelines require offline probability-flow ODE trajectories costing tens of thousands of GPU-hours. The framework decouples the denoising trajectory into two complementary slow and fast experts, each guided by stage-specific objectives, which the authors say cuts trajectory-collection cost from thousands of GPU-hours to about one A100-GPU-hour. Reducing supervision to a single sample caused severe quality collapse in monolithic students, but the two-expert split held up, raising the VBench Total score from 84.04 to 84.22. m:…0518

The fourth, Diff Diffusion, makes the same point for code editing: regenerating full code segments in AI-assisted IDEs wastes computation because most edits are sparse. The diffusion-based framework predicts structured edit operations instead, supporting multi-position parallel modification, and reports 2.78x faster inference on public edit-completion benchmarks plus a 39.0% P50 model-latency reduction in production. This suggests the efficiency frontier is moving from how models are built to how their work is framed. m:…1119