Memory is the efficiency frontier: smaller caches, leaner training, and robot models squeezed to 1.5 bits

Halving the context cache

In a preprint, Slim attention offers a mathematically identical implementation of standard multi-head attention, which means it produces the same result while storing less. The authors report that it reduces context memory by 2x for standard transformer models with multi-head attention, and the gains are larger for some architectures: 8x for Whisper models and 32x for T5-11B. m:…5658

The practical point is about context windows. Memory for the context cache is often what caps how much conversation, document, or tool history a deployment can hold at once. A drop-in change that halves that cache, without changing outputs, lets the same hardware serve longer contexts or more concurrent sessions. The preprint framing is that only the key cache is needed, which is where the savings come from, though operators would still want to verify the identical-output claim on their own workloads before treating it as free. m:…5658

When bigger compressors do worse

A second preprint challenges a common assumption: that a larger model is always the better choice for compressing context. In a compressor-decoder setup, larger models showed lower faithfulness in reconstructed contexts than mid-sized models, according to the authors. The reason, they argue, is that larger models bring more of their own prior beliefs to the task. m:…4908

Two failure modes are named. Knowledge overwriting happens when larger models replace source facts with their own prior beliefs during compression. Semantic drift happens when larger models paraphrase or restructure content instead of reproducing it faithfully. For builders, the implication is that model selection for compression should be driven by the goal of faithful preservation, not by parameter count. A smaller compressor that stays close to the source can be the better engineering choice, even if it looks weaker on a general benchmark. m:…4908

Training without momentum buffers

A third preprint attacks memory on the training side. Muon-RT eliminates momentum buffers for hidden weight matrices by reusing teacher-student displacement during self-distillation, so the optimizer no longer needs to keep persistent momentum state for those weights. The authors report that this reduces persistent training memory by 42% compared to AdamW and by 27% compared to momentum Muon. m:…4449

Memory savings alone would be a modest win if accuracy fell, but the preprint reports the opposite on its test case: Muon-RT improved average accuracy on Qwen2.5-7B-Instruct by 2.6 points over AdamW and by 5.4 points over momentum Muon. The result is specific to that model and to self-distillation, so it should be read as evidence that reusing the teacher trajectory is a promising direction, not as a general claim that momentum buffers are obsolete. m:…4449

Robot models at ultra-low bits

The fourth preprint takes the memory fight to robotics. SACQ is a quantization framework for vision-language-action models, which are large because they combine vision, language, and action prediction. Using structure-aware codebook fitting and action-aware mixed-bit allocation, the authors report 93.85% average success across four LIBERO suites at 1.5 bits per language and action weight, with a memory footprint 4.83x smaller than BF16. m:…2824

Speed follows the shrinkage: with INT8 activation quantization and low-precision attention, SACQ runs 1.51x faster than BF16 while retaining 93.55% success. The design detail worth noting is the action-aware allocation, which assigns bits based on how much the final action degrades, rather than treating every weight equally. For robots running on edge hardware, that is the difference between a model that fits and a model that stays in the data center. m:…2824

What ties them together

Across all four preprints, the scarce resource is not parameters or data but memory: the cache that holds context, the buffers that hold training state, and the bits that hold a deployed model. Each paper trades a different kind of redundancy for savings, and each reports that the savings come without giving up the thing that matters, whether that is identical attention output, faithful reconstruction, accuracy, or policy success. That pattern is worth watching, because if it holds up outside the authors own tests, the next round of cost and latency wins will come from memory engineering rather than from bigger models. m:…5658 m:…4908 m:…4449 m:…2824