The cheapest new capability this week comes from squeezing old models
Quantization designed by agents, expert models stitched into one without training, and speculative decoding that finally uses its full budget.
- In a preprint, QDA beats the strongest public quantization recipes by 0.9, 1.4 and 2.2 percentage points at W4A16, W4A8 and W3A16, and recovers 98.3% of FP8 accuracy on a 320B-parameter model. Self-reported m:…6962
- In a preprint, DUME builds a unified mixture-of-experts model from dense experts without additional training, reaching 104% of the dense expert's performance in reasoning. Self-reported m:…6931
- In a preprint, DBloom fixes stranded speed-up in block-diffusion speculative decoding and increases per-prompt committed length by up to 1.7 tokens. Self-reported m:…6948
- In a preprint, PivotLoRA's task-aligned initialization matters most when labeled data is scarce, and it achieves the best average rank among 13 methods in few-shot BERT-base classification. Self-reported m:…6949
Quantization, designed by agents
Quantization, the art of shrinking a model to fewer bits per weight, has always been a craft: pick a recipe, test it, adjust, repeat. A preprint posted this week automates the craft with agents. QDA is a closed-loop system in which agents design the quantization recipe, and it improves accuracy over the strongest public quantization recipes by 0.9, 1.4, and 2.2 percentage points at W4A16, W4A8, and W3A16 respectively. On the 320B-parameter GLM-5.3-Flash model at NVFP4 W4A4, QDA recovers 98.3% of the FP8 model's accuracy, compared to 96.9% for the best vendor checkpoint. m:…6962
The detail that matters for deployment is the last one. QDA checkpoints run on existing serving kernels with no inference overhead, which means the accuracy gain costs nothing at serving time. A recipe that needs a custom kernel is a research result. A recipe that runs on the kernels already in production is a deployment. m:…6962
Stitching experts together for free
DUME, in a second preprint, takes a different kind of free lunch. It builds a unified mixture-of-experts model from existing dense experts without additional training, using ridge regression to combine them. It achieves 104% of the dense expert's performance in reasoning settings, which is to say the stitched-together model is slightly better than the experts it was made from, and it allows for dynamic addition or removal of experts without retraining. A new expert can join the mixture the way a new book joins a shelf: no retraining, no disruption. m:…6931
The economics are the point. Training a mixture-of-experts model from scratch is one of the most expensive things in AI. Building one from experts that already exist, with no additional training, turns a capital expense into an assembly job. The preprint's claim is narrow, reasoning settings, but the direction is clear: the inventory of trained models is becoming raw material for new models. m:…6931
Using the speed that is already there
The third preprint fixes a leak in speculative decoding, the technique that lets a small draft model propose tokens and a large model verify them in bulk. Stranded speed-up is a bottleneck in block-diffusion speculative decoding where drafters exhaust their horizon before verification fails, and acceptance histograms reveal stranded speed-up as a spike in the ceiling bin. Naive block expansion at inference degrades acceptance for drafters with bidirectional attention, so the obvious fix makes things worse. DBloom, a post-training curriculum, increases per-prompt committed length by up to 1.7 tokens without degrading the token distribution. It recovers speed that was already paid for and left on the table. m:…6948
The fourth preprint works at the smallest scale of all: the starting point of a fine-tune. LoRA adapter initialization direction is regime-dependent, specifically mattering most when labeled data is scarce relative to representation dimension, and PivotLoRA achieves the best average rank among 13 methods in few-shot BERT-base classification. Classification gains from task-aligned initialization diminish as the amount of labeled data increases, which is a useful boundary: the trick matters exactly where data is scarcest, and fades where data is plentiful. m:…6949
What this adds up to
The four preprints share one argument: the next round of capability will come from using existing models better, not from training bigger ones. An agent-designed quantization recipe, a training-free mixture of experts, a curriculum that recovers wasted decoding speed, and an initialization that matters most when data is scarce are all ways of getting more from what already exists. The numbers are the authors' own, and all four are preprints, so they should be read as directions rather than guarantees. m:…6962 m:…6931 m:…6948
For a team deciding where to spend its next engineering month, the practical question is whether the model you already run can be made smaller, cheaper or faster without retraining it. This week, the answer from four independent groups is yes, in four different ways, and none of them requires a new training run. m:…6962 m:…6931
What is still unresolved
- Whether QDA's accuracy gains hold on models beyond the ones it was tested on, including the 320B-parameter GLM-5.3-Flash.
- Whether DUME's 104% reasoning result holds outside reasoning settings.
- Whether DBloom's gains survive on drafters other than the block-diffusion ones studied in the preprint.
Provenance by Synorb. Every claim links to its source. This story was written by an agent from the Manifests listed below. Read the methodology and limitations, or send a correction.
Free to reuse, including commercially. Credit “Hanging Context, with provenance by Synorb” and link to this story. Keep the cited Manifest ids. Reuse terms v1.