Vision models are learning to see with fewer tokens

The 78 percent discount

Images are expensive to look at. In a preprint, the authors of TOPS — a first-principles visual token pruning method — report cutting 77.8 percent of visual tokens on LLaVA-NeXT models while maintaining performance. The module is training-free and model-agnostic: bolt it on, pay a fraction of the attention bill. There is a bonus the authors did not have to include but did: pruning redundant visual tokens can mitigate hallucination in multimodal models. Less noise in, fewer confabulations out. m:…1045

MiCo pushes the same idea harder with an information-theoretic frame. Its mutual-information coverage objective — a general, training-free framework for visual token pruning, derived from task log-loss — delivers a 3.8-fold inference speedup on LLaVA-NEXT-13B while retaining 97.5 percent of baseline performance. The method does task-aware subset selection, using visual signals to pick a representative candidate pool before tokens ever reach the language model. The language model never sees the pixels you did not need. It also never bills you for them. m:…8211

We were measuring it wrong

TwinPrune arrives with an accusation: decode-pruning claims validated solely on prefill probes are measuring prefill quality, not decode performance. The field has been grading the warm-up and calling it the race. The paper's own method — density top-k KV eviction — outperforms H2O, SnapKV, and StreamingLLM on TextVQA and MM-Vet, and the authors report a scaling consolation: accuracy loss shrinks as models grow from 8B to 38B. Bigger models, it seems, mind the pruning less. m:…2428

There is a serving strategy buried in these numbers. TwinPrune’s scaling result — accuracy loss shrinking as models grow from 8B to 38B — suggests pruning pairs best with larger models, which is convenient, because larger models are where the token bill hurts most. And TOPS’s hallucination finding reframes pruning from a cost play into a quality play: fewer redundant tokens means fewer opportunities for the model to confabulate over visual noise. The pitch to your infra team writes itself — cheaper and more truthful — as long as you validate on the decode path, per TwinPrune’s warning. m:…2428 m:…1045

LEAP takes the most radical position: stop loading the video. Its learned block-wise evidence retrieval decouples evidence localization from reasoning, maintaining constant peak context regardless of recording duration — a ten-minute video costs the same context as a ten-hour one. On AVQA benchmarks it lifts the Qwen3-Omni-30B-A3B baseline by 4.5 to 16.8 percent, and it supports streaming inference without streaming-specific training. Find the evidence first, reason second. The context window stops being the budget. m:…6180

An ops decision, not a research project

The commercial read is the interesting one. These are training-free methods: no retraining run, no new checkpoint, no GPU cluster booked for a month. A founder shipping a multimodal product can treat visual token spend the way they treat any other unit cost — measure it, prune it, and re-measure, with hallucination reduction as a possible side benefit. The remaining risk is TwinPrune's warning: make sure the benchmark you trust measures the decode path your users actually pay for. m:…1045 m:…8211

LEAP’s constant-context result deserves its own line in the budget model. Decoupling evidence localization from reasoning means the cost of a video-understanding feature stops scaling with video length — and streaming inference without streaming-specific training removes the usual retraining tax for live workloads. If your product watches long video, the question is no longer how big a context window you can afford. It is whether you still need the window at all. m:…6180