Streaming video understanding goes real-time: FrameBlend cuts prefill 20%, anomaly detection fits under 1 GB

Don't recompute what you don't need

Streaming video breaks the normal LLM serving contract. Frames arrive continuously, the attention window gets fixed during idle-time prefill, and by the time a query arrives the cached context no longer matches what the question needs. FrameBlend, in a preprint, fixes the mismatch at query time with selective KV recomputation: keep the short idle-time attention windows that make prefill cheap, then recompute only the KV entries the query actually needs. The authors report per-frame prefill FLOPs down by over 20%, total processing time down 14 to 59% for retrieval-based streaming video understanding frameworks, and accuracy up by as much as 3.9 percentage points across five streaming and offline benchmarks. m:…6938

Note the direction of the accuracy number: this is not a trade where cheaper means dumber. Recomputing the right entries at query time also beat the standard frameworks on accuracy, which suggests the idle-time cache was not just expensive but wrong. That is the authors' reading of their own results, and it is the result most worth reproducing before building on it. m:…6938

The agent should choose what to watch

A survey preprint zooms out and asks what the architecture should look like when video understanding stops being passive inference and becomes an agent's job. The survey's taxonomy maps the field's bottlenecks to design patterns: where long-form video creates context bottlenecks, build state adaptively as the stream arrives; where evidence is sparse, acquire it actively instead of processing every frame; where modalities disagree, coordinate specialized roles to resolve the ambiguity. In this framing, the fix for FrameBlend-style context problems is structural: agents that decide what to remember and what to look at next, rather than pipelines that watch everything. m:…1649

The taxonomy is a design vocabulary more than a verdict. It gives builders a way to audit a video-agent architecture for gaps in evidence memory and temporal tracking, which is useful precisely because the field does not yet agree on how to measure these systems. m:…1649

Anomaly detection on a gigabyte

The third paper shows how far efficiency engineering has come on the detection side. MS2CAST is a video anomaly detection framework that drops optical flow estimation, traditionally the expensive step, and replaces it with interchangeable spatiotemporal modules, SwinLSTM or Vision Mamba blocks operating in the latent space, plus a multi-scale spatial cross-attention encoder. The headline constraint is peak GPU memory under 1 GB, with competitive ROC AUC scores on the UCSD Ped2, CUHK Avenue and ShanghaiTech datasets. m:…4169

Sub-gigabyte anomaly detection is a deployment story, not a leaderboard story. It puts always-on video monitoring within reach of edge hardware and small cloud instances where optical-flow pipelines never fit, which is where most real surveillance and monitoring budgets actually live. m:…4169

What this adds up to

Read together, the three preprints describe video AI growing out of the batch-processing era. FrameBlend makes streaming inference cheaper and more accurate at the same time, the survey gives builders a design vocabulary for agents that sample video actively, and MS2CAST shows the detection workload can live on modest hardware. The honest caveat, flagged by the survey authors themselves, is measurement: without standardized benchmarks for agentic and streaming video, every efficiency claim in this space is still self-graded. m:…6938 m:…1649 m:…4169