Vision models pass the single frame and lose the scene: consistency fails across views and time

A good-looking frame is not a consistent world

In a preprint, researchers introduce WorldProbe, a diagnostic benchmark for video world models during long-horizon, goal-conditioned spatial exploration, with 164 test cases across four probing families. The finding is a split: current models degrade in long-range memory and evolution despite maintaining local visual quality. A generated world can look right moment by moment while forgetting where things were, which is precisely the failure that matters if the model is meant to support exploration, planning or a robot that must return to a place it has already seen. m:…4885

OneScene-Bench, a second preprint, asks a narrower version of the same question: can a vision-language model integrate evidence across multiple views of one scene, rather than processing each image in isolation? The authors report that models largely do not, relying instead on independent per-view processing. Counting accuracy drops by 4.5 to 50.0% when moving from complete-overlap views, where every view shows the same thing, to complementary views, where each view adds something new, and cross-view consistency remains near 50% across both open-weight and proprietary models. A model that counts well when it can see everything at once, and fails when it must combine partial views, does not have a scene representation. It has a collection of pictures. m:…4674

The model can know a fact and still not use it

A third preprint, introducing the controlled BedBench benchmark across thirteen procedurally generated 3D tasks, finds a more specific breakdown. Even frontier models struggle with basic spatial property recovery. More revealingly, probing a Qwen3.5-27B model shows it can recover per-colour counts at the input level, at 65.0% probe accuracy, while scoring only 3.0% when it must use those counts in its final reasoning. Fine-tuning on 3D reasoning tasks improves accuracy but does not resolve the inconsistency between internal property counts and final answers, the authors report. The knowledge is present and the reasoning does not use it, which means aggregate accuracy on a benchmark can improve while the underlying failure stays in place. m:…3288

BEHAVIOR-Spatial, a fourth preprint, carries the question into robotics, testing vision-language and vision-language-action models on 3,256 spatial questions and 1,240 grounding queries. Vision-language-action models fall from 41.6% to 16.1% when instructions change within a fixed layout, even though the scene itself has not changed. Giving the model oracle visual target cues lifts performance to 60.3% on held-out tasks and 38.0% under changed instructions, which suggests that grounding, knowing which thing in the scene the instruction refers to, is a large part of the gap. The authors also report a policy-dependent correlation between spatial question-answering accuracy and manipulation performance, meaning a perception score is not a reliable proxy for robot task success on its own. m:…9497

Grade consistency, not just the answer

The practical lesson across these preprints is about evaluation. If a vision system is only tested on single images, single views or fixed instructions, it will look more capable than it is in any deployment where the camera moves, the task continues, or the user rephrases the request. The tests that expose the gap are complementary views, long rollouts, changed instructions and probes that compare what a model internally recovers with what it finally says. A developer choosing a vision model for a robot, a world simulator or a multi-camera system should ask for those numbers first, because a model that cannot keep a scene consistent will fail in the exact conditions where consistency is the job. m:…4885 m:…4674 m:…3288 m:…9497