World models learn by predicting what happens next

The shared bet

Five papers in a preprint point in the same direction: a model that acts in the world should first predict how the world will change. World action models learn latent variables from observations and use them both to predict how a scene evolves and to generate actions, so prediction and action are two outputs of one model rather than two separate systems. m:…7757

The papers differ in what they fix, but they share a diagnosis of the current gap. Existing methods model short-term dynamics and long-term task objectives separately, according to the HiFOR authors, which leaves a disconnect between what happens next and what the task ultimately requires. Standard world action models have a second weakness: they do not guarantee that actions are generated solely from action-relevant latent variables, and may entangle those variables with nuisance variables that prediction happens to use. m:…4345 m:…7757

Predicting at several horizons

HiFOR, a hierarchical foresight framework for robotic manipulation, predicts at three levels at once: short-term state evolution, semantic milestones along the way, and final goal states. A Hierarchical Dependency Mask establishes dependencies across those temporal horizons, so the near-term forecast and the distant goal are not produced independently. m:…4345

The framework also organizes its picture of the future before it acts on it. Structure-Aware World Modeling organizes future latent representations along horizon, view, and spatial dimensions, and Action-Guided World Retrieval then retrieves relevant future information based on evolving action representations during denoising, providing targeted guidance for action generation. The authors tested HiFOR on two simulation benchmarks and six real-world robotic manipulation tasks. m:…4345

Separating what causes an action

CausalWAM starts from a sharper worry: even if a world model predicts well, its actions may rest on the wrong variables. The method separates action-causing variables from prediction-dependent variables, and the authors show that under a sparsity constraint, the latent variables used to generate actions are an invertible function of the action-relevant latent variables alone. In practical terms, the action head is meant to depend only on what genuinely causes the action. m:…7757

The implementation keeps the rest of the model intact. CausalWAM incorporates a sparse routing path into the action head of a pretrained world action model, while the world modeling objective remains unchanged. Experiments on synthetic data and on real-world robots validate the theory, and the authors report improved efficiency in closed-loop success. Identification holds even when action-relevant variables are statistically dependent on others, and requires no temporal independence assumptions. m:…7757

For developers, the point is discipline in what the action head is allowed to see: a model that predicts a scene well can still act for the wrong reasons if its actions borrow correlations that break when conditions change. m:…7757

Video as training signal, code as action

WorldBridge tackles a different bottleneck: robot action data is scarce, while video is plentiful. The model improves robot task generalization by combining video prediction with sparse action supervision, using separate video and action experts. Broader video coverage enables task transfer by leveraging existing inverse-dynamics capabilities, according to the authors, so video teaches the model how the world moves and a small amount of action data teaches it how to intervene. m:…7224

The reported numbers show why the combination matters. On RoboTwin 2.0, video-only supervision reaches 55.90%, while in-distribution video-action supervision reaches only 25.07%. Adding a single action-labeled demonstration to video lifts the result to 70.70%, and the full data setting reaches 96.23%. The model also reaches 80.9% on a clean-to-random RoboTwin 2.0 setting, 82.8% on RoboCasa GR1 TableTop, and 57.6% on DOMINO Level-1, with visual memory at two temporal scales and joint flow matching training behind those results. m:…7224

CaPtain takes yet another route to action: code. It is an open post-training pipeline that enables small open vision-language models to perform robot manipulation through code, without requiring frontier models or vision-language-action backbones at test time. The base model is aligned via supervised fine-tuning on synthetic trajectories, and a physical critic reads privileged simulator state not seen by the policy to credit multi-turn agent requests during reinforcement learning. m:…4309

The trajectory is steep. CaPtain-4B starts from Qwen3.5-4B, which has 0% zero-shot success, reaches 55.4% held-out LIBERO success after supervised fine-tuning, and reaches 73.8% after reinforcement learning. It also reaches 78.5% on LIBERO-PRO and 83.3% on SimplerEnv WidowX without target-domain training, and runs on a real robot after simulation-only adaptation. The authors released all code, data, and model weights, which makes the pipeline checkable in a way closed demonstrations are not. m:…4309

Games as a proving ground

Wizard shows the same forecasting idea in a setting where real-time control is unforgiving: games. It is an agentic game world model that uses structured motion proxies and adaptive masking, and it decouples character animation, scene manipulation, and gameplay logic from visual generation. That separation is what lets it generalize to novel environments and characters rather than memorizing one scene. m:…2898

The system achieves interaction at 24 frames per second using causal adaptation and few-step distillation, trained on a dataset spanning 750 scenes and 178 characters with 50.3 hours of aligned proxy-video pairs. That speed matters because a slow predictor cannot steer an interactive character. m:…2898

Across all five papers, the pattern is consistent: prediction is becoming the control surface. Whether the forecast is hierarchical, causally filtered, learned from video, expressed as code, or run at game speed, the model acts by consulting what it expects to happen next. The open question is how far that pattern transfers beyond the benchmarks and tasks each team tested, and whether the discipline each paper adds survives contact with messier real-world conditions. m:…4345 m:…7224 m:…4309