A video generator asked to continue a clip of a ball rolling off a table usually produces a smooth, well-lit continuation in which the ball falls, often along the wrong trajectory or with an implausible bounce. Benchmarks of physical plausibility find that the physical accuracy of video generators is poor and does not improve with visual quality (Section 7.5), and Chapter 8 reported the same pattern across its subfields: shapes that change between views, feet that slide over the ground, and motions that violate conservation (Section 8.8). A plausible continuation reproduces the appearance of a physical event without predicting its trajectory, contacts, or consequences. Yet the same class of models learns substantial scene structure, since image generators adapted for perception segment, estimate depth, and ground referring expressions at the level of specialist models [1136]. Structure and dynamics are apparently learned to different degrees from the same data.
World-model correctness has several distinct meanings, and the criteria are often run together. Visual plausibility concerns realistic appearance; geometric consistency concerns coherent spatial structure across views and time; dynamical fidelity concerns how the world evolves under actions; decision reliability concerns whether actions planned in the model succeed in reality. The criteria are related but not interchangeable, and they are ordered by how much they depend on action. Common video metrics measure the first and do not register physically impossible motion (Section 8.3.6); view-synthesis metrics measure the second and cannot tell a solid object from a cut-out that matches the training views (Section 8.2.1); the third has no standard protocol (Section 8.6.4); the fourth is tested, when it is tested at all, by deploying the resulting policy. A model can score well on the first two and fail the last two, and vision-language models used as judges share the spatial blind spots of the models they evaluate (Section 8.8).
Learning how scenes typically evolve from observational data does not guarantee correct predictions under novel interventions. In the action-conditioned definition of Chapter 2, maps a state and an action to the next state, but a generator trained on video sees only the actions that happened to be recorded, mostly everyday activity and camera motion, and sees them correlated with the states and circumstances in which they occurred. A model that has watched thousands of doors being pushed open has learned what usually follows a push, not what determines the outcome, and so has little basis for predicting the locked door, the obstructed door, or the push applied at an unusual angle. The regularities it learns are real and useful for geometry, appearance, and common motion; they constrain little about the consequences of an action outside the data. An interventional test, which fixes the initial state, varies the action, and checks whether predicted outcomes vary as real outcomes do, is the most direct way to distinguish a model of what tends to happen from a model of what an action causes.
Accurate geometry does not imply an actionable physical model. Mass, friction, articulation, and compliance are often visually unobservable yet critical for manipulation (Sections 7.1.2 and 9.1), and a Gaussian-splat replica of a room can be photometrically faithful while encoding nothing about which drawers open or how much force a door requires (Section 11.1). Real-to-sim pipelines estimate such quantities by manipulating objects and observing the response [196], thereby estimating properties needed for prediction, and world-action models that predict compact encodings of future observations rather than pixels avoid spending capacity on appearance that control does not use [3016]. Confidence sets calibrated under one data-collection policy may become unreliable under policy shifts [3017], and model-based action selection may induce policy shifts that invalidate this calibration.
A tractable form of the question is where, rather than whether, a world model can be trusted. Treated like a measuring instrument, a world model would be calibrated against the environment it claims to simulate, carry a stated range of states and actions within which it is valid, and be used with caution outside it. Establishing that range requires tests under controlled interventions and unfamiliar actions, under environmental change, and against downstream decisions, in place of average visual or predictive accuracy. World models expressed as code, whose rules are explicit and correctable when they conflict with observation, admit such tests more readily than learned ones (Section 8.6.2.3), and the natural sciences already hold generative models to this standard, counting a proposed material only once it has been synthesized (Section 12.2). When an action falls outside a model’s established range, the agent must decide how to obtain further evidence, which actions are safe meanwhile, and how to use the outcome to extend the model.