Understanding a scene at one moment differs from maintaining an understanding of a changing world. An assistant asked where the keys were left must have observed them, retained their location after the user walked away, distinguished that remembered location from the current view, and allowed for the possibility that someone moved them in the meantime. Egocentric benchmarks built around such queries show that an explicit memory, updated as video arrives and tracking objects relative to the moving user, substantially improves leading multimodal models [3009], whereas most commercial glasses answer from a single frame and keep no map (Section 9.3). The household robot in this part’s introduction faces the same requirement at every step. In the formalism of Chapter 2 this is the role of the update , the component that Parts II to IV examined least.
Access to a long observation history does not ensure a consistent world state. Video world models steered through long explorations fail to render revisited places consistently even as image quality improves [3010]; multimodal models given several views of a room rarely detect that one view was captured after an object moved [3011]; and over routes spanning several streets, models answer by recognizing landmarks and matching text, with little evidence of path integration [3012]. A model may retain every earlier frame and still fail to recognize a revisited place, preserve an object’s identity, or decide which remembered properties have become outdated. Research on spatial memory has concentrated on recall, through longer contexts, better retrieval, and richer memory structures (Section 7.3.6), while these failures concern revision. Persistence therefore involves maintaining and revising beliefs about the world, not merely retaining observations.
A persistent world state must distinguish absence of observation from evidence of absence, maintain uncertainty during occlusion, and reconcile new observations with prior beliefs. When a new observation conflicts with a stored one, the system must decide whether the environment changed, the earlier estimate was wrong, or the two observations refer to different entities, each requiring different state updates. Classical systems partly address this problem: SLAM detects loop closures and corrects the map upon revisiting a known place (Section 6.3.5), object trackers maintain identities through occlusion, and dynamic scene graphs timestamp their edges. Learned systems often struggle to make such revisions reliably over extended interactions. Models retain outdated beliefs after the world changes [231], append-only memories accumulate contradictory entries, and agents drift over long episodes as their context fills (Section 7.3.7). Increasing capacity alone may not suffice, since a memory that only grows keeps each outdated belief beside its replacement. A write policy that decides what enters the state, with what confidence, and when an entry is revised or removed, is a prerequisite that current systems leave largely implicit.
Where such a state should be kept is a question of architecture rather than of representation format. Explicit structures such as maps, scene graphs, and splats (Section 4.2) persist by construction and make selected state variables accessible and correctable, but they must anticipate every type of object, property, and change they will be asked to encode. Learned memories and recurrent latent states can encode properties no designer specified, and whether they track the world is difficult to verify [3008]. Generated frames are the most flexible option and lose whatever leaves the context window. Hybrid designs that store static geometry and dynamic objects separately and retrieve them on revisit [3013], or that keep slowly changing layout and identities in an explicit layer and leave appearance and short-term dynamics to a learned one, combine these advantages at a cost of their own: geometry, object identity, uncertainty, and temporal updates must stay synchronized across layers, and coordinate frames and scales can silently diverge at the interface (Section 7.4). The separation can be taken further. State World Models keep a typed state ledger that evolves under executable rules written by a coding agent and use a frozen video model only to render each resulting event; on counterfactual twin pairs that share an initial frame and action but differ in hidden state, this harness produces the rule-required outcome in 92% of pairs, where the best of eleven video models prompted with the same rules reaches 32% [3014]. The result suggests that what fails in generator-only designs is the maintenance of state the current frame does not show, rather than rendering. The appropriate architecture follows from the requirements of state maintenance, how long each part must persist and whether it must be inspected, rather than from a general preference for explicit or learned form.
World states should remain persistent without becoming rigid and revisable without losing established structure, and evaluation currently measures neither property. Most benchmarks refer to a single moment, and the protocols that would expose persistence failures are not yet standard (Section 7.5): streaming protocols that withhold future frames [3015], revisit protocols that move the camera away and back [3010], and change protocols that alter the scene behind the agent. A more informative evaluation reads out the state itself, as Theory of Space does at each exploration step [231], and scores separately what the system has observed, what it remembers, and what it predicts, since conflating the three is how a stale belief passes as a current one. Deployed systems could likewise report how far their world state lags behind the world it describes, a quantity that digital twins call the liveness gap (Section 11.1) and that is rarely measured elsewhere. A state that remains consistent over hours can nonetheless describe appearance accurately and physical behavior poorly.