Advances in spatial perception, reasoning, and generation have enabled AI systems to recover geometry from images, answer questions about layout, and simulate complex environments, as Parts II to IV documented. Success on these tasks does not establish that a system maintains a reliable model of the physical world. A model can reconstruct a room to millimeter accuracy, answer most questions about a photograph, or continue a video convincingly, and still fail when observations are incomplete, when the environment changes while it is not looking, or when its actions depart from the conditions under which it was trained. The failures reported throughout this handbook cluster at these points, which suggests a shared deficiency rather than a collection of unrelated task-level shortcomings.
This chapter asks what makes an internal model of physical space usable for continued reasoning and action. By usable we mean that the model can be queried for what a task requires, extended when new evidence arrives, and relied upon when its predictions are acted on; we do not presuppose that such a model must be an explicit 3D reconstruction, nor that it must be a video generator. The question decomposes into five, each posed at a different level. What can be inferred from limited observations, and what remains underdetermined (Section 15.1)? Which information must a representation retain, and for which operations (Section 15.2)? How can a world state be maintained and revised over time (Section 15.3)? When can its predictions be trusted (Section 15.4)? How is spatial knowledge acquired and used through interaction, in environments shared with other agents (Section 15.5)? The five are interdependent: action produces new evidence, which re-enters at the first level and constrains all of the others.