Spatial understanding begins with an incomplete view of the world. A system never has direct access to the world state of Chapter 2; it has images, video, depth, or sound, from which that state must be inferred. Images and videos carry rich evidence about appearance and structure, but they do not uniquely determine every geometric, physical, or functional property of a scene. A single image fixes the scale of a scene only through the prior knowledge brought to it (Section 6.2.3), an occluding surface hides whatever lies behind it, and the mass of a box or the state of a lock leaves no trace in a photograph. A usable model must therefore keep track of which of its contents are observed and which are inferred.
Whether a spatial inference fails because the model cannot extract the evidence available, or because the evidence itself is insufficient, is a distinction current evaluation rarely draws. The former is a limitation of learning or computation, and better models or more data can be expected to reduce it. The latter is a problem of identifiability, which more capable inference alone does not remove. Asked whether a sofa seen in one photograph will fit through a door seen in another, a multimodal model typically produces a confident answer with a coherent explanation, yet without a common scale between the two images the question has no determinate answer, and a model with perfect geometric reasoning should say so. Evaluation that counts every wrong answer as a model failure conceals this class of cases, while evaluation that accepts every ambiguity as irreducible would excuse failures that are not.
Additional modalities constrain different aspects of the underlying state, and their contributions are neither interchangeable nor universally sufficient. A second viewpoint supplies parallax, which fixes relative depth and reveals part of what the first view occluded. A depth sensor supplies distances directly, with an accuracy and a scale behavior that depend on the sensing principle (Section 6.2.1). Motion adds constraints on dynamic geometry; sound carries direction, distance, and room structure (Section 6.5.1); contact and force reveal mass, friction, and compliance, which no camera observes (Section 7.1.2). Assessing a modality therefore requires specifying which ambiguities it resolves, under which assumptions, and which properties remain unidentifiable once it has been added. Part II treated the modalities as alternative representations of space (Chapter 4); here they appear as complementary constraints on one underdetermined state.
Observability is not fixed by the initial input. Moving to another viewpoint, changing the illumination, or pushing an object can reveal previously ambiguous properties, and some properties, such as a drawer’s operability, may require physical interaction. Theory of Space and VABench bear directly on this point: models perform better with supplied observations than when they must gather evidence themselves, and allowing a model to move its camera and look again raises its manipulation success substantially [231, 3005]. Spatial intelligence therefore depends partly on recognizing when an additional observation would change what is knowable. How an agent should choose such observations is a question of decision making, taken up in Section 15.5, which properties are identifiable depends on the agent’s available actions.
Distinguishing uncertainty from fundamental underdetermination, and stating what evidence would resolve it, are abilities that current systems are not asked to demonstrate. Static question accuracy cannot distinguish correct reasoning under insufficient evidence from incorrect reasoning despite sufficient evidence, and controls as simple as removing the image or shuffling the input remain rare (Section 6.5.2). A benchmark designed around identifiability would include questions that are unanswerable from the given input and reward a model for saying so, pair each ambiguous case with the modality or action that resolves it, and score whether the model asks for it. Given these limits, which obtainable properties should a spatial representation preserve?