At its core, spatial intelligence is not a single technique, model, or task category. Rather, it constitutes a complex interdisciplinary ecosystem, spanning multiple technical paradigms, sensory modalities, scientific disciplines, and application domains. Different communities approach spatial intelligence with distinct motivations and meanings: for some, it concerns geometric consistency and localization; for others, it is about perception–action coupling, embodied reasoning, or spatial grounding of language. As a result, any attempt to define spatial intelligence in isolation risks oversimplification. Understanding spatial intelligence therefore requires acknowledging its breadth, heterogeneity, and layered structure as an evolving research ecosystem rather than a narrowly scoped subfield.
2.1.1Beyond A Single Dimension#
Spatial intelligence is fundamentally not simple. Unlike many AI topics that can be reasonably framed around a dominant abstraction or learning paradigm, spatial intelligence spans a wide spectrum of representational, computational, and conceptual elements. It touches almost every stage of the perception–cognition–action loop and manifests differently depending on the task, environment, and embodiment of the agent. This intrinsic complexity is not accidental; it reflects the fact that space itself is a unifying substrate of the physical world, cutting across sensory input, internal representation, and external interaction.
Historically, research on spatial problems in AI has progressed along parallel and often disconnected trajectories. Simultaneous localization and mapping (SLAM), 3D reconstruction, robotic manipulation, geographic information systems, spatial language understanding, and neural rendering all evolved largely within their own communities, with specialized benchmarks, metrics, and assumptions. While each of these areas addressed a slice of spatial reasoning, they rarely converged on a shared conceptual framework [207]. As intelligent agents are increasingly expected to operate in open, dynamic, and embodied environments, the limitations of this fragmentation have become increasingly evident.
One major source of complexity arises from the fact that spatial intelligence cuts across multiple technical paradigms. It cannot be reduced to either symbolic rules or neural representations alone. Instead, it involves the coexistence and interaction of diverse computational approaches:
Cross-Technical Paradigms. Spatial intelligence spans heterogeneous technical formalisms, including:
- •
- •
- •
- •
Equally important is the fact that spatial intelligence is inherently cross-modal. Space is not perceived through a single sensory channel, but emerges from the integration of multiple modalities, each offering partial and complementary information:
Cross-Modal Integration. Spatial intelligence operates across a diverse set of sensory and representational modalities:
-
•
Language, which encodes spatial relations, instructions, and abstract descriptions (e.g., , “behind”, “inside”, “near”) [213].
-
•
2D visual perception, including images and video, which provide appearance-level cues and spatial layout hints.
-
•
3D representations, such as point clouds, meshes, and volumetric fields, which capture geometric structure and physical extent.
-
•
4D spatiotemporal signals, incorporating motion, dynamics, and temporal evolution of scenes.
- •
A further, frequently overlooked axis is spatial scale. Cognitive research has long established that space at different scales engages different psychological processes: Montello’s classic analysis distinguishes figural (table-top), vista (single view), environmental (learned through locomotion), and geographical (map-mediated) spaces [214]. The same stratification recurs in AI systems, where an agent manipulating objects on a desk, a robot navigating a building, and a model reasoning over a city or the Earth’s surface demand different representations, memory structures, and update mechanisms [207]. Competence at one scale does not transfer automatically to another, which is one reason benchmark results on room-scale tasks say little about urban-scale spatial reasoning.
Cross-Scale Operation. Spatial intelligence must therefore operate across nested spatial scales:
-
•
Object and figural scale, covering manipulation workspaces, shape understanding, and mental transformation of objects.
-
•
Scene and vista scale, covering room-level layout, occlusion, and viewpoint reasoning.
-
•
Environmental scale, covering buildings, streets, and spaces learned through movement and exploration.
- •
Beyond modality, technique, and scale, spatial intelligence is deeply cross-disciplinary. Its theoretical foundations and practical realizations draw simultaneously from several scientific traditions:
Cross-Disciplinary Foundations. Spatial intelligence intersects with:
-
•
Computer Vision, through scene understanding, reconstruction, and spatial perception.
-
•
Robotics and Embodied AI, where spatial reasoning underlies navigation, manipulation, and interaction.
- •
- •
- •
- •
Finally, spatial intelligence manifests across a wide spectrum of application domains, further complicating any attempt at a narrow definition:
Cross-Application Presence. Spatial intelligence is central to applications such as:
- •
- •
- •
- •
Together, these dimensions highlight that spatial intelligence is not a standalone capability but a unifying layer across perception, reasoning, and action. This complexity precludes a minimal definition and instead calls for viewing spatial intelligence as a structured, evolving system that emerges across methods, modalities, and domains.
| Discipline | Primary focus | Representations | Canonical tasks | Evaluation emphasis |
| Computer Vision & Graphics | Geometry recovery and view-consistent scene appearance | Depth maps, multi-view geometry, meshes, neural fields (NeRF), 3D Gaussian representations | Depth/pose estimation, 3D reconstruction, neural rendering, scene synthesis | Geometric consistency, rendering fidelity, view synthesis quality |
| Robotics & Embodied AI | Perception–action coupling under physical constraints | Occupancy grids, semantic maps, pose graphs, affordance maps, world models for planning | SLAM, navigation, manipulation, motion planning, closed-loop control | Task success, robustness, real-time performance, generalization in dynamic environments |
| GIS & Large-scale Spatial Modeling | Global-scale spatial abstraction, querying, and planning | Geodetic coordinate systems, map layers, spatial graphs, spatial databases | Route optimization, spatial analytics, trajectory analysis, urban-scale simulation | Global consistency, query accuracy, scalability, long-horizon planning utility |
| Psychology & Psychometrics | Measurement of individual spatial abilities and their training | Factor models of spatial ability (rotation, visualization, orientation), standardized test batteries | Mental rotation tests, paper folding, perspective taking, spatial visualization assessment | Test reliability and validity, individual differences, malleability and transfer |
| Cognitive Science & Neuroscience | Mechanisms of human spatial cognition and internal representations | Cognitive maps, mental imagery, spatial memory representations, neural coding hypotheses | Mental rotation, navigation studies, spatial memory and attention experiments | Behavioral validity, explanatory power, neural plausibility |
| Linguistics & Spatial Language | How language encodes, schematizes, and constrains spatial relations | Spatial prepositions and closed-class relations, frames of reference (intrinsic, relative, absolute) | Spatial reference resolution, cross-linguistic comparison, grounded instruction understanding | Descriptive adequacy, cross-linguistic coverage, grounding fidelity |
| Social & Human-centered Perspectives | Spatial behavior in social contexts and shared environments | Proxemics, social space schemas, spatial cues for interaction, situational context models | Human–robot interaction, social navigation, spatial language pragmatics, collaborative tasks | Interaction naturalness, safety, alignment with human norms and intentions |
2.1.2Disciplinary Perspectives#
Although spatial intelligence is increasingly discussed as one concept, its meaning varies substantially across academic disciplines. Each field approaches spatial intelligence through its own problem formulations, representational assumptions, and evaluation criteria. As a result, what counts as spatial intelligence in one domain may only capture a partial aspect of what another domain considers essential. This part reviews how spatial intelligence is understood from several major disciplinary perspectives, highlighting both their unique emphases and their limitations.
Computer Vision and Graphics. From the perspective of computer vision, spatial intelligence is primarily associated with the ability to infer geometric structure from visual input. Research in this area focuses on problems such as depth estimation, multi-view geometry, camera pose estimation, and 3D scene reconstruction. The core objective is to recover spatially consistent representations from 2D images or videos, often under strong geometric constraints. These representations provide powerful spatial priors, enabling reasoning about layout, scale, and relative positioning of objects.
Closely related, computer graphics emphasizes the synthesis and rendering of spatial scenes. Modern neural rendering approaches, including NeRF and 3D Gaussian Splatting, prioritize appearance fidelity, view consistency, and continuous scene representation. While these methods excel at producing visually coherent 3D environments, they typically do not model agent-centric interaction, decision-making, or task-driven behavior. As a result, spatial intelligence in vision and graphics is often appearance-centered and offline, rather than action-oriented.
Robotics and Embodied AI. In robotics and embodied AI, spatial intelligence is traditionally defined in terms of an agent’s ability to localize itself, navigate environments, and manipulate objects. This includes spatial perception, motion planning in three-dimensional space, and interaction with representations such as occupancy grids, semantic maps, and volumetric models. Here, spatial intelligence is inseparable from embodiment and control, as spatial reasoning directly informs action execution. The central concern in this discipline is not only representing space, but using spatial representations to support real-time decision-making under physical constraints. Consequently, spatial intelligence is evaluated through task success, robustness, and adaptability in dynamic environments. This action-centric perspective highlights aspects of spatial intelligence that are often absent in purely perceptual or representational frameworks.
Geographic Information Systems and Large-Scale Spatial Modeling. At a larger spatial scale, geographic information systems view spatial intelligence through the lens of structured spatial data, coordinate systems, and topological relations. GIS focuses on modeling, querying, and reasoning over large-scale geographic environments using maps, graphs, and spatial databases. These systems excel at spatial querying, route optimization, and long-term planning across city or regional scales. However, traditional GIS frameworks are rarely coupled with embodied agents or interactive learning. Spatial intelligence in this context is largely static, symbolic, and detached from perception or action. This contrast highlights a scale-dependent interpretation of spatial intelligence, where global consistency and abstraction take precedence over local interaction.
Psychology and Psychometrics. Long before AI adopted the term, differential psychology treated spatial ability as a measurable dimension of human intelligence. Shepard and Metzler’s mental rotation experiments showed that the time needed to compare rotated 3D shapes grows linearly with their angular disparity, providing the first chronometric evidence of analog spatial representation [71], and the factor-analytic tradition from Thurstone’s space factor to Carroll’s broad visuospatial factor established spatial ability as a distinct, testable construct [226]. The term spatial intelligence itself was popularized by Gardner’s Frames of Mind (1983), which listed it among seven human intelligences and defined it as the capacity to perceive the visual world accurately, to perform transformations upon one’s initial perceptions, and to re-create aspects of visual experience even in the absence of the relevant stimuli; the construct, however, predates that label by half a century. Two findings from this literature matter directly for AI. First, spatial skills are malleable: a meta-analysis across hundreds of training studies shows they improve with practice and transfer across tasks [216], suggesting spatial competence is acquired rather than fixed. Second, the psychometric task inventory, covering mental rotation, perspective taking, and spatial visualization, has become a direct template for probing spatial understanding in multimodal foundation models, so this measurement tradition now feeds AI benchmarking rather than sitting apart from it [227].
Cognitive Science and Neuroscience. In cognitive science and neuroscience, spatial intelligence is studied as an emergent property of interacting neural systems rather than as an engineered representation. Research has identified key brain regions associated with spatial cognition, including the hippocampus for spatial memory and navigation (place cells, and grid cells in entorhinal cortex), the parietal cortex for spatial attention and transformation, and occipito-parietal networks for visuospatial processing [51, 217]. From this perspective, spatial intelligence arises from neural mechanisms supporting cognitive maps, mental imagery, and spatial reasoning; it is primarily internal and cognitive, centered on perception and imagination rather than external interaction.
Linguistics and Spatial Language. A further disciplinary lens comes from linguistics, which asks how language encodes and constrains spatial thought. Talmy’s analysis of how language schematizes space [228] and Landau and Jackendoff’s account of the asymmetry between rich object naming and the small closed class of spatial prepositions [213] established that linguistic descriptions compress spatial structure into coarse, topology-like relations. Levinson’s cross-linguistic work further showed that languages differ systematically in their spatial frames of reference, distinguishing intrinsic, relative, and absolute systems [215]. This vocabulary has migrated directly into AI evaluation, where egocentric versus allocentric reference handling is now a standard axis of spatial benchmarks for multimodal models, making the linguistics of space an operative rather than merely historical influence.
Social and Human-Centered Perspectives. Beyond physical geometry, spatial intelligence also carries meaning at the social and mental levels. Human beliefs, emotions, and behaviors are situated within social environments that are themselves spatially structured. From this viewpoint, spatial intelligence encompasses the ability to reason about others in shared spaces, interpret spatial cues in social contexts, and align internal mental states with external environments. This perspective extends spatial intelligence beyond physical navigation toward a unified model of the physical and mental world. It points to broader scientific and societal opportunities, including AI systems that operate in unstructured real-world settings and interact naturally with humans. In this sense, spatial intelligence supports a joint modeling of physical space and human cognition, serving as a bridge between external environments and internal mental states.
2.1.3Toward A Unified View#
For decades, research on spatial reasoning and spatial representation has progressed from markedly different perspectives across multiple subfields. These efforts often evolved in isolation, lacking a shared vocabulary, common research objectives, or directly comparable evaluation benchmarks. As a result, techniques developed in areas such as SLAM, 3D vision, scene modeling, world modeling, and embodied planning frequently addressed overlapping problems without converging toward a unified conceptual framework. Spatial intelligence emerges against this backdrop as a response to fragmentation, offering a higher-level abstraction that can organize and align these historically disjoint research trajectories.
Under the emerging spatial intelligence paradigm, a set of shared demands and core concepts has begun to surface. A central driving force behind this convergence is the growing need for agents that exhibit spatial consistency and physical groundedness, and that can operate continuously within dynamic environments. Such agents must integrate perception, representation, and action in a coherent manner. This requirement has catalyzed the fusion of visual perception with geometry-aware modeling, as seen in navigation-oriented neural rendering approaches and interactive NeRF-based systems. Research at the intersection of neural scene representation and active planning has given rise to a new generation of 3D world models that are not only reconstructive, but also executable and actionable. Similarly, vision-language models are increasingly extended to operate in three-dimensional environments, enabling multimodal agents to localize, describe, and interact with objects within voxelized or volumetric scenes.
At the conceptual level, a set of shared primitives is gradually taking shape. Spatial representations, whether instantiated as voxel grids, signed distance fields, neural feature volumes, or topological graph structures, are becoming central components across domains ranging from autonomous driving and AR/VR to assistive robotics. Fundamental spatial relations, such as containment, adjacency, support, and traversability, are being formalized and embedded into reasoning modules, scene graph systems, and relational neural networks. These abstractions provide a common representational substrate that enables knowledge transfer and system interoperability across tasks and modalities.
We note that the research community has yet to settle on a single taxonomy for this emerging field. Recent surveys and position papers organize spatial intelligence along markedly different axes: by task family and benchmark [229], by the cognitive functions that video-based spatial tasks exercise [227], by spatial scale from embodied agents through cities to Earth systems [207], or, in Li’s position essay, by the capabilities a world model must possess, namely being generative, multimodal, and interactive [5]. These framings overlap without coinciding, and none has achieved consensus status. This handbook’s representation-centric organization, which follows how spatial representations are built, used, and acted upon, is our deliberate choice among these alternatives: it is the axis along which the historical evolution of Section 1.2 unfolds most naturally, and the one that lets fragmented communities be compared on common ground.
Taken together, these developments indicate that spatial intelligence is no longer confined to any single domain. It is evolving into a unifying functional substrate that spans perception, reasoning, and interaction; bridges vision, language, and action; and connects symbolic representations with neural models. The unification runs deeper than shared technique. Spatial intelligence aligns and integrates multiple disciplines, including computer vision, robotics, psychology, cognitive science, linguistics, geospatial systems, human-computer interaction, and neuro-inspired AI, and through this alignment it functions as the connective tissue that enables the next generation of generalist AI systems to perceive the world, reason about it, and act within it.