Audio is fundamentally tied to spatial information. Unlike vision, which directly captures the geometric layout of a scene, acoustic signals encode spatial structure only implicitly, through the physical transformations imposed on sound during propagation. Recovering space from sound is therefore a matter of reading these transformations backward: from the perceptual cues that human hearing exploits, to the computational representations that formalize them, to the learned models and cross-modal couplings that ground sound in the surrounding geometry. This section follows that arc. Human auditory perception interprets the resulting acoustic cues, shaped by these transformations, to infer the spatial layout of the environment. The primary cues are binaural: the Interaural Time Difference (ITD) and the Interaural Level Difference (ILD), both arising from the geometry of the head and the spatial separation of the ears [320, 321]. ITD reflects the difference in arrival time between the two ears and is highly reliable for localizing low-frequency sounds whose long wavelengths exceed head size. Conversely, ILD captures sound pressure level differences caused by head-shadowing effects and serves as a dominant cue for high-frequency localization where such attenuation is more pronounced. However, binaural cues alone cannot fully resolve depth, elevation, or front-back ambiguities.
To address these limitations, the auditory system further exploits monaural spectral cues and environmental reverberation. Direction-dependent spectral filtering produced by the torso, head, and pinnae introduces characteristic notches and peaks, enabling inference of elevation and disambiguation of positions on the cone of confusion [322]. In addition, the room impulse response (RIR) shaped by early reflections, late reverberation, and surface materials encodes information about source distance, enclosure geometry, and environmental structure [323, 324]. These monaural and environmental cues complement binaural information, jointly supporting robust spatial inference under naturalistic conditions.
Computational models formalize these perceptual cues through structured acoustic representations. Head-related transfer functions (HRTFs) [325] capture listener-specific spectral filtering and are foundational to perceptual research and spatial audio rendering. In parallel, spherical-harmonics–based ambisonics representations [326] provide a scene-centric encoding of the sound field around a point in space, with controllable angular resolution. Together, these representations describe how sound interacts with the body and the environment, enabling principled approaches to spatial reasoning, source localization, and scene reconstruction. Taken together, the ability of the auditory system to integrate binaural, monaural, and environmental cues demonstrates that sound carries rich implicit structure about the spatial configuration of sources and the surrounding environment.
Recent learning-based approaches aim to capture spatial structure directly from raw audio, going beyond analytic models. Neural encoders trained on spatially annotated or simulated data learn to infer direction, distance, and environmental geometry from multi-channel waveforms or spectrograms [47, 327]. Self-supervised frameworks further exploit the natural co-occurrence between sound and space: SoundSpaces [46] and VisualEchoes [328] train models to predict sound propagation or reconstruct echoes from static images. These learned representations encode not only spatial acoustics but also grounding in surrounding geometry, forming a key component of spatial-aware representation.
In real-world environments, spatial audio interacts closely with other sensory modalities. Audio–visual models align acoustic cues with visual observations to localize sounding objects and understand where events occur [329, 330, 331]. Embodied agents further leverage auditory signals to perceive layout and navigate toward acoustic targets, as demonstrated in audio–visual navigation [332, 333] and egocentric audio reasoning [46]. Through such cross-modal grounding, audio provides complementary spatial cues that enhance robustness under visual occlusion and enable reasoning about events beyond the agent’s field of view. This makes audio a powerful complementary modality for spatial representation, especially in scenarios where visual information is partial, noisy, or unavailable.