The representations surveyed so far are organized by modality, each general enough to span many applications. A large body of practice, however, develops representations tailored to a specific sensor or downstream task rather than to a modality as such, and these are collected here. What unites them is not a shared signal but a shared design logic: each encodes the inductive biases of its data and trades spatial fidelity against semantic compressibility in the way its sensors and target tasks demand. This section walks through five such domains: embodied robotics, virtual reality, geospatial and remote sensing, clinical and scientific imaging, and indoor intelligent assistants, examining in each the representations that have become canonical and the constraints that shaped them. Beginning with robot perception and planning, systems commonly take as input RGB-D sequences and point clouds (often fused over time into semantic maps that ground observations into a common 3D frame [114, 334]), and enable tracking, map building, and manipulation. Many such systems also encode visual input into compressed spatial representations, such as keypoints, dense correspondences [335], and point tracks, used to parameterize 6-DoF grasps [336] and object-centric actions. When available, free space is typically represented with occupancy grids [79, 89] and signed-distance representations [109], and is often additionally augmented with semantic labels such as traversability and cost for planning. To further enable action from perception, planners frequently operate on affordance representations [337] over parts and candidate contact points, encoding feasible interactions such as graspability or predicted contact quality, thereby enabling policies to execute tasks successfully.
Real-time virtual reality systems often represent scenes with triangle meshes and physically based rendering (PBR) materials, where geometry, normals, and texture define appearance, while 6-DoF camera poses keep virtual content registered to the environment. To enable flexible interaction, such systems include captured geometry (e.g., , spatial meshes, planes, or TSDF and SDF volumes [109, 114]) and depth-based occlusion layers, which are often paired with collision proxies and semantic labels. Toward photoreal view synthesis, methods increasingly use neural scene representations (e.g., , NeRF-style radiance fields and 3D Gaussian splats [119, 121]), to support novel-view rendering and relighting. Additionally, for the purposes of enabling characters in virtual reality systems, such characters are commonly represented with skeletal pose and expression parameters (e.g., , SMPL-style body models [338] or Blendshapes), and their navigation relies on navmeshes combined with acceleration structures (e.g., , octrees), to handle moving agents in simulation at high frame rates.
In remote sensing applications, geographic information systems typically represent the world (the Earth), as georeferenced raster fields, for example, optical multispectral and hyperspectral imagery as well as SAR backscatter. These fields are aligned to explicit coordinate reference systems and sensor geometry metadata. Such systems encode terrain through multiple elevation products (e.g., , DEM, DTM, DSM), and compress human infrastructure into vector networks (e.g., , roads, power grids, pipelines), often across scales. And increasingly, they use LiDAR to not only output raw point clouds, but also to output rasterized height maps that support flexible 3D reasoning. These frameworks also commonly operate on multi-temporal stacks and spatiotemporal data with validity masks, which can generate change maps for land-use dynamics and disaster response, as well as feed digital-twin pipelines that generate orthomosaics and photogrammetric point clouds [115], for simulation and virtual modeling.
Clinical and scientific imaging operate on volumetric scans (e.g., , CT and MRI), in which 3D voxel grids capture geometry and intensities [268], and explicit spatial transforms (e.g., , rigid, affine, and nonlinear deformation fields) align subjects and modalities into a shared atlas space. In addition, functional measurements (e.g., , fMRI BOLD and calcium imaging) are often stored as 4D spatiotemporal sensors, but are frequently compressed into time series and activation maps after parcellation. When available, such systems represent structural delineations (e.g., , tissue, organ, lesion) as segmentation masks and labeled maps [174], which are often paired with surface meshes (e.g., , cortical surfaces) that enable surface-based topological comparison across subjects. For diffusion imaging applications, tensor and orientation fields encode local diffusion directions and magnitudes, while tractography streamlines uses these fields to reconstruct fiber pathways and uncover possible structural connectivity. These diverse representations across granularities and temporal scales are often mapped onto one another through atlas spaces and parcellations, enabling multimodal fusion and cross-subject analysis.
Intelligent assistants that operate in indoor environments typically ground language in 3D space using multi-scale maps, for example, 3D reconstructions and floorplans anchored by an estimated 6-DoF agent pose [94, 339, 340], which are often combined with semantic maps that encode navigable routes. Context for interactions is captured in structured world models (e.g., , scene graphs with typed entities and relations [341, 342, 343]), which can be augmented with additional semantics like anchors, keypoints, and regions of interest. For the purposes of intelligent and persistent assistance, representations that encode some form of memory are necessary, for example, episodic records and long-horizon object trajectories, which are often represented as probabilistic beliefs over object locations in a room. In addition, assistant systems can optionally contain geofences and occupancy masks, to help define actionable regions for planning.
In summary, many domains have developed representations specifically designed for specialized sensors and encoding the inductive biases of the underlying data structure. We see that these representations model a tradeoff between (spatial) fidelity and (semantic) compressibility, reflecting how the aforementioned sensors sample space, what priors are built in, and what downstream tasks are to be supported.