The problems in this chapter are widely acknowledged and concern missing methods, inadequate measurements, and systems that work in the laboratory but not yet in open environments. Section 16.1 describes research gaps along the loop of Figure 2.1, Section 16.2 gaps in how progress is measured, Section 16.3 bottlenecks in deployment, and Section 16.4 directions for future work.
Chapter 15 asked why these problems are hard in principle; this chapter asks where current systems fail, how the failures can be measured, and which methods could be tested against them. Four themes are its own: competence at tabletop or room scale does not transfer to buildings and cities; correct spatial judgments do not reliably become completed long-horizon actions; much evaluation does not measure the abilities it is named for; and deployed systems face latency, distribution shift, security, data scarcity, and multi-agent synchronization that research systems rarely confront. Where a failure has a principled explanation in the preceding chapter, we point to it rather than repeat it.