A Foundation-First Perspective on Spatial AI: Geometry, Memory, and Agents
Associate Professor of Aeronautics and Astronautics, Director of MIT SPARK Lab
Attendance Link: https://docs.google.com/forms/d/e/1FAIpQLSdzU9VehavfTCy6zZk7vXcXHEx-FtcWxWNqa091cx10YlUvSQ/closedform
Abstract
Recent advances in geometric foundation models and multimodal large language models (MLLMs) are challenging decades of conventional wisdom in robot perception, opening new opportunities to rethink how robots perceive, understand, and remember the world. In this talk, I will first explore how the internal representations learned by geometric foundation models can provide a common foundation for a wide range of tasks, from SLAM and visual-inertial navigation to scene-change detection and robotic manipulation. I will then shift the focus from geometry to memory, asking what information robots should retain from their observations and experiences to support long-term autonomy. I will discuss recent advances in structured and hierarchical memory representations, examining their benefits and limitations compared to MLLM-based full-context approaches. Finally, I will share my perspective on the emergence, impact, and limitations of agentic AI in robotics.
Bio:
I am an Associate Professor in the Department of Aeronautics and Astronautics at the Massachusetts Institute of Technology, a Principal Investigator in the MIT Laboratory for Information & Decision Systems (LIDS), and an Amazon Scholar.
I am the director of the MIT SPARK Lab, where we work at the cutting edge of robotics and autonomous systems research. My goal is to enable human-level perception and world understanding on mobile robotics platforms (drones, self-driving vehicles, ground robots) operating in the real world. Towards this goal, my work involves a combination of rigorous theory and practical implementations. In particular, my research interests include nonlinear estimation and probabilistic inference, numerical and distributed optimization, machine learning, and geometric computer vision applied to sensing, perception, and decision-making in single and multi-robot systems.
Human-level perception will increase reliability in safety-critical applications of robotics and autonomous vehicles (including self-driving cars and robots for disaster response), and increase efficiency and effectiveness in service robotics and consumer applications (manufacturing, healthcare, domestic robotics, augmented reality).