Back to All Events

Luca Carlone (MIT)

  • Stanford University Skilling Auditorium 494 Lomita Mall Stanford, California, 94305 United States (map)

A Foundation-First Perspective on Spatial AI: Geometry, Memory, and Agents

Luca Carlone

Associate Professor of Aeronautics and Astronautics, Director of MIT SPARK Lab

Attendance Link: https://docs.google.com/forms/d/e/1FAIpQLSdzU9VehavfTCy6zZk7vXcXHEx-FtcWxWNqa091cx10YlUvSQ/closedform


Abstract

Recent advances in geometric foundation models and multimodal large language models (MLLMs) are challenging decades of conventional wisdom in robot perception, opening new opportunities to rethink how robots perceive, understand, and remember the world. In this talk, I will first explore how the internal representations learned by geometric foundation models can provide a common foundation for a wide range of tasks, from SLAM and visual-inertial navigation to scene-change detection and robotic manipulation. I will then shift the focus from geometry to memory, asking what information robots should retain from their observations and experiences to support long-term autonomy. I will discuss recent advances in structured and hierarchical memory representations, examining their benefits and limitations compared to MLLM-based full-context approaches. Finally, I will share my perspective on the emergence, impact, and limitations of agentic AI in robotics.

Bio: 

I am an Associate Professor in the Department of Aeronautics and Astronautics at the Massachusetts Institute of Technology, a Principal Investigator in the MIT Laboratory for Information & Decision Systems (LIDS), and an Amazon Scholar.

I am the director of the MIT SPARK Lab, where we work at the cutting edge of robotics and autonomous systems research. My goal is to enable human-level perception and world understanding on mobile robotics platforms (drones, self-driving vehicles, ground robots) operating in the real world. Towards this goal, my work involves a combination of rigorous theory and practical implementations. In particular, my research interests include nonlinear estimation and probabilistic inference, numerical and distributed optimization, machine learning, and geometric computer vision applied to sensing, perception, and decision-making in single and multi-robot systems.

Human-level perception will increase reliability in safety-critical applications of robotics and autonomous vehicles (including self-driving cars and robots for disaster response), and increase efficiency and effectiveness in service robotics and consumer applications (manufacturing, healthcare, domestic robotics, augmented reality).


Please visit https://stanfordasl.github.io/robotics_seminar/ for this quarter’s lineup of speakers. Although we encourage in-person attendance, recordings of talks will be posted also.

  

Previous
Previous
November 13

Igor Gilitschenski (University of Toronto)