Towards Trustworthy Autonomy: Guardrails to detect, avoid, and learn from failures / Learning Dexerity from Humans
Speakers: Rohan Sinha and Changhao Wang
Location: 353 Jane Stanford Way, Gates B03, Stanford, CA 94305
Time: Friday May 15th, 3:00 - 4:00 PM
Title: Towards Trustworthy Autonomy: Guardrails to detect, avoid, and learn from failures
Speaker #1: Rohan Sinha
Time: Friday May 15th, 3:00-3:30PM
Abstract: While learning algorithms have powered tremendous progress in autonomy in recent years, failures persist when deployed systems inevitably encounter rare scenarios dissimilar from training data—so-called out-of-distribution (OOD) scenarios. We will present a holistic perspective on the use of guardrails to mitigate the negative consequences of such OOD scenarios on the safety and reliability of learning-based robotic systems, organized around three questions: how to detect when a system is unreliable, how to enact safety interventions when it degrades, and how to improve it through principled use of deployment experience. First, to detect impending failures, we show how the commonsense reasoning capabilities of foundation models, e.g., large language models (LLMs), make them attractive as runtime monitors for intricate, context-driven hazards. Second, we introduce a 'Thinking, Fast & Slow' reasoning framework that tightly integrates LLMs into the robot's control loop, thereby enabling their reactive, real-time use to avert closed-loop failures caused by system-level deficiencies in the robot's semantic reasoning. Finally, beyond averting immediate failures during deployment, we aim to address observed failure modes at their source by training improved policies. Towards this goal, we trace policy successes and failures back to individual training samples, enabling data level diagnosis and data curation that leads to improved policy performance and robustness. Taken together, these contributions advance towards the goal of trustworthy autonomous systems: systems that know what they can and cannot do, recover safely when conditions become too challenging, and can be interpretably analyzed and improved when failure modes emerge.
Bio: Rohan Sinha is a PhD candidate in the Autonomous Systems Lab at Stanford University, advised by Prof. Marco Pavone. He also spent time at Google DeepMind Robotics as a student researcher. His research focuses on developing methodologies that improve the reliability of ML-enabled robotic systems, particularly when these systems encounter out-of-distribution conditions with respect to their training data. His work on this topic was recognized with the outstanding paper award at the 2024 Robotics: Science and Systems (RSS) conference and a best paper award at the 2025 RSS RoboEval workshop. Previously, he received bachelor's degrees in computer science and mechanical engineering from the University of California, Berkeley with honors and a distinction in general scholarship. As an undergraduate, Rohan worked with Prof. Francesco Borrelli in the Model Predictive Control Lab and Prof. Benjamin Recht in the Berkeley Artificial Intelligence Research Lab.
Title: Learning Dexerity from Humans
Speaker #2: Changhao Wang
Time: Friday May 15th, 3:30-4:00PM
Abstract: Humans are naturally dexterous, and human behavior provides a rich source of data for robot learning. Even for a simple task such as rotating a ball, there are many possible strategies: using finger motions, rotating the wrist, or performing repeated regrasps with a parallel gripper. The human-like strategy is often the most desirable not because it is the only solution, but because humans provide one of the most abundant and expressive sources of demonstrations for learning general-purpose manipulation. To effectively leverage human data, robots must develop dexterity that can align with human demonstrations while accounting for their own embodiment. In this talk, I will discuss two directions toward this goal: 1) learning dexterous teleoperation from humans, and 2) learning dexterous skills from human videos. I will show how these approaches can bridge human motion and robotic control, enabling robots to acquire more human-like dexterous skills.
Bio: Changhao is a postdoctoral researcher at Stanford University working with Prof. Shuran Song. Previously, he worked at Meta Fundamental AI Research (FAIR) in Menlo Park. His research centers on enabling dexterous manipulation through learning from human demonstrations and interactions.
Please visit https://stanfordasl.github.io/robotics_seminar/ for this quarter’s lineup of speakers. Although we encourage live in-person attendance, recordings of talks will be posted also.