 Event Information |
|
MRC Seminar: Vision-Language Maps: Planning over Spatially Grounded Semantic Information Vision-Language Maps: Planning over Spatially Grounded Semantic Information Bernadette Bucher Abstract Recent advances in the fields of computer vision and natural language have demonstrated impressive natural language grounded understanding of visual information. In robotics, LLMs have achieved some success in developing action plans from this language-based understanding of scenes. However, directly leveraging text-based planners in a 3D world creates an information bottleneck in which much visual and geometric information is lost before planning. In this presentation, I will demonstrate how we plan with a modified version of frontier-based exploration in a hierarchical language-grounded value map and occupancy map to enable efficient navigation to semantic objects in novel environments. This approach achieves state-of-the-art results as measured by success weighted by path length (SPL) in the Object Goal Navigation task on the Gibson, Habitat-Matterport 3D (HM3D), and Matterport 3D (MP3D) datasets in the Habitat simulator. We also deploy our approach on Spot and demonstrate its capability to efficiently navigate to target objects within an office building in the real world, without any prior knowledge of the environment. I will then present some longer term research plans and progress in my team at the Boston Dynamics AI Institute toward enabling dexterous mobile manipulation from generalized instruction following. Biography Bernadette Bucher is a research scientist at the Boston Dynamics AI Institute. She is also an incoming Assistant Professor in the Robotics Department at University of Michigan starting Fall 2024. Previously, Bernadette received her PhD in computer science in the GRASP lab at University of Pennsylvania co-advised by Dr. Kostas Daniilidis and Dr. Nikolai Matni. During her PhD, Bernadette interned in the Seattle Robotics Lab at NVIDIA Research. Prior to starting her PhD, Bernadette was a Senior Software Engineer at Lockheed Martin Corporation where she worked from 2014 to 2019. She received an M.A. in Mathematics, M.A. in Economics, and B.S. in Mathematics and Economics from The University of Alabama in 2014. Host: Dinesh Manocha
|
