Wednesday, June 26, 2024, 1pm

Large pre-trained models and internet data sources are key to general and efficient robot task learning. However, learning contact-rich behaviors, semantic task constraints, and robust task planning from internet data sources remains an open challenge. This proposal seeks to make progress towards a general robot task learning system leveraging pre-trained models and internet data. We limit our study to tasks in the cooking and food preparation domain. This proposal discusses our work in two key areas: skill learning and task planning. To improve skill learning, we present an approach for selecting between basic robot behaviors to accomplish cooking skills. We then propose further work to learn these basic robot behaviors and reduce the need for real-world execution when performing behavior selection. To improve task planning we conduct a large-scale study on Large Language Models as task planners. We identify key deficiencies and propose a new planning framework to address them. Finally, we propose a method for learning task ordering constraints from human video based on our prior constraint-learning work. Ultimately, we seek to produce a robot task-learning system capable of quickly acquiring new tasks in a home environment.Thesis Committee:
Christopher Atkeson (Chair)
David Held
Oliver Kroemer
Ruta Desai (Meta FAIR)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Newell-Simon 3305 and Zoom
Speaker's Name: MRINAL VERGHESE
Speaker Website: mrinal.verghese.org
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Accelerating Robot Task Learning with Large Pretrained Models and Internet Data
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…