Publication
Our paper, titled LLM-GROP: Visually Grounded Robot Task and Motion Planning with Large Language Models, has been accepted for publication in The International Journal of Robotics Research (IJRR) 2025.
This work represents a significant advancement in combining large language models with robot task and motion planning, enabling robots to understand and execute complex tasks through natural language instructions while being grounded in visual perception.
Abstract
We present LLM-GROP, a novel framework that leverages large language models for visually grounded robot task and motion planning. Our approach enables robots to interpret natural language instructions and translate them into executable plans while maintaining visual grounding in the environment.
Key Contributions
- Integration of LLMs with robot task and motion planning
- Visually grounded understanding of complex instructions
- Demonstration of effectiveness across various robotic tasks
Authors: Xiaohan Zhang, Yan Ding, Yohei Hayamizu, Zainab Altaweel, Yifeng Zhu, Yuke Zhu, Peter Stone, Chris Paxton, Shiqi Zhang (*Equal contribution)
Venue: The International Journal of Robotics Research (IJRR) 2025