Probabilistically Correct Language-Based Multi-Robot Planning Using Conformal Prediction
- 21 November 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10 (1) , 160-167
Abstract
This paper addresses task planning problems for language-instructed robot teams. Tasks are expressed in natural language (NL), requiring the robots to apply their skills at various locations and semantic objects. Several recent works have addressed similar planning problems by leveraging pre-trained Large Language Models (LLMs) to design effective multi-robot plans. However, these approaches lack performance guarantees. To address this challenge, we introduce a new distributed LLM-based planner, called S-ATLAS for Safe plAnning for Teams of Language-instructed AgentS, that can achieve user-defined mission success rates. This is accomplished by leveraging conformal prediction (CP), a distribution-free uncertainty quantification tool. CP allows the proposed multi-robot planner to reason about its inherent uncertainty, due to imperfections of LLMs, in a distributed fashion, enabling robots to make local decisions when they are sufficiently confident and seek help otherwise. We show, both theoretically and empirically, that the proposed planner can achieve user-specified task success rates, assuming successful plan execution, while minimizing the average number of help requests. We provide comparative experiments against related works showing that our method is significantly more computational efficient and achieves lower help rates.Keywords
Funding Information
- ARL (DCIST CRA W911NF-17-2-0181)
- NSF (CNS #2231257, CCF #2403758)
This publication has 15 references indexed in Scilit:
- Fast Temporal Logic Mission Planning of Multiple Robots: A Planning Decision Tree ApproachIEEE Robotics and Automation Letters, 2024
- Text2Reaction : Enabling Reactive Task Planning Using Large Language ModelsIEEE Robotics and Automation Letters, 2024
- Collaborative Multi-Object Tracking With Conformal Uncertainty PropagationIEEE Robotics and Automation Letters, 2024
- Robust Validation: Confident Predictions Even When Distributions ShiftJournal of the American Statistical Association, 2024
- GraspGPT: Leveraging Semantic Knowledge From a Large Language Model for Task-Oriented GraspingIEEE Robotics and Automation Letters, 2023
- Safe Planning in Dynamic Environments Using Conformal PredictionIEEE Robotics and Automation Letters, 2023
- ProgPrompt: Generating Situated Robot Task Plans using Large Language Models2023
- Code as Policies: Language Model Programs for Embodied Control2023
- Least Ambiguous Set-Valued Classifiers With Bounded Error LevelsJournal of the American Statistical Association, 2018
- Conditional validity of inductive conformal predictorsMachine Learning, 2013