DiMSam: Diffusion Models as Samplers for Task and Motion Planning under Partial Observability
Authors:
- Xiaolin FangXFXiaolin Fang
- Caelan Reed GarrettCGCaelan Reed Garrett
- Clemens EppnerCEClemens Eppner
- Tomás Lozano-PérezTLTomás Lozano-Pérez
- Leslie Pack KaelblingLKLeslie Pack Kaelbling
- Dieter FoxDFDieter Fox
Preprint
Abstract
Task and Motion Planning (TAMP) approaches are effective at planning long-horizon autonomous robot manipulation. However, because they require a planning model, it can be difficult to apply them to domains where the environment and its dynamics are not fully known. We propose to overcome these limitations by leveraging deep generative modeling, specifically diffusion models, to learn constraints and samplers that capture these difficult-to-engineer aspects of the planning model. These learned samplers are composed and combined within a TAMP solver in order to find action parameter values jointly that satisfy the constraints along a plan. To tractably make predictions for unseen objects in the environment, we define these samplers on low-dimensional learned latent embeddings of changing object state. We evaluate our approach in an articulated object manipulation domain and show how the combination of classical TAMP, generative learning, and latent embeddings enables long-horizon constraint-based reasoning.Keywords
All Related Versions
This publication has 5 references indexed in Scilit:
- NeRP: Neural Rearrangement Planning for Unknown Objects2021
- Integrated Task and Motion PlanningAnnual Review of Control, Robotics, and Autonomous Systems, 2021
- Learning compositional models of robot skills for task and motion planningThe International Journal of Robotics Research, 2021
- PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive PlanningProceedings of the International Conference on Automated Planning and Scheduling, 2020
- Online Replanning in Belief Space for Partially Observable Task and Motion Problems2020