Developing Modular Grasping and Manipulation Pipeline Infrastructure to Streamline Performance Benchmarking
Preprint
Abstract
The robot manipulation ecosystem currently faces issues with integrating open-source components and reproducing results. This limits the ability of the community to benchmark and compare the performance of different solutions to one another in an effective manner, instead relying on largely holistic evaluations. As part of the COMPARE Ecosystem project, we are developing modular grasping and manipulation pipeline infrastructure in order to streamline performance benchmarking. The infrastructure will be used towards the establishment of standards and guidelines for modularity and improved open-source development and benchmarking. This paper provides a high-level overview of the architecture of the pipeline infrastructure, experiments conducted to exercise it during development, and future work to expand its modularity.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Run to the Source: The Effective Reproducibility of Robotics Code RepositoriesIEEE Robotics & Automation Magazine, 2023
- The Cluttered Environment Picking Benchmark (CEPB) for Advanced Warehouse Automation: Evaluating the Perception, Planning, Control, and Grasping of Manipulation SystemsIEEE Robotics & Automation Magazine, 2023
- GRASPA-fying the Panda: Easily Deployable, Fully Reproducible Benchmarking of Grasp Planning AlgorithmsIEEE Robotics & Automation Magazine, 2023
- Benchmarking Protocol for Grasp Planning AlgorithmsIEEE Robotics and Automation Letters, 2019
- Learning ambidextrous robot grasping policiesScience Robotics, 2019
- Try to Start It! The Challenge of Reusing Code in Robotics ResearchIEEE Robotics and Automation Letters, 2018
- Grasp Pose Detection in Point CloudsThe International Journal of Robotics Research, 2017
- Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics2017
- The Open Motion Planning LibraryIEEE Robotics & Automation Magazine, 2012