Strawberry picking point localization ripeness and weight estimation
Authors:
- 23 May 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2295-2302
Abstract
Labour shortage, difficulties in labour management, the digitalization of fruit production pipeline to reduce the fruit production costs have made robotic systems for selective harvesting of strawberries an important industry and academic research. One of the important components of such technologies yet to be developed is fruit picking perception. For picking strawberries, a robot needs to infer the location of picking points from the images of strawberries. Moreover, the size and weight of strawberries to be picked can help the robot to place the picked strawberries in proper punnets directly to be delivered to customers in supermarkets. This can save significant time and packing costs in packhouses. Geometry-based approaches are the most common approach to determine the picking point but they suffer from inaccuracies due to noise, occlusion, and varying shape and orientation of the berries. In contrast, we present two novel datasets of strawberries annotated with picking points, key-points (such as the shoulder points, the contact point between the calyx and flesh, and the point on the flesh farthest from the calyx), and the weight and size of the berries. We performed experiments with Detectron-2, which is an extended version of Mask-RCNN with key-points detection capability. The results show that the key-points detection approach works well for picking and grasping point localization. The second dataset also presents the dimensions and weight of strawberries. Our novel baseline model for weight estimation outperforms many state-of-the-art deep networks. The datasets and annotations are available at https://github.com/imanlab/strawberry-pp-w-r-dataset.Keywords
This publication has 31 references indexed in Scilit:
- Automatic apple recognition based on the fusion of color and 3D feature for robotic fruit pickingComputers and Electronics in Agriculture, 2017
- Real-time segmentation of strawberry flesh and calyx from images of singulated strawberries during postharvest processingComputers and Electronics in Agriculture, 2017
- Towards Automated Strawberry Harvesting: Identifying the Picking Point2017
- DeepFruits: A Fruit Detection System Using Deep Neural NetworksSensors, 2016
- Deep Residual Learning for Image Recognition2016
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Facial Landmark Detection by Deep Multi-task Learning2014
- Evaluation of a strawberry-harvesting robot in a field testBiosystems Engineering, 2010
- Machine Vision Algorithm for Robots to Harvest Strawberries in Tabletop Culture GreenhousesEngineering in Agriculture, Environment and Food, 2009
- Random decision forests2002