Green Fruit‐Stem Pairing and Clustering for Machine Vision System in Robotic Thinning of Apples
Authors:
Open Access
- 5 November 2024
- journal article
- research article
- Published by Wiley in Journal of Field Robotics
- Vol. 42 (4) , 1463-1490
Abstract
Apples are one of the most highly‐valued specialty crops in the United States. Recent labor shortages have made crop production difficult for fruit growers, including the task of green fruit thinning. Current methods including hand, chemical, and mechanical thinning impose tradeoffs between selectivity, cost, tree damage, and speed. A robotic green fruit thinning system could potentially selectively thin fruit in a quick, cost‐effective, and non‐damaging manner. The machine vision system would be a critical component for robotic thinning, and would not only need to perform green fruit detection/segmentation, but also fruit‐stem pairing and clustering to facilitate proper decision‐making for thinning. A neural network‐based fruit and stem pairing algorithm was devised and evaluated; an LSTM‐based clustering algorithm was devised and compared to the density‐based clustering algorithm, OPTICS. The algorithms were evaluated on an image data set consisting of GoldRush, Fuji, and Golden Delicious cultivars at the green fruit stage, with evaluations on overall performance, cultivar‐wise performance, cluster size‐specific performance, and feature importance. For fruit and stem pairing, the neural network‐based pairing algorithm achieved an AP of 81.4% on all fruits and stems, and that reached 90.6% when only fruits and stems with labeled angles were considered. For green fruit clustering, the LSTM‐based clustering achieved a clustering success rate of 68.4%, whereas the OPTICS algorithm obtained 50.9%. The algorithms will be further implemented in a pipeline of a future green fruit thinning vision system, as well as integrate the use of point clouds and other 3D orchard information to improve pairing and clustering performance.Keywords
This publication has 23 references indexed in Scilit:
- Design, integration, and field evaluation of a robotic blossom thinning system for tree fruit cropsJournal of Field Robotics, 2024
- Dynamic visual servo control methods for continuous operation of a fruit harvesting robot working throughout an orchardComputers and Electronics in Agriculture, 2024
- Green fruit segmentation and orientation estimation for robotic green fruit thinning of applesComputers and Electronics in Agriculture, 2023
- Channel pruned YOLO V5s-based deep learning approach for rapid and accurate apple fruitlet detection before fruit thinningBiosystems Engineering, 2021
- Multiscale Digital Porous Rock Reconstruction Using Template MatchingWater Resources Research, 2019
- A weighted multivariate spatial clustering model to determine irrigation management zonesComputers and Electronics in Agriculture, 2019
- Deep Orange: Mask R-CNN based Orange Detection and SegmentationIFAC-PapersOnLine, 2019
- Apple Crop Load Management with Special Focus on Early Thinning Strategies2018
- Multi-template matching algorithm for cucumber recognition in natural environmentComputers and Electronics in Agriculture, 2016
- Disease detection of Cercospora Leaf Spot in sugar beet by robust template matchingComputers and Electronics in Agriculture, 2014