Ting Yuan

In greenhouse environments, using automated machines for tomato harvesting to reduce labor consumption is a future development trend. Accurate and effective visual recognition is essential to accomplish harvesting tasks. However, most current studies use various models to gain harvesting information in multiple steps, resulting in heavy calculation costs, poor real-time availability, and weak recognition precision. In this study, an improved YOLOv8np-RCW end-to-end model based on YOLOv8n pose is proposed to simultaneously detect tomato bunches, maturity, and keypoints using a decoupled-head structure. The model integrates a ResNet-enhanced RepVGG architecture for a balance of accuracy and speed, employs the CARAFE upsampling algorithm for a larger receptive field with lightweight design, and optimizes the loss function with WIoU loss to enhance bounding box prediction, maturity detection, and keypoint extraction. Experimental results indicate that mAP50 of YOLOv8np-RCW model for bounding box and keypoints is 87.3% and 86.8% respectively, which is 6.2% and 5.5% higher than YOLOv8n pose model. Completing the tasks of bunch detection, maturity assessment, and keypoint localization requires only 9.8 ms. Euclidean distance error is less than 20 pixels in detecting keypoints. Based on this model, a method is proposed to quickly determine the orientation of tomato bunches using geometric cross-product and cross-multiplication calculations from keypoint 2D information, providing guidance for the motion planning of the end-effector. In field experiments, the robot achieved a harvesting success rate of 68%, with an average time of 10.8366 seconds per tomato bunch.

Currently, tomato plant lowering is performed manually, which is both inefficient and costly. The manual process presents challenges in terms of efficiency and cost, creating a need for automated solutions in greenhouse environments. This paper addresses this issue by presenting the design and development of a tomato-plant-lowering robot utilizing machine vision and deep learning techniques. The study includes the design of an end effector optimized for plant-lowering operations based on the physical characteristics of tomato vines and roller hooks; precise positioning of roller hooks achieved through kinematic analysis and a custom dataset; integration of the RepC3 module from RT-DETR with YOLOv5s for enhanced object detection and positioning; and real-time camera feed display through an integrated application. Performance evaluation through experimental tests shows improvements in recognition accuracy, positioning precision, and operational efficiency, although the robot’s success rate in leaf removal needs further enhancement. This research provides a solid foundation for future developments in plant-lowering robots and offers practical insights and technical guidance.

Over the past decade, there have been increasing attempts to integrate robotic harvesting technology into agricultural scenarios to reduce growing labour costs and increase crop yields. In this paper, we demonstrate a prototype harvesting robot for picking watermelons in greenhouses. For robotic harvesting, we design a dedicated end-effector for grasping fruits and shearing pedicels, which mainly consists of a flexible gripper and a cutting device. The improved YOLOv5s–CBAM is employed to locate the watermelon fruits with 89.8% accuracy on the test dataset, while the K-means method is used to further refine the segmentation of the watermelon point cloud in the region of interest. Then, the ellipsoid is fitted with the segmented fruit point cloud to obtain the lowest point of the ellipsoid as the grasping point. A series of tests conducted in a laboratory simulation scenario proved that the overall harvesting success rate was 93.3% with a positioning error of 8.7 mm when the watermelon was unobstructed. The overall harvesting success rate was 85.0% with a positioning error of 14.6 mm when the watermelon was partially obscured by leaves.

In the process of vegetable plug seedling cultivation, packaging, and transportation, there may be missing, unhealthy or injured seedlings in the tray, which results in a missed planting or a low seedling survival rate after automatic transplanting. In this study, a seedling selection system with the function of seedlings identification, week seedlings elimination, and missing seedlings supplement was developed for an automatic transplanter. A plug seedling identification system based on a machine vision was used to detect vegetable plug seedlings based on the area characteristics of plug seedlings, stem leaves and plug bodies. The identification results were transmitted to a programmable logic controller (PLC), which controlled a nozzle to eliminate the unqualified seedlings from the conveyor belt lattice. When the empty conveyor belt lattice reaches the seedling throwing funnel, the rear conveyor belt lattice with the plug seedling is accelerated to ensure the continuity of seedlings supply. The adaptive fuzzy PID control algorithm was used to control the stepper motor of the conveyor belt to realize accurate seedling conveying and a seedling supplement. Using 30 days pepper plug seedlings as experimental seedlings, a comparative field experiment was carried out to evaluate the performance of the seedling selection system. The results showed that when the seedling selection system was turned on and the seedling extracting frequencies were 60, 80, and 100 plants/min, the success rates of plug seedling identification were 98.84%, 98.38%, and 96.99%, and the robust seedling rates were 98.05%, 97.78%, and 95.83%. The robust seedling rates were increased by 15.64%, 16.07%, and 13.89%, respectively, in contrast to turning off the seedling selection system.