Moteaal Asadi Shirzi

Image segmentation is a fundamental step in computer vision and a cornerstone of robotic perception, serving as the foundation for interpreting data acquired from vision sensors, enabling robots to analyze complex visual environments, identify and localize objects, and support intelligent decision-making and autonomous control. It plays a critical role in applications such as autonomous navigation, robotic manipulation, medical robotics, agricultural robotics, autonomous vehicles, and human–robot interaction. Image segmentation has evolved from classical methods, which relied on handcrafted rules and mathematical models, to deep learning approaches that learn complex visual patterns directly from data. This evolution reflects advances in algorithms, computational power, and the theoretical foundations of mathematics and data science. Modern deep learning methods rely heavily on large, well-annotated datasets to train sophisticated neural networks. Yet, classical techniques remain valuable in certain scenarios, offering faster, reliable results without extensive computational requirements. Understanding the strengths and limitations of both approaches is key to selecting the right method. This paper surveys image segmentation techniques, comparing them in terms of accuracy, computational cost, and processing speed to guide informed method selection.

This article presents a novel robotic system designed to couple seedling stems to wooden stakes using supportive clips. This process, known as stem–stake coupling or clipping, is a labor-intensive task required for millions of seedlings and plants, providing additional support during transportation, growth, and fruiting stages. Our robotic system utilizes computer vision, machine learning, and a clipping mechanism to recognize seedlings, identify coupling points, and apply the clips. The clipping mechanism forms and attaches clips around the seedling stem and wooden stake, eliminating the need for the plastic clips currently used in manual operations. All components of the system are carried by a robotic arm equipped with additional sensors and actuators for precise environmental interactions. Considering the number of seedlings produced annually, the developed robotic system offers significant cost savings by relieving humans from a physically demanding and laborious task. In addition to stem–stake clipping, the robotic system with its integrated machine vision can be used for a broad spectrum of applications in precision agriculture, such as disease detection, pest control, and grafting.

In the field of robotics, the integration of machine vision-commonly referred to as vision-based measurement-empowers robotic systems to transform visual information into digital data, thereby enabling a broad spectrum of innovative applications across diverse sectors, including precision agriculture. This paper presents a new real-time vision-based method designed for the recognition and precise localization of specific points on young plants. Identifying these key points is essential for many tasks intended to be automated using robotic systems. A particular task of an interest is the robotic coupling of the plant’s stem to a wooden stake using plastic clip, aka stem-stake coupling. In this task, a human or a robot attaches the plant to a stake at a specific point along the plant’s stem using a clip to support the plant during transportation or throughout its growth phase. We have developed a new real-time vision-based method to identify the clipping point. This method has been implemented and evaluated using a robotic system and real images of plants commonly cultivated in propagation facilities. The algorithm’s accuracy was assessed and compared with end-to-end deep learning approaches. The results demonstrate the effectiveness of our method and its superior performance over learning algorithms that are purely driven by data from input to output. Our approach enhances autonomous operations in precision agriculture and improves the accuracy of vision-based measurements in robotic systems.

This article introduces a novel mechatronic system for coupling the stems of seedlings and plants to wooden stakes or ropes, a crucial process for supporting them during growth, transportation, and fruiting in plant propagation facilities and greenhouses. The stem–stake coupling device utilizes interconnected mechanisms and an impedance control method to adjust motor torque and speed, shaping metallic wire into clips of various shapes and dimensions, effectively securing plant stems to stakes or ropes. In a robotic system, a claw-shaped arm mechanism, a stereo camera, and real-time vision techniques are integrated into the stem–stake coupling device to identify the optimal coupling point and automate the coupling task. This innovation addresses the labor-intensive task of manual coupling, offering a scalable solution for growers through handheld devices or fully automated robotic systems. In the context of increasing labor shortages and rising costs, the technology offers a sustainable and efficient alternative with significant potential to enhance operational efficiency in greenhouses and propagation facilities.

In this article, we propose a new algorithm to improve plant recognition through the use of feature descriptors. The accurate results from this identification method are essential for enabling autonomous tasks, such as stem-stake coupling, in precision agriculture. The proposed method divides the input seedling color image into subimages within the International Commission on Illumination, for three color axes, L for lightness, A for the green-red component, and B for the blue-yellow component, color space and extracts seven key feature descriptors for each subimage. It then uses feature descriptors to create a matrix, which is employed to train an artificial neural network to determine optimized cutoff values. This network suggests cutoff values for a multilevel threshold segmentation for plant recognition. The method provides robust and real-time adaptive segmentation adaptable to various seedlings, backgrounds, and lighting conditions. By enabling accurate segmentation of the plant, morphological image processing can more effectively eliminate leaves to locate the seedling stem. This methodology automates image analysis in seedling propagation facilities and greenhouses and enables a wide range of precision agricultural tasks.

This paper introduces a new real-time method based on a combination of kernel density estimators and pyramid histogram of oriented gradients for identifying a point of interest along the stem of seedlings suitable for stem–stake coupling, also known as the ‘clipping point’. The recognition of a clipping point is a required step for automating the stem–stake coupling task, also known as the clipping task, using the robotic system under development. At present, the completion of this task depends on the expertise of skilled individuals that perform manual clipping. The robotic stem–stake coupling system is designed to emulate human perception (in vision and cognition) for identifying the clipping points and to replicate human motor skills (in dexterity of manipulation) for attaching the clip to the stem at the identified clipping point. The system is expected to clip various types of vegetables, namely peppers, tomatoes, and cucumbers. Our proposed methodology will serve as a framework for automatic analysis and the understanding of the images of seedlings for identifying a suitable clipping point. The proposed algorithm is evaluated using real-world image data from propagation facilities and greenhouses, and the results are verified by expert farmers indicating satisfactory performance. The precise outcomes obtained through this identification method facilitate the execution of other autonomous functions essential in precision agriculture and horticulture.