YOLO-MSRF: A Multimodal Segmentation and Refinement Framework for Tomato Fruit Detection and Segmentation with Count and Size Estimation Under Complex Illumination
Top Cited Papers
Open Access
- 22 January 2026
- journal article
- research article
- Published by MDPI AG in Agriculture
- Vol. 16 (2) , 277
Abstract
Segmentation of tomato fruits under complex lighting conditions remains technically challenging, especially in low illumination or overexposure, where RGB-only methods often suffer from blurred boundaries and missed small or occluded instances, and simple multimodal fusion cannot fully exploit complementary cues. To address these gaps, we propose YOLO-MSRF, a lightweight RGB–NIR multimodal segmentation and refinement framework for robust tomato perception in facility agriculture. Firstly, we propose a dual-branch multimodal backbone, introduce Cross-Modality Difference Complement Fusion (C-MDCF) for difference-based complementary RGB–NIR fusion, and design C2f-DCB to reduce computation while strengthening feature extraction. Furthermore, we develop a cross-scale attention fusion network and introduce the proposed MS-CPAM to jointly model multi-scale channel and position cues, strengthening fine-grained detail representation and spatial context aggregation for small and occluded tomatoes. Finally, we design the Multi-Scale Fusion and Semantic Refinement Network, MSF-SRNet, which combines the Scale-Concatenate Fusion Module (Scale-Concat) fusion with SDI-based cross-layer detail injection to progressively align and refine multi-scale features, improving representation quality and segmentation accuracy. Extensive experiments show that YOLO-MSRF achieves substantial gains under weak and low-light conditions, where RGB-only models are most prone to boundary degradation and missed instances, and it still delivers consistent improvements on the mixed four-light validation set, increasing by 2.3 points, by 2.4 points, and mIoU by 3.60 points while maintaining real-time inference at 105.07 FPS. The proposed system further supports counting, size estimation, and maturity analysis of harvestable tomatoes, and can be integrated with depth sensing and yield estimation to enable real-time yield prediction in practical greenhouse operations.
Keywords
Funding Information
- National Key Research and Development Program of China (2022YFD2000500)
- China Postdoctoral Science Foundation funded project (2025MD784115)
- Heilongjiang Postdoctoral Foundation (LBH-Z24192)
- Heilongjiang Graduate Excellence Funding Program (LJYXL2024-023)
- Postdoctoral Innovation Program of Shandong Province (SDCX-ZG-202503172)
This publication has 29 references indexed in Scilit:
- A lightweight method for apple disease segmentation using multimodal transformer and sensor fusionComputers and Electronics in Agriculture, 2025
- Automatic segmentation and hyperspectral information extraction of Camellia oleifera fruits based on CA-TransUNet++ under complex natural environmentComputers and Electronics in Agriculture, 2025
- Development of an autonomous navigation system for orchard spraying robots integrating a thermal camera and LiDAR using a deep learning algorithm under low- and no-light conditionsComputers and Electronics in Agriculture, 2025
- MLP-based multimodal tomato detection in complex scenarios: Insights from task-specific analysis of feature fusion architecturesComputers and Electronics in Agriculture, 2024
- A lightweight improved YOLOv5s model and its deployment for detecting pitaya fruits in daytime and nighttime light-supplement environmentsComputers and Electronics in Agriculture, 2024
- Real-time statistical algorithm for cherry tomatoes with different ripeness based on depth information mappingComputers and Electronics in Agriculture, 2024
- Multi-view gripper internal sensing for the regression of strawberry ripeness using a mini-convolutional neural network for robotic harvestingComputers and Electronics in Agriculture, 2023
- DSW-YOLO: A detection method for ground-planted strawberry fruits under different occlusion levelsComputers and Electronics in Agriculture, 2023
- YOLACTFusion: An instance segmentation method for RGB-NIR multimodal image fusion based on an attention mechanismComputers and Electronics in Agriculture, 2023
- Picking point recognition for ripe tomatoes using semantic segmentation and morphological processingComputers and Electronics in Agriculture, 2023