Growth Stages Discrimination of Multi-Cultivar Navel Oranges Using the Fusion of Near-Infrared Hyperspectral Imaging and Machine Vision with Deep Learning

Authors:
Open Access
Publisher website
Google Scholar
Abstract
To noninvasively and precisely discriminate among the growth stages of multiple cultivars of navel oranges simultaneously, the fusion of the technologies of near-infrared (NIR) hyperspectral imaging (HSI) combined with machine vision (MV) and deep learning is employed. NIR reflectance spectra and hyperspectral and RGB images for 740 Gannan navel oranges of five cultivars are collected. Based on preprocessed spectra, optimally selected hyperspectral images, and registered RGB images, a dual-branch multi-modal feature fusion convolutional neural network (CNN) model is established. In this model, a spectral branch is designed to extract spectral features reflecting internal compositional variations, while the image branch is utilized to extract external color and texture features from the integration of hyperspectral and RGB images. Finally, growth stages are determined via the fusion of features. To validate the availability of the proposed method, various machine-learning and deep-learning models are compared for single-modal and multi-modal data. The results demonstrate that multi-modal feature fusion of HSI and MV combined with the constructed dual-branch CNN deep-learning model yields excellent growth stage discrimination in navel oranges, achieving an accuracy, recall rate, precision, F1 score, and kappa coefficient on the testing set are 95.95%, 96.66%, 96.76%, 96.69%, and 0.9481, respectively, providing a prominent way to precisely monitor the growth stages of fruits.
Funding Information
  • National Natural Science Foundation of China (62165006, 2024-069, 20224ACB202004, 20242BAB20065, 20243BBI91011, GJJ2401207, 2019-NCZDSY-008, 2022BSQD02)
  • Jiangxi Province Ganpo Juncai Support Plan-High level and High skilled Leading Talent Training Project (62165006, 2024-069, 20224ACB202004, 20242BAB20065, 20243BBI91011, GJJ2401207, 2019-NCZDSY-008, 2022BSQD02)
  • Project of Natural and Science Foundation of Jiangxi Province (62165006, 2024-069, 20224ACB202004, 20242BAB20065, 20243BBI91011, GJJ2401207, 2019-NCZDSY-008, 2022BSQD02)
  • Key Research and Development Program Project of Jiangxi Province (62165006, 2024-069, 20224ACB202004, 20242BAB20065, 20243BBI91011, GJJ2401207, 2019-NCZDSY-008, 2022BSQD02)
  • Science and Technology Project Foundation of the Education Department of Jiangxi Province (62165006, 2024-069, 20224ACB202004, 20242BAB20065, 20243BBI91011, GJJ2401207, 2019-NCZDSY-008, 2022BSQD02)
  • Nanchang City Key Laboratory of Optic-electronic Detection and Information Processing (62165006, 2024-069, 20224ACB202004, 20242BAB20065, 20243BBI91011, GJJ2401207, 2019-NCZDSY-008, 2022BSQD02)
  • Doctoral Startup Fund Project of Jiangxi Science and Technology Normal University (62165006, 2024-069, 20224ACB202004, 20242BAB20065, 20243BBI91011, GJJ2401207, 2019-NCZDSY-008, 2022BSQD02)

This publication has 54 references indexed in Scilit: