Study on the detection of water status of tomato (Solanum lycopersicum L.) by multimodal deep learning
Open Access
- 31 May 2023
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Plant Science
- Vol. 14, 1094142
Abstract
Water plays a very important role in the growth of tomato (Solanum lycopersicum L.), and how to detect the water status of tomato is the key to precise irrigation. The objective of this study is to detect the water status of tomato by fusing RGB, NIR and depth image information through deep learning. Five irrigation levels were set to cultivate tomatoes in different water states, with irrigation amounts of 150%, 125%, 100%, 75%, and 50% of reference evapotranspiration calculated by a modified Penman-Monteith equation, respectively. The water status of tomatoes was divided into five categories: severely irrigated deficit, slightly irrigated deficit, moderately irrigated, slightly over-irrigated, and severely over-irrigated. RGB images, depth images and NIR images of the upper part of the tomato plant were taken as data sets. The data sets were used to train and test the tomato water status detection models built with single-mode and multimodal deep learning networks, respectively. In the single-mode deep learning network, two CNNs, VGG-16 and Resnet-50, were trained on a single RGB image, a depth image, or a NIR image for a total of six cases. In the multimodal deep learning network, two or more of the RGB images, depth images and NIR images were trained with VGG-16 or Resnet-50, respectively, for a total of 20 combinations. Results showed that the accuracy of tomato water status detection based on single-mode deep learning ranged from 88.97% to 93.09%, while the accuracy of tomato water status detection based on multimodal deep learning ranged from 93.09% to 99.18%. The multimodal deep learning significantly outperformed the single-modal deep learning. The tomato water status detection model built using a multimodal deep learning network with ResNet-50 for RGB images and VGG-16 for depth and NIR images was optimal. This study provides a novel method for non-destructive detection of water status of tomato and gives a reference for precise irrigation management.Keywords
This publication has 20 references indexed in Scilit:
- Optimizing irrigation frequency and amount to balance yield, fruit quality and water use efficiency of greenhouse tomatoAgricultural Water Management, 2019
- EmbraceNet: A robust deep learning architecture for multimodal classificationInformation Fusion, 2019
- Fruit and Leaf Sensing for Continuous Detection of Nectarine Water StatusFrontiers in Plant Science, 2019
- Multi-modal deep learning for Fuji apple detection using RGB-D cameras and their radiometric capabilitiesComputers and Electronics in Agriculture, 2019
- RGB-D-Based Object Recognition Using Multimodal Convolutional Neural Networks: A SurveyIEEE Access, 2019
- Comparing RGB-D Sensors for Close Range Outdoor Agricultural PhenotypingSensors, 2018
- Plant temperature-based indices using infrared thermography for detecting water status in sesame under greenhouse conditionsAgricultural Water Management, 2018
- Multimodal Machine Learning: A Survey and TaxonomyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- Suitable indicators using stem diameter variation-derived indices to monitor the water status of greenhouse tomato plantsPLOS ONE, 2017
- Identify Plant Drought Stress by 3D-Based ImageJournal of Integrative Agriculture, 2012