A large language model for multimodal identification of crop diseases and pests
Authors:
Top Cited Papers
Open Access
- 1 July 2025
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 15 (1) , 1-15
Abstract
Pests and diseases significantly impact the growth and development of crops. When attempting to precisely identify disease characteristics in crop images through dialogue, existing multimodal models face numerous challenges, often leading to misinterpretation and incorrect feedback regarding disease information. This paper proposed a large language model for multimodal identification of crop diseases and pests, which can be called LLMI-CDP. It builds up on the VisualGLM model and introduces improvements to achieve precise identification of agricultural crop disease and pest images, along with providing professional recommendations for relevant preventive measures. The use of Low-Rank Adaptation (LoRA) technology, which adjusts the weights of pre-trained models, achieves significant performance improvements with a minimal increase in parameters. This ensures the precise capture and efficient identification of crop pest and disease characteristics, greatly enhancing the model’s application flexibility and accuracy in the field of pest and disease recognition. Simultaneously, the model incorporates the Q-Former framework for effective modal alignment between language models and image features. Through this approach, the LLMI-CDP model is able to more deeply understand and process the complex relationships between language and visual information, further enhancing its performance in multimodal recognition tasks. Experiments are carried out in the homemade datasets, The results demonstrate that the LLMI-CDP model surpasses five leading multimodal large language models in relevant evaluation metrics, confirming its outstanding performance in Chinese multimodal dialogues related to agriculture.Keywords
Funding Information
- Major Project of Scientific and Technological Innovation 2030 (2021ZD0113603)
- National Natural Science Foundation of China (62276028)
- Major Research Plan of the National Natural Science Foundation of China (92267110)
This publication has 18 references indexed in Scilit:
- Deep learning-based classification of alfalfa varieties: A comparative study using a custom leaf image datasetMethodsX, 2024
- Advancing automatic plant classification system in Saudi Arabia: introducing a novel dataset and ensemble deep learning approachModeling Earth Systems and Environment, 2024
- Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A SurveyACM Computing Surveys, 2023
- ChatGPT for shaping the future of dentistry: the potential of multi-modal large language modelInternational Journal of Oral Science, 2023
- Fatigue database of complex metallic alloysScientific Data, 2023
- EVA: Exploring the Limits of Masked Visual Representation Learning at Scale2023
- Holistic Evaluation of Language ModelsAnnals of the New York Academy of Sciences, 2023
- Parameter-efficient fine-tuning of large-scale pre-trained language modelsNature Machine Intelligence, 2023
- A Survey on Smart Agriculture: Development Modes, Technologies, and Security and Privacy ChallengesIEEE/CAA Journal of Automatica Sinica, 2020
- 12-in-1: Multi-Task Vision and Language Representation Learning2020