AgroGPT : Efficient Agricultural Vision-Language Model with Expert Tuning
Authors:
- 26 February 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5687-5696
Abstract
Significant progress has been made in advancing large multimodal conversational models (LMMs), capitalizing on vast repositories of image-text data available online. Despite this progress, these models often encounter substantial domain gaps, hindering their ability to engage in complex conversations across new domains. Recent efforts have aimed to mitigate this issue, albeit relying on domainspecific image-text data to curate instruction-tuning data. However, many domains, such as agriculture, lack such vision-language data. In this work, we propose an approach to construct instruction-tuning data that harnesses vision-only data for the agriculture domain. We utilize diverse agricultural datasets spanning multiple domains, curate class-specific information, and employ large language models (LLMs) to construct an expert-tuning set, resulting in a 70k AgroInstruct. Subsequently, we expert-tuned and created AgroGPT, an efficient LMM that can hold complex agriculture-related conversations and provide useful insights. We also develop AgroEvals for evaluation and compare AgroGPT's performance with large open and closedsource models. AgroGPT excels at identifying fine-grained agricultural concepts, can act as an agriculture expert, and provides helpful information for multimodal agriculture questions. The code, datasets, and models are available at https://github.com/awaisrauf/agroGPT.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- XrayGPT: Chest Radiographs Summarization using Large Medical Vision-Language Models2024
- Sigmoid Loss for Language Image Pre-Training2023
- Summary of ChatGPT-Related research and perspective towards the future of large language modelsMeta-Radiology, 2023
- Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM2023
- A Vegetable Leaf Disease Identification Model Based on Image-Text Cross-Modal Feature FusionFrontiers in Plant Science, 2022
- VGG-ICNN: A Lightweight CNN model for crop disease identificationMultimedia Tools and Applications, 2022
- BARD: A Structured Technique for Group Elicitation of Bayesian Networks to Support Analytic ReasoningRisk Analysis, 2021
- Plant leaf disease classification using EfficientNet deep learning modelEcological Informatics, 2021
- PlantDoc2020
- Automatic model based dataset generation for fast and accurate crop and weeds detection2017