Shumin Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Shumin Wang has a total of 8 co-authors. Here are the top co-authors by number of shared publications:
- Dong Huang
Key Laboratory of Advanced Manufacturing Technology, Ministry of Education, Guizhou University, Guiyang, China
1 shared publication - Maolin Zhang
Xihua University
1 shared publication - Songlin Chen
School of Computer and Software Engineering Xihua University 610065 Chengdu Sichuan China
1 shared publication - Weicheng Wang
Xihua University
1 shared publication - Xianyong Li
School of Computer and Software Engineering, Xihua University, Chengdu, China
1 shared publication - Xiaoliang Chen
School of Computer and Software Engineering, Xihua University, Chengdu 610039, China
1 shared publication
Latest Publications
- Open Access
- 18 October 2023
- Published by Springer Nature in Research Square
A Multimodal Large Language Model for Forestry in pest and disease recognition
In the context of generative multimodal dialogue, the amalgamation of image and text forms the fundamental basis for a wide array of visual and language tasks. Despite the proliferation of large language models (LLMs), research on training LLMs for multimodal reasoning has remained relatively scarce. The progression of existing multimodal models has been propelled by using expansive image-text datasets in broader domains. Nevertheless, these models need refinement when comprehending and engaging in dialogue concerning Chinese forestry biological images. This paper proposes an efficient and adaptive multimodal large language model for forestry (MLLM-For) in pest and disease recognition. The central concept involves harnessing our meticulously constructed, expansive Chinese forestry biological image caption dataset. We utilize open-ended instruction-following data to master the intricacies of open-ended dialogue semantics, facilitating the comprehensive assimilation of knowledge within forestry biology. In particular, the Querying Transformer (Q-Former) module is a conduit linking pre-trained visual and language models. Using image-text pairs, the China Open Source Chat Language Model (ChatGLM) can now align vocabulary specific to forestry pests and diseases. By aligning image features with the pre-trained language model, we undertake concurrent Low-Rank Adaptive (LoRA) fine-tuning of the ChatGLM model. Furthermore, we engage in high-caliber instruction data fine-tuning to 1 tailor it to the proposed multimodal task. The proposed MLLM-For model show-cases impressive proficiency in comprehending and engaging in dialogue regarding Chinese forestry pests and disease images. It also displays exceptional capabilities in multimodal dialogue and is adept at adhering to open-ended directives for retrieving information about forestry pests and diseases. Empirical findings affirm the competitive performance of the MLLM-For model within the realm of Chinese multimodal dialogue, surpassing five leading multimodal large language models.
- 18 October 2023
- Published by Springer Nature in Research Square
Open AccessA Multimodal Large Language Model for Forestry in pest and disease recognition
In the context of generative multimodal dialogue, the amalgamation of image and text forms the fundamental basis for a wide array of visual and language tasks. Despite the proliferation of large language models (LLMs), research on training LLMs for multimodal reasoning has remained relatively scarce. The progression of existing multimodal models has been propelled by using expansive image-text datasets in broader domains. Nevertheless, these models need refinement when comprehending and engaging in dialogue concerning Chinese forestry biological images. This paper proposes an efficient and adaptive multimodal large language model for forestry (MLLM-For) in pest and disease recognition. The central concept involves harnessing our meticulously constructed, expansive Chinese forestry biological image caption dataset. We utilize open-ended instruction-following data to master the intricacies of open-ended dialogue semantics, facilitating the comprehensive assimilation of knowledge within forestry biology. In particular, the Querying Transformer (Q-Former) module is a conduit linking pre-trained visual and language models. Using image-text pairs, the China Open Source Chat Language Model (ChatGLM) can now align vocabulary specific to forestry pests and diseases. By aligning image features with the pre-trained language model, we undertake concurrent Low-Rank Adaptive (LoRA) fine-tuning of the ChatGLM model. Furthermore, we engage in high-caliber instruction data fine-tuning to 1 tailor it to the proposed multimodal task. The proposed MLLM-For model show-cases impressive proficiency in comprehending and engaging in dialogue regarding Chinese forestry pests and disease images. It also displays exceptional capabilities in multimodal dialogue and is adept at adhering to open-ended directives for retrieving information about forestry pests and diseases. Empirical findings affirm the competitive performance of the MLLM-For model within the realm of Chinese multimodal dialogue, surpassing five leading multimodal large language models.