Mini-CarbonGPT: A Resource-Efficient Knowledge-Augmented Language Model for Industrial Decarbonization Knowledge Work
Authors:
- Jianhua MaJianhua Ma
- Yongzhang ZhouYongzhang Zhou
- Xian LiuXLXian Liu
- Lu-Hao HeLHLu-Hao He
- Yijun ZhengYZYijun Zheng
Preprint
- 24 March 2026
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
Industrial decarbonization involves knowledge-intensive tasks that require engineers and practitioners to access, interpret, and integrate heterogeneous information from technical literature, policy documents, standards, and domain reports. General-purpose large language models often struggle in this setting because of limited domain alignment and weak factual grounding. This study presents Mini-CarbonGPT, a resource-efficient domain language model for industrial decarbonization knowledge work. The system combines parameter-efficient fine-tuning using Low-Rank Adaptation (LoRA) with retrieval-augmented generation (RAG) to couple domain adaptation with evidence-grounded response generation under limited computational resources. A multi-source knowledge pipeline was constructed using 50,000 general instructions, 5,382 domain-specific instructions, 6,096 research papers, and approximately 60,000 curated encyclopedia entries. To evaluate performance in engineering-oriented knowledge tasks, we developed a benchmark containing 700 objective questions and 249 open-ended questions across five carbon-neutrality subdomains. Results show that Mini-CarbonGPT achieved an average objective accuracy of 80.57%, outperforming the baseline model and showing robust performance in factual accuracy, terminology coverage, and response completeness. The proposed framework demonstrates how lightweight adaptation and external knowledge retrieval can be integrated to support industrial knowledge services in complex engineering domains. The study provides a transferable approach for building domain language models for industrial applications where knowledge reliability, deployability, and reproducibility are critical.Keywords
This publication has 19 references indexed in Scilit:
- A Survey on RAG with LLMsProcedia Computer Science, 2024
- Large language models (LLMs): survey, technical frameworks, and future challengesArtificial Intelligence Review, 2024
- BB-GeoGPT: A framework for learning a large language model for geographic information scienceInformation Processing & Management, 2024
- Large language models in law: A surveyAI Open, 2024
- Biophysical and economic constraints on China’s natural climate solutionsNature Climate Change, 2022
- Terrestrial carbon sinks in China and around the world and their contribution to carbon neutralityScience China Life Sciences, 2022
- Assessment of energy storage technologies: A reviewEnergy Conversion and Management, 2020
- The effect of emission trading policy on carbon emission reduction: Evidence from an integrated study of pilot regions in ChinaJournal of Cleaner Production, 2020
- Potential for large-scale CO2 removal via enhanced rock weathering with croplandsNature, 2020
- Ambient weathering of magnesium oxide for CO2 removal from airNature Communications, 2020