Optimizing and Fine-tuning Large Language Model for Urban Renewal
Preprint
Abstract
This study aims to innovatively explore adaptive applications of large language models (LLM) in urban renewal. It also aims to improve its performance and text generation quality for knowledge question-answering (QA) tasks. Based on the ChatGLM, we automatically generate QA datasets using urban renewal scientific literature corpora in a self-instruct manner and then conduct joint fine-tuning training on the model using the Prefix and LoRA fine-tuning methods to create an LLM for urban renewal. By guiding the LLM to automatically generate QA data based on prompt words and given text, it is possible to quickly obtain datasets in the urban renewal field and provide data support for the fine-tuning training of LLMs. The experimental results show that the joint fine-tuning training method proposed in this study can significantly improve the performance of LLM on the QA tasks. Compared with LoRA fine-tuning, the method improves the Bleu and Rouge metrics on the test by about 5%; compared with the model before fine-tuning, the method improves the Bleu and Rouge metrics by about 15%-20%. This study demonstrates the effectiveness and superiority of the joint fine-tuning method using Prefix and LoRA for ChatGLM in the urban renewal knowledge QA tasks. It provides a new approach for fine-tuning LLMs on urban renewal-related tasks.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Mapping essential urban land use categories (EULUC) using geospatial big data: Progress, challenges, and opportunitiesBig Earth Data, 2021
- Public Services Equalization in Urbanizing China: Indicators, Spatiotemporal Dynamics and Implications on Regional Economic DisparitiesSocial Indicators Research, 2020
- From diversity to justice – Unraveling pluralistic rationalities in urban designCities, 2019
- Heritage, urban regeneration and place-makingJournal of Urban Design, 2017
- Global land cover mapping at 30 m resolution: A POK-based operational approachISPRS Journal of Photogrammetry and Remote Sensing, 2015
- Integrating ensemble-urban cellular automata model with an uncertainty map to improve the performance of a single modelInternational Journal of Geographical Information Science, 2015
- Finer resolution observation and monitoring of global land cover: first mapping results with Landsat TM and ETM+ dataInternational Journal of Remote Sensing, 2012
- Remote sensing of environmental change over China: A reviewChinese Science Bulletin, 2012
- China’s urban expansion from 1990 to 2010 determined with satellite remote sensingChinese Science Bulletin, 2012
- The dimensions of global urban expansion: Estimates and projections for all countries, 2000–2050Progress in Planning, 2011