ChatClimate: Grounding Conversational AI in Climate Science
Authors:
Preprint
- 30 May 2023
- preprint
- Published by Springer Nature in Research Square
Abstract
Large Language Models (LLMs) have made significant progress in recent years, achieving remarkable results in question-answering tasks (QA). However, they still face two major challenges at inference time: hallucination and outdated information. These challenges take center stage in critical domains like climate change, where obtaining accurate and up-to-date information from reliable sources in a timely fashion is essential. To overcome these barriers, one potential solution is to provide LLMs with access to external, scientifically accurate, and robust sources (long-term memory). This access assists LLMs in continuously updating their knowledge and preventing the propagation of inaccurate, incorrect, or outdated information. In this study, we enhanced GPT-4 by providing it access to the Sixth Assessment Report of the Intergovernmental (IPCC-AR6), the most comprehensive, up-to-date, and reliable source in this domain. We present our conversational AI prototype, available at www.chatclimate.ai. The product answers challenging questions accurately in three different QA scenarios: asking from 1) GPT-4, 2) ChatClimate, and 3) hybrid ChatClimate. The answers and their sources were evaluated by a team of IPCC authors, who used their expert knowledge to score the accuracy of the answers from 1 (very-low) to 5 (very-high). The evaluation showed that the Standalone assistant provided more accurate answers.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- A Bibliometric Review of Large Language Models Research from 2017 to 2023ACM Transactions on Intelligent Systems and Technology, 2024
- Towards Healthy AI: Large Language Models Need Therapists Too2024
- Self-Prompting Large Language Models for Zero-Shot Open-Domain QA2024
- Prompting Large Language Models with Answer Heuristics for Knowledge-Based Visual Question Answering2023
- Survey of Hallucination in Natural Language GenerationACM Computing Surveys, 2023
- The Choice of Textual Knowledge Base in Automated Claim CheckingJournal of Data and Information Quality, 2023
- Cheap talk and cherry-picking: What ClimateBert has to say on corporate climate risk disclosuresFinance Research Letters, 2022
- Machine-learning-based evidence and attribution mapping of 100,000 climate impact studiesNature Climate Change, 2021
- It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners2021
- In bot we trust: A new methodology of chatbot performance measuresBusiness Horizons, 2019