Measuring and Modifying Factual Knowledge in Large Language Models
Author:
- 15 December 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 831-838
Abstract
Large Language Models (LLMs) store an extensive amount of factual knowledge obtained from vast collections of text. To effectively utilize these models for downstream tasks, it is crucial to have reliable methods for measuring their knowledge. However, existing approaches for knowledge measurement have certain limitations, and despite recent efforts, they fail to provide accurate measurements and the necessary insights for modifying the knowledge within LLMs. In this work, we employ information theory-based measurements to provide a framework estimating the factual knowledge contained within large language models. More specifically, we measure knowledge by analyzing the LLM's prediction probability distribution before and after instilling the target knowledge, employing metrics such as entropy and KL-divergence. Introducing our metrics, we first assess their accuracy in comparison to previous ranking-based methods, surpassing them by around 30% in a synthetic experiment. Then, we explore two prominent methods of knowledge instillation, discovering that LLMs exhibit limitations in capturing new knowledge under specific circumstances for one of these methods. Lastly, we demonstrate the applicability of our methods in extracting unlearned and mislearned facts in LLMs through their application to in-context learning.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Survey of Hallucination in Natural Language GenerationACM Computing Surveys, 2023
- Crawling The Internal Knowledge-Base of Language Models2023
- Training Language Models to Follow Instructions with Human Feedback2022
- Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settings2022
- Retrieval Augmentation Reduces Hallucination in Conversation2021
- Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases2021
- Probing Pretrained Language Models for Lexical Semantics2020
- RoBERTa: A Robustly Optimized BERT Pretraining Approach2019
- Assessing The Factual Accuracy of Generated Text2019
- Language Models as Knowledge Bases?2019