LLMs as Repositories of Factual Knowledge: Limitations and Solutions
- 6 April 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 2213-2226
Abstract
LLMs' sources of knowledge are data snapshots containing factual information about entities collected at different timestamps and from different media types (e.g. wikis, social media, etc.). Such unstructured knowledge is subject to change due to updates through time from past to present. Equally important are the inconsistencies and inaccuracies occurring in different information sources. Consequently, the model's knowl edge about an entity may be perturbed while training over the sequence of snapshots or at inference time, resulting in inconsistent and inaccurate model performance. In this work, we study the appropriateness of Large Language Models (LLMs) as repositories of factual knowledge. We consider twenty-four state of-the-art LLMs that are either closed-, partially (weights), or fully (weight and training data) open-source. We evaluate their reliability in responding to time-sensitive factual questions in terms of accuracy and consistency when prompts are perturbed. We further evaluate the effectiveness of state-of-the-art methods to improve LLMs' accuracy and consistency. We then propose “ENtity-Aware Fine-tuning” (ENAF), a soft neurosymbolic approach aimed at providing structured representation of entities during fine-tuning to reduce inconsistencies and improve response stability under prompt variations.Keywords
All Related Versions
Funding Information
- MUR PNRR Project FAIR - Future AI Research (PE00000013)
- NextGenerationEU
This publication has 29 references indexed in Scilit:
- A Survey on Knowledge Editing of Neural NetworksIEEE Transactions on Neural Networks and Learning Systems, 2024
- The Power of Noise: Redefining Retrieval for RAG Systems2024
- Evaluating the Ripple Effects of Knowledge Editing in Language ModelsTransactions of the Association for Computational Linguistics, 2024
- Time-Aware Language Models as Temporal Knowledge BasesTransactions of the Association for Computational Linguistics, 2022
- Evaluation of Response Generation Models: Shouldn’t It Be Shareable and Replicable?2022
- TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models2022
- Transforming Sequence Tagging Into A Seq2Seq Task2022
- WikidataCommunications of the ACM, 2014
- On Some Properties of the Star GraphVLSI Design, 1995
- Addition and removal of information for a knowledge base with incomplete informationData & Knowledge Engineering, 1992