Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
Authors:
Preprint
Abstract
We introduce Inference-Time Intervention (ITI), a technique designed to enhance the truthfulness of large language models (LLMs). ITI operates by shifting model activations during inference, following a set of directions across a limited number of attention heads. This intervention significantly improves the performance of LLaMA models on the TruthfulQA benchmark. On an instruction-finetuned LLaMA called Alpaca, ITI improves its truthfulness from 32.5% to 65.1%. We identify a tradeoff between truthfulness and helpfulness and demonstrate how to balance it by tuning the intervention strength. ITI is minimally invasive and computationally inexpensive. Moreover, the technique is data efficient: while approaches like RLHF require extensive annotations, ITI locates truthful directions using only few hundred examples. Our findings suggest that LLMs may have an internal representation of the likelihood of something being true, even as they produce falsehoods on the surface.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Discovering Language Model Behaviors with Model-Written Evaluations2023
- Self-Instruct: Aligning Language Models with Self-Generated Instructions2023
- Probing Classifiers: Promises, Shortcomings, and AdvancesComputational Linguistics, 2022
- TruthfulQA: Measuring How Models Mimic Human Falsehoods2022
- BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models2022
- Extracting Latent Steering Vectors from Pretrained Language Models2022
- Retrieval Augmentation Reduces Hallucination in Conversation2021
- GeDi: Generative Discriminator Guided Sequence Generation2021
- Natural Questions: A Benchmark for Question Answering ResearchTransactions of the Association for Computational Linguistics, 2019
- TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension2017