Future Lens: Anticipating Subsequent Tokens from a Single Hidden State
Preprint
Abstract
We conjecture that hidden state vectors corresponding to individual input tokens encode information sufficient to accurately predict several tokens ahead. More concretely, in this paper we ask: Given a hidden (internal) representation of a single token at position $t$ in an input, can we reliably anticipate the tokens that will appear at positions $\geq t + 2$? To test this, we measure linear approximation and causal intervention methods in GPT-J-6B to evaluate the degree to which individual hidden states in the network contain signal rich enough to predict future hidden states and, ultimately, token outputs. We find that, at some layers, we can approximate a model's output with more than 48% accuracy with respect to its prediction of subsequent tokens through a single hidden state. Finally we present a "Future Lens" visualization that uses these methods to create a new view of transformer states.
Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- A Survey on Non-Autoregressive Generation for Neural Machine Translation and BeyondIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- Preventing Generation of Verbatim Memorization in Language Models Gives a False Sense of Privacy2023
- Transformer Feed-Forward Layers Are Key-Value Memories2021
- BERxiT: Early Exiting for BERT with Better Fine-Tuning and Extension to Regression2021
- Non-Autoregressive Text Generation with Pre-trained Language Models2021
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Universal Adversarial Triggers for Attacking and Analyzing NLP2019
- Finding structure in timeCognitive Science, 1990