UNQOVERing Stereotyping Biases via Underspecified Questions
Preprint
Abstract
While language embeddings have been shown to have stereotyping biases, how these biases affect downstream question answering (QA) models remains unexplored. We present UNQOVER, a general framework to probe and quantify biases through underspecified questions. We show that a naive use of model scores can lead to incorrect bias estimates due to two forms of reasoning errors: positional dependence and question independence. We design a formalism that isolates the aforementioned errors. As case studies, we use this metric to analyze four important classes of stereotypes: gender, nationality, ethnicity, and religion. We probe five transformer-based QA models trained on two QA datasets, along with their underlying language models. Our broad study reveals that (1) all these models, with and without fine-tuning, have notable stereotyping biases in these classes; (2) larger models often have higher bias; and (3) the effect of fine-tuning on bias varies strongly with the dataset and the model size.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings2021
- StereoSet: Measuring stereotypical bias in pretrained language models2021
- Hurtful words2020
- Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection2020
- Language (Technology) is Power: A Critical Survey of “Bias” in NLP2020
- Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer2020
- The Woman Worked as a Babysitter: On Biases in Language Generation2019
- Word embeddings quantify 100 years of gender and ethnic stereotypesProceedings of the National Academy of Sciences, 2018
- SQuAD: 100,000+ Questions for Machine Comprehension of Text2016
- Most people are not WEIRDNature, 2010