Investigating African-American Vernacular English in Transformer-Based Text Generation
Preprint
Abstract
The growth of social media has encouraged the written use of African American Vernacular English (AAVE), which has traditionally been used only in oral contexts. However, NLP models have historically been developed using dominant English varieties, such as Standard American English (SAE), due to text corpora availability. We investigate the performance of GPT-2 on AAVE text by creating a dataset of intent-equivalent parallel AAVE/SAE tweet pairs, thereby isolating syntactic structure and AAVE- or SAE-specific language for each pair. We evaluate each sample and its GPT-2 generated text with pretrained sentiment classifiers and find that while AAVE text results in more classifications of negative sentiment than SAE, the use of GPT-2 generally increases occurrences of positive sentiment for both. Additionally, we conduct human evaluation of AAVE and SAE text generated with GPT-2 to compare contextual rigor and overall quality.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Dialect-Specific Models for Automatic Speech Recognition of African American Vernacular English2019
- Testifying While Black: An Experimental Study of Court Reporter Accuracy in Transcription of African American EnglishLanguage, 2019
- The Woman Worked as a Babysitter: On Biases in Language Generation2019
- GLTR: Statistical Detection and Visualization of Generated Text2019
- The Risk of Racial Bias in Hate Speech Detection2019
- Demographic Dialectal Variation in Social Media: A Case Study of African-American English2016
- Learning a POS tagger for AAVE-like language2016
- Toward a Description of African American Vernacular English Dialect Regions Using “Black Twitter”American Speech, 2015
- VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media TextProceedings of the International AAAI Conference on Web and Social Media, 2014
- African American English: A Linguistic IntroductionEnglish World-Wide, 2004