Quantifying How Hateful Communities Radicalize Online Users
Preprint
Abstract
While online social media offers a way for ignored or stifled voices to be heard, it also allows users a platform to spread hateful speech. Such speech usually originates in fringe communities, yet it can spill over into mainstream channels. In this paper, we measure the impact of joining fringe hateful communities in terms of hate speech propagated to the rest of the social network. We leverage data from Reddit to assess the effect of joining one type of echo chamber: a digital community of like-minded users exhibiting hateful behavior. We measure members' usage of hate speech outside the studied community before and after they become active participants. Using Interrupted Time Series (ITS) analysis as a causal inference method, we gauge the spillover effect, in which hateful language from within a certain community can spread outside that community by using the level of out-of-community hate word usage as a proxy for learned hate. We investigate four different Reddit sub-communities (subreddits) covering three areas of hate speech: racism, misogyny and fat-shaming. In all three cases we find an increase in hate speech outside the originating community, implying that joining such community leads to a spread of hate speech throughout the platform. Moreover, users are found to pick up this new hateful speech for months after initially joining the community. We show that the harmful speech does not remain contained within the community. Our results provide new evidence of the harmful effects of echo chambers and the potential benefit of moderating them to reduce adoption of hateful speech.Keywords
All Related Versions
- Version 1, 2022-09-19, ArXiv
- Version 2, 2022-10-02, ArXiv
- Published version: , 139.
This publication has 14 references indexed in Scilit:
- Quarantined! Examining the Effects of a Community-Wide Moderation Intervention on RedditACM Transactions on Computer-Human Interaction, 2022
- Strategic identity signaling in heterogeneous networksProceedings of the National Academy of Sciences, 2022
- Assessing the Extent and Types of Hate Speech in Fringe Communities: A Case Study of Alt-Right Communities on 8chan, 4chan, and RedditSocial Media + Society, 2021
- The Language of Derogation and Hate: Functions, Consequences, and ReappropriationJournal of Language and Social Psychology, 2020
- Upvoting extremism: Collective identity formation and the extreme right on RedditNew Media & Society, 2020
- Reddit quarantined: can changing platform affordances reduce hateful material online?Internet Policy Review, 2020
- Ingroup Love or Outgroup Hate (or Both)? Mapping Distinct Bias Profiles in the PopulationPersonality and Social Psychology Bulletin, 2019
- #Gamergate and The Fappening: How Reddit’s algorithm, governance, and culture support toxic technoculturesNew Media & Society, 2016
- Matching Methods for Causal Inference: A Review and a Look ForwardStatistical Science, 2010
- Does Head Start Improve Children's Life Chances? Evidence from a Regression Discontinuity DesignThe Quarterly Journal of Economics, 2007