A Change in Perspective: The Trade-Off Between Perspective API and Custom Models in Classifying Hate Speech in Portuguese
Open Access
- 17 November 2024
- conference paper
- Published by Sociedade Brasileira de Computação
- p. 23-31
Abstract
This paper examines the performance of the Perspective API, developed by Jigsaw, in detecting hate speech in Portuguese. Although the Perspective API supports multiple languages, its performance metrics are often aggregated, obscuring specific details. Our study reveals that the API’s AUC-ROC score for Portuguese is significantly lower than for English (0.744 vs. 0.942). To address this, we developed a BERT classifier model trained on a Portuguese Twitter hate speech dataset. Our model, with just 100 messages in it’s training set, outperformed the Perspective API. These findings highlight the need for more granular performance metrics and suggest that custom models may offer better solutions for specific languages.Keywords
This publication has 8 references indexed in Scilit:
- Toxic Bias: Perspective API Misreads German as More ToxicProceedings of the International AAAI Conference on Web and Social Media, 2025
- A Sentiment Analysis Benchmark for Automated Machine Learning Applications and a Proof of Concept in Hate Speech Detection2023
- Should We Translate? Evaluating Toxicity in Online Comments when Translating from Portuguese to English2022
- A New Generation of Perspective API: Efficient Multilingual Character-level Transformers2022
- BERTimbau: Pretrained BERT Models for Brazilian Portuguese2020
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- A Hierarchically-Labeled Portuguese Hate Speech Dataset2019
- Automated Hate Speech Detection and the Problem of Offensive LanguageProceedings of the International AAAI Conference on Web and Social Media, 2017