Analyzing Bias in Swiss Federal Supreme Court Judgments Using Facebook's Holistic Bias Dataset: Implications for Language Model Training
Authors:
Preprint
Abstract
Natural Language Processing (NLP) is vital for computers to process and respond accurately to human language. However, biases in training data can introduce unfairness, especially in predicting legal judgment. This study focuses on analyzing biases within the Swiss Judgment Prediction Dataset (SJP-Dataset). Our aim is to ensure unbiased factual descriptions essential for fair decision making by NLP models in legal contexts. We analyze the dataset using social bias descriptors from the Holistic Bias dataset and employ advanced NLP techniques, including attention visualization, to explore the impact of dispreferred descriptors on model predictions. The study identifies biases and examines their influence on model behavior. Challenges include dataset imbalance and token limits affecting model performance.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Improving Robustness in Language Models for Legal Textual Entailment Through Artifact-Aware Training2024
- MultiLegalPile: A 689GB Multilingual Legal Corpus2024
- Gender Attitudes in the Judiciary: Evidence from US Circuit CourtsAmerican Economic Journal: Applied Economics, 2024
- Gender bias in legal corpora and debiasing itNatural Language Engineering, 2022
- An Annotation Schema for the Detection of Social Bias in Legal Text Corpora2022
- Perturbation Augmentation for Fairer NLP2022
- “I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset2022
- In-group bias in the Indian judiciary: Evidence from 5.5 million criminal cases2021
- StereoSet: Measuring stereotypical bias in pretrained language models2021
- Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation2020