A study on the distribution of social biases in self-supervised learning visual models
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 10432-10441
Abstract
Deep neural networks are efficient at learning the data distribution if it is sufficiently sampled. However, they can be strongly biased by non-relevant factors implicitly incorporated in the training data. These include operational biases, such as ineffective or uneven data sampling, but also ethical concerns, as the social biases are implicitly present—even inadvertently, in the training data or explicitly defined in unfair training schedules. In tasks having impact on human processes, the learning of social biases may produce discriminatory, unethical and untrustworthy consequences. It is often assumed that social biases stem from supervised learning on labelled data, and thus, Self-Supervised Learning (SSL) wrongly appears as an efficient and bias-free solution, as it does not require labelled data. However, it was recently proven that a popular SSL method also incorporates biases. In this paper, we study the biases of a varied set of SSL visual models, trained using ImageNet data, using a method and dataset designed by psychological experts to measure social biases. We show that there is a correlation between the type of the SSL model and the number of biases that it incorporates. Furthermore, the results also suggest that this number does not strictly depend on the model's accuracy and changes throughout the network. Finally, we conclude that a careful SSL model selection process can reduce the number of social biases in the deployed model, whilst keeping high performance. The code is available at https://github.com/vpulab/SB-SSL.Keywords
All Related Versions
Funding Information
- Consejería de Educación e Investigación (SII/PJI/2019-00414)
This publication has 20 references indexed in Scilit:
- Unsupervised Feature Learning via Non-parametric Instance Discrimination2018
- Dual-Glance Model for Deciphering Social Relationships2017
- Network Dissection: Quantifying Interpretability of Deep Visual Representations2017
- Semantics derived automatically from language corpora contain human-like biasesScience, 2017
- Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints2017
- Deep Residual Learning for Image Recognition2016
- Unsupervised Visual Representation Learning by Context Prediction2015
- Going deeper with convolutions2015
- Dimensionality Reduction by Learning an Invariant Mapping2006
- Measuring individual differences in implicit cognition: The implicit association test.Journal of Personality and Social Psychology, 1998