A Penalty Approach for Normalizing Feature Distributions to Build Confounder-Free Models
- 16 September 2022
- book chapter
- Published by Springer Nature
- Vol. 13433, 387-397
Abstract
Translating machine learning algorithms into clinical applications requires addressing challenges related to interpretability, such as accounting for the effect of confounding variables (or metadata). Confounding variables affect the relationship between input training data and target outputs. When we train a model on such data, confounding variables will bias the distribution of the learned features. A recent promising solution, MetaData Normalization (MDN), estimates the linear relationship between the metadata and each feature based on a non-trainable closed-form solution. However, this estimation is confined by the sample size of a mini-batch and thereby may cause the approach to be unstable during training. In this paper, we extend the MDN method by applying a Penalty approach (referred to as PDMN). We cast the problem into a bi-level nested optimization problem. We then approximate this optimization problem using a penalty method so that the linear parameters within the MDN layer are trainable and learned on all samples. This enables PMDN to be plugged into any architectures, even those unfit to run batch-level operations, such as transformers and recurrent models. We show improvement in model accuracy and greater independence from confounders using PMDN over MDN in a synthetic experiment and a multi-label, multi-site dataset of magnetic resonance images (MRIs). Keywords Confounders Neuroscience Fairness Deep learningKeywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Deep learning identifies morphological determinants of sex differences in the pre-adolescent brainNeuroImage, 2020
- Momentum Batch Normalization for Deep Learning with Small Batch Size2020
- Chained regularization for identifying brain patterns specific to HIV infectionNeuroImage, 2018
- Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition2017
- Focal Loss for Dense Object Detection2017
- An overview on data representation learning: From traditional feature learning to recent deep learningThe Journal of Finance and Data Science, 2016
- Extracting patterns of morphometry distinguishing HIV associated neurodegeneration from mild cognitive impairment via group cardinality constrained classificationHuman Brain Mapping, 2016
- Alzheimer's Disease Neuroimaging Initiative (ADNI)Neurology, 2010
- Heterogeneity in mild cognitive impairment: Differences in neuropsychological profile and associated white matter lesion pathologyJournal of the International Neuropsychological Society, 2009
- A Stochastic Approximation MethodThe Annals of Mathematical Statistics, 1951