Improving Inverse Probability Weighting by Post-calibrating Its Propensity Scores
Open Access
- 15 April 2024
- journal article
- research article
- Published by Wolters Kluwer Health in Epidemiology
- Vol. 35 (4) , 473-480
Abstract
Theoretical guarantees for causal inference using propensity scores are partially based on the scores behaving like conditional probabilities. However, prediction scores between zero and one do not necessarily behave like probabilities, especially when output by flexible statistical estimators. We perform a simulation study to assess the error in estimating the average treatment effect before and after applying a simple and well-established postprocessing method to calibrate the propensity scores. We observe that postcalibration reduces the error in effect estimation and that larger improvements in calibration result in larger improvements in effect estimation. Specifically, we find that expressive tree-based estimators, which are often less calibrated than logistic regression-based models initially, tend to show larger improvements relative to logistic regression-based models. Given the improvement in effect estimation and that postcalibration is computationally cheap, we recommend its adoption when modeling propensity scores with expressive models.Keywords
This publication has 26 references indexed in Scilit:
- Transparency of high‐dimensional propensity score analyses: Guidance for diagnostics and reportingPharmacoepidemiology and Drug Safety, 2022
- The Integrated Calibration Index (ICI) and related metrics for quantifying the calibration of logistic regression modelsStatistics in Medicine, 2019
- A calibration hierarchy for risk models was defined: from utopia to empirical dataJournal of Clinical Epidemiology, 2016
- Propensity score estimation: neural networks, support vector machines, decision trees (CART), and meta-classifiers as alternatives to logistic regressionJournal of Clinical Epidemiology, 2010
- Random ForestsMachine Learning, 2001
- Ridge Regression: Biased Estimation for Nonorthogonal ProblemsTechnometrics, 2000
- Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity ScoreThe American Statistician, 1985
- Reducing Bias in Observational Studies Using Subclassification on the Propensity ScoreJournal of the American Statistical Association, 1984
- Estimating causal effects of treatments in randomized and nonrandomized studies.Journal of Educational Psychology, 1974
- A Generalization of Sampling Without Replacement from a Finite UniverseJournal of the American Statistical Association, 1952