Balancing efficacy and computational burden: weighted mean, multiple imputation, and inverse probability weighting methods for item non-response in reliable scales
Open Access
- 13 August 2024
- journal article
- research article
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 31 (12) , 2869-2879
Abstract
Scales often arise from multi-item questionnaires, yet commonly face item non-response. Traditional solutions use weighted mean (WMean) from available responses, but potentially overlook missing data intricacies. Advanced methods like multiple imputation (MI) address broader missing data, but demand increased computational resources. Researchers frequently use survey data in the All of Us Research Program (All of Us), and it is imperative to determine if the increased computational burden of employing MI to handle non-response is justifiable. Using the 5-item Physical Activity Neighborhood Environment Scale (PANES) in All of Us, this study assessed the tradeoff between efficacy and computational demands of WMean, MI, and inverse probability weighting (IPW) when dealing with item non-response. Synthetic missingness, allowing 1 or more item non-response, was introduced into PANES across 3 missing mechanisms and various missing percentages (10%-50%). Each scenario compared WMean of complete questions, MI, and IPW on bias, variability, coverage probability, and computation time. All methods showed minimal biases (all 8000 and >100 times longer than WMean and IPW in full data analysis, respectively. The marginal performance advantages of MI for item non-response in highly reliable scales do not warrant its escalated cloud computational burden in All of Us, particularly when coupled with computationally demanding post-imputation analyses. Researchers using survey scales with low missingness could utilize WMean to reduce computing burden.Keywords
Funding Information
- National Institutes of Health
- Office of the Director: Data and Research Center (3OT2OD035404)
- National Institute on Minority Health and Health Disparities (R21MD019103)
This publication has 22 references indexed in Scilit:
- Measuring social determinants of health in the All of Us Research ProgramScientific Reports, 2024
- The “All of Us” Research ProgramNew England Journal of Medicine, 2019
- Development of the Initial Surveys for the All of Us Research ProgramEpidemiology, 2019
- Accounting for missing data in statistical analyses: multiple imputation is not always the answerInternational Journal of Epidemiology, 2019
- Missing data in a multi-item instrument were best handled by multiple imputation at the item score levelJournal of Clinical Epidemiology, 2014
- Missing Data and Multiple ImputationJAMA Pediatrics, 2013
- Evaluating a Brief Self-Report Measure of Neighborhood Environments for Physical Activity Research and Surveillance: Physical Activity Neighborhood Environment Scale (PANES)Journal of Physical Activity and Health, 2010
- How Many Imputations are Really Needed? Some Practical Clarifications of Multiple Imputation TheoryPrevention Science, 2007
- A review of techniques for treating missing data in OM survey researchJournal of Operations Management, 2005
- Coefficient Alpha and the Internal Structure of TestsPsychometrika, 1951