Is Risk-Adjustor Selection More Important Than Statistical Approach for Provider Profiling? Asthma as an Example
Authors:
- 1 January 2005
- journal article
- Published by SAGE Publications in Medical Decision Making
- Vol. 25 (1) , 20-34
Abstract
Objectives. To examine how the selections of different risk adjustors and statistical approaches affect the profiles of physician groups on patient satisfaction. Data sources. Mailed patient surveys. Patients with asthma were selected randomly from each of 20 California physician groups between July 1998 and February 1999. A total of 2515 patients responded. Research design. A cross-sectional study. Patient satisfaction with asthma care was the performance indicator for physician group profiling. Candidate variables for risk-adjustment model development included sociodemographic, clinical characteristics, and self-reported health status. Statistical strategies were the ratio of observed-to-expected rate (OE), fixed effects (FE), and the random effects (RE) approaches. Model performance was evaluated using indicators of discrimination (C-statistic) and calibration (Hosmer-Lemeshow χ2). Ranking impact of using different risk adjustors and statistical approaches was based on the changes in absolute ranking (AR) and quintile ranking (QR) of physician group performance and the weighted kappa for quintile ranking. Results. Variables that added significantly to the discriminative power of risk-adjustment models included sociodemographic (age, sex, prescription drug coverage), clinical (asthma severity), and health status (SF-36 PCS and MCS). Based on an acceptable goodness-of-fit (P > 0.1)and higher C-statistics, models adjusting for sociodemographic, clinical, and health status variables (Model S-C-H) using either the FE or RE approach were more favorable. However, the C-statistic (=0.68) was only fair for both models. The influence of risk-adjustor selection on change of performance ranking was more salient than choice of statistical strategy (AR: 50%-80% v. 20%-55%; QR: 10%-30% v. 0%-10%). Compared to the model adjusting for sociodemographic and clinical variables only and using OE approach, the Model S-C-H using RE approach resulted in 70% of groups changing in AR and 25% changing in QR (weighted kappa: 0.88). Compared to the Consumer Assessment of Health Plans model, the Model S-C-H using RE approach resulted in 65% of groups changing in AR and 20% changing in QR (weighted kappa: 0.88). Conclusions. In comparing the performance of physician groups on patient satisfaction with asthma care, the use of sociodemographic, clinical, and health status variables maximized risk-adjustment model performance. Selection of risk adjustors had more influence on ranking profiles than choice of statistical strategies. Stakeholders employing provider profiling should pay careful attention to the selection of both variables and statistical approach used in risk-adjustment.Keywords
This publication has 52 references indexed in Scilit:
- A Comparison of Bayesian Methods for Profiling Hospital PerformanceMedical Decision Making, 2002
- Factoring Socioeconomic Status Into Cardiac Performance Profiling for HospitalsMedical Care, 2002
- California Provider Group Report Cards: What Do They Tell Us?American Journal of Medical Quality, 2001
- Compliance with US asthma management guidelines and specialty care: a regional variation or national concern?Journal of Evaluation in Clinical Practice, 1999
- Assessing Quality of Care via HEDIS 3.0: Is There a Better Way?Archives of Family Medicine, 1998
- Statistical Methods for Profiling Providers of Medical Care: Issues and ApplicationsJournal of the American Statistical Association, 1997
- Age and health status in a patient satisfaction surveySocial Science & Medicine, 1996
- Variations in the Utilization of Coronary Angiography for Elderly Patients with an Acute Myocardial InfarctionMedical Care, 1995
- How Will Outcomes Management Work?Health Affairs, 1994
- A new method of classifying prognostic comorbidity in longitudinal studies: Development and validationJournal of Chronic Diseases, 1987