Treatment effect estimation by comparing observed and predicted outcomes: conditions for valid inference and practical illustration
Open Access
- 24 July 2026
- journal article
- research article
- Published by Springer Nature in Diagnostic and Prognostic Research
- Vol. 10 (1) , 20
Abstract
Background: One non-randomized approach to estimate the average effect of a newly introduced treatment is to compare observed outcomes under the new treatment with predicted outcomes under the standard treatment. These counterfactual predictions are made using a model developed before the new treatment was introduced, using patient characteristics and individualized treatment information. Although the approach has been intuitively applied (e.g., as model-based clinical evaluation in radiotherapy) and can be recognized as a specific case of standardization, a method of virtual controls or a g-method, the theory and conditions required for unbiased treatment effect estimation have not been formally described. The objective of this paper is to formalize the approach and clarify these conditions. Methods: We formalize the approach within the potential outcomes framework for causal inference. We explain the methodology, its necessary conditions, and approaches for assessing their validity. These conditions are furthermore illustrated through a case study from radiotherapy, estimating the benefit of proton therapy compared to photon therapy on dysphagia in patients with head and neck cancer. Results: We describe a set of five sufficient conditions, including examples of violations: transportability, ignorability of treatment assignment, consistency, positivity, and correct model specification. While these conditions are largely untestable, we describe how empirical evidence, such as comparing predicted and observed outcomes in related samples, can increase confidence in their plausibility. Conclusion: When the prediction model predicts well in relevant (sub)populations, the approach can yield unbiased treatment effect estimates. However, there are many possible sources of bias. Therefore, we recommend systematic consideration of all required conditions, informed by domain expertise, and the use of empirical evidence whenever possible to support their plausibility.Keywords
All Related Versions
Funding Information
- ZonMw (10580012210025)
- ZonMw (10580012210025)
- ZonMw (10580012210025)
This publication has 27 references indexed in Scilit:
- Defining the role of real-world data in cancer clinical research: The position of the European Organisation for Research and Treatment of CancerEuropean Journal of Cancer, 2023
- Sensitivity analysis using bias functions for studies extending inferences from a randomized trial to a target populationStatistics in Medicine, 2023
- Comprehensive toxicity risk profiling in radiation therapy for head and neck cancer: A new concept for individually optimised treatmentRadiotherapy and Oncology, 2021
- Modeling the Potential Benefits of Proton Therapy for Patients With Oropharyngeal Head and Neck CancerInternational Journal of Radiation Oncology - Biology - Physics (IJROBP), 2019
- A Model-Based Approach to Predict Short-Term Toxicity Benefits With Proton Therapy for Oropharyngeal CancerInternational Journal of Radiation Oncology - Biology - Physics (IJROBP), 2019
- Swallowing sparing intensity modulated radiotherapy (SW-IMRT) in head and neck cancer: Clinical validation according to the model-based approachRadiotherapy and Oncology, 2016
- Selection of patients for radiotherapy with protons aiming at reduction of side effects: The model-based approachRadiotherapy and Oncology, 2013
- Targeted Learning2011
- Predictive value of the Seattle Heart Failure Model in patients undergoing left ventricular assist device placementThe Journal of Heart and Lung Transplantation, 2010
- The combination of randomized and historical controls in clinical trialsJournal of Chronic Diseases, 1976