Wim Delva
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Wim Delva has a total of 195 co-authors. Here are the top co-authors by number of shared publications:
Marleen Temmerman
Faculty of Medicine and Health Sciences, Ghent University, Gent, Belgium
11 shared publications- Niel Hens
Vaccin- en Infectieziekten Instituut, Universiteit Antwerpen, Antwerp, Belgium
10 shared publications - Yves Lafort
International Centre for Reproductive Health, Ghent University, Ghent, Belgium
9 shared publications - Marleen Temmerman
Aga Khan University Centre of Excellence in Women and Child Health, P.O. Box 30270 - 00100, Nairobi, Kenya
8 shared publications - Matthew Francis Chersich
Wits Planetary Health Research Division, Faculty of Health Sciences, University of Witwatersrand
8 shared publications - Roxanne Beauclair
International Centre for Reproductive Health, Ghent University, Gent, Belgium
8 shared publications
Latest Publications
- Open Access
- 26 July 2022
- Published by Center for Open Science in OSF Preprints
The Peril of Imprecise Data in Healthcare – A Simple Example of Blood Pressure Measurements and Cardiovascular Mortality Risk
In the current era of big biomedical data, the field of medical data science has made many advances in predictive modelling and medical decisions derived from electronic health records. Issues around data quality have received comparatively much less attention. To create awareness around the importance of high quality data, we set out to develop a number of case studies that illustrate what can go wrong when medical analytics and decision making is based on poor quality data. In this short paper, we focus on imprecise data. Specifically, we demonstrate how even relatively small random measurement errors in blood pressure data may lead to a gross overestimation of the impact of a blood pressure lowering intervention on cardiovascular mortality.
- 26 July 2022
- Published by Center for Open Science in OSF Preprints
Open AccessThe Peril of Imprecise Data in Healthcare – A Simple Example of Blood Pressure Measurements and Cardiovascular Mortality Risk
In the current era of big biomedical data, the field of medical data science has made many advances in predictive modelling and medical decisions derived from electronic health records. Issues around data quality have received comparatively much less attention. To create awareness around the importance of high quality data, we set out to develop a number of case studies that illustrate what can go wrong when medical analytics and decision making is based on poor quality data. In this short paper, we focus on imprecise data. Specifically, we demonstrate how even relatively small random measurement errors in blood pressure data may lead to a gross overestimation of the impact of a blood pressure lowering intervention on cardiovascular mortality.
- Open Access
- 20 October 2021
- Published by MDPI AG in Mathematics
- Vol. 9 (21) , 2645
Inferring HIV Transmission Network Determinants Using Agent-Based Models Calibrated to Multi-Data Sources
(1) Background: Calibration of Simpact Cyan can help to improve estimates related to the transmission dynamics of the Human Immunodeficiency Virus (HIV). Age-mixing patterns in sexual partnerships, onward transmissions, and temporal trends of HIV incidence are determinants which can inform the design of efficient prevention, and linkage-to-care programs. Using an agent-based model (ABM) simulation tool, we investigated, through a simulation study, if estimates of these determinants can be obtained with high accuracy by combining summary features from different data sources. (2) Methods: With specific parameters, we generated the benchmark data, and calibrated the default model in three scenarios based on summary features for comparison. For calibration, we used Latin Hypercube Sampling approach to generate parameter values, and Approximation Bayesian Computation to choose the best fitting ones. In all calibration scenarios the mean square root error was used as a measure to depict the estimates accuracy. (3) Results: The accuracy measure showed relatively no difference between the three scenarios. Moreover, we found that in all scenarios, age and gender strata incidence trends were poorly estimated. (4) Conclusions: Using synthetic benchmarks, we showed that it is possible to infer HIV transmission dynamics using an ABM of HIV transmission. Our results suggest that any type of summary feature provides adequate information to estimate HIV transmission network determinants. However, it is advisable to check the level of accuracy of the estimates of interest using benchmark data.
- 20 October 2021
- Published by MDPI AG in Mathematics
- Vol. 9 (21) , 2645
Open AccessInferring HIV Transmission Network Determinants Using Agent-Based Models Calibrated to Multi-Data Sources
(1) Background: Calibration of Simpact Cyan can help to improve estimates related to the transmission dynamics of the Human Immunodeficiency Virus (HIV). Age-mixing patterns in sexual partnerships, onward transmissions, and temporal trends of HIV incidence are determinants which can inform the design of efficient prevention, and linkage-to-care programs. Using an agent-based model (ABM) simulation tool, we investigated, through a simulation study, if estimates of these determinants can be obtained with high accuracy by combining summary features from different data sources. (2) Methods: With specific parameters, we generated the benchmark data, and calibrated the default model in three scenarios based on summary features for comparison. For calibration, we used Latin Hypercube Sampling approach to generate parameter values, and Approximation Bayesian Computation to choose the best fitting ones. In all calibration scenarios the mean square root error was used as a measure to depict the estimates accuracy. (3) Results: The accuracy measure showed relatively no difference between the three scenarios. Moreover, we found that in all scenarios, age and gender strata incidence trends were poorly estimated. (4) Conclusions: Using synthetic benchmarks, we showed that it is possible to infer HIV transmission dynamics using an ABM of HIV transmission. Our results suggest that any type of summary feature provides adequate information to estimate HIV transmission network determinants. However, it is advisable to check the level of accuracy of the estimates of interest using benchmark data.
- Open Access
Four-week forecasts of COVID-19 epidemic trajectories in South Africa, Chile, Peru and Brazil: a model evaluation
Introduction From the beginning of the COVID-19 pandemic, epidemiological models have been used in a number of ways to aid governments and organizations in efficient planning of resources and decision making. These models have elucidated important epidemiological transmission parameters, in addition to making short-term projections. Methods We constructed a compartmental mathematical model for the transmission, detection and prevention of SARS-CoV-2 infections for regions where Anglo American has mining operations. We fitted the model to publicly available data and used it to make short-term projections. Finally, we evaluated how the model performed by comparing short-term projections to actual confirmed cases, retrospectively. Findings The average forecast errors for four-week-ahead projections ranged between 1% and 8% in all the countries and regions considered in this study. All but one region had more than 75% of the true values falling within the range of four-week-ahead projections. The quality of the projections improved with time as expected due to increased historical data. Conclusion Our model produced four-week forecasts with a sufficiently high level of accuracy to guide operational and strategic planning for business continuity and COVID-19 responses in Anglo American mining sites.
Open AccessFour-week forecasts of COVID-19 epidemic trajectories in South Africa, Chile, Peru and Brazil: a model evaluation
Introduction From the beginning of the COVID-19 pandemic, epidemiological models have been used in a number of ways to aid governments and organizations in efficient planning of resources and decision making. These models have elucidated important epidemiological transmission parameters, in addition to making short-term projections. Methods We constructed a compartmental mathematical model for the transmission, detection and prevention of SARS-CoV-2 infections for regions where Anglo American has mining operations. We fitted the model to publicly available data and used it to make short-term projections. Finally, we evaluated how the model performed by comparing short-term projections to actual confirmed cases, retrospectively. Findings The average forecast errors for four-week-ahead projections ranged between 1% and 8% in all the countries and regions considered in this study. All but one region had more than 75% of the true values falling within the range of four-week-ahead projections. The quality of the projections improved with time as expected due to increased historical data. Conclusion Our model produced four-week forecasts with a sufficiently high level of accuracy to guide operational and strategic planning for business continuity and COVID-19 responses in Anglo American mining sites.
- Open Access
Calibration of models to data: a comparison of methods
Complex models are often fitted to data using simulation-based calibration, a computationally challenging process. Several calibration methods to improve computational efficiency have been developed with no consensus on which methods perform best. We did a simulation study comparing the performance of 5 methods that differed in their Goodness-of-Fit (GOF) metrics and parameter search strategies. Posterior densities for two parameters of a simple Susceptible-Infectious-Recovered epidemic model were obtained for each calibration method under two scenarios. Scenario 1 (S1) allowed 60K model runs and provided two target statistics, whereas scenario 2 (S2) allowed 75K model runs and provided three target statistics. For both scenarios, we obtained reference posteriors against which we compare all other methods by running Rejection ABC for 5M parameter combinations, retaining the 0.1% best. We assessed performance by applying a 2D-grid to all posterior densities and quantifying the percentage overlap with the reference posterior. We considered basic and adaptive sampling calibration methods. Of the basic calibration methods, Bayesian calibration (Bc) Sampling Importance Resampling (S1: 34.8%, S2: 39.8%) outperformed Rejection Approximate Bayesian Computation (ABC) (S1: 2.3%, S2: 1.8%). Among the adaptive sampling methods, Bc Incremental Mixture Importance Sampling (S1: 72.7%, S2: 85.5%) outperformed sequential Monte Carlo ABC (AbcSmc) (S1: 53.9%, S2: 72.9%) and Sequential ABC (S1: 21.6%, S2: 62.7%). Basic methods led to sub-optimal calibration results. Methods using the surrogate Likelihood as a GOF outperformed methods using a distance measure. Adaptive sampling methods were more efficient compared to their basic counterparts and resulted in accurate posterior distributions. BcIMIS was the best performing method. When three rather than two target statistics were available, the difference in performance between the adaptive sampling methods was less pronounced. Although BcIMIS outperforms the other methods, limitations related to the target statistics and available computing infrastructure may warrant the choice of an alternative method. Author summary As mathematical models become more realistic, they tend to become more complex. Calibration, the process of tuning a model to better reproduce empirical data, can become dramatically more computationally intensive as model complexity increases. Researchers have responded by developing a range of more efficient, adaptive sampling calibration methods. However, the relative performance of these calibration methods remains unclear. To this end, we quantified the performance of five commonly used calibration methods. We found that adaptive sampling methods were more efficient compared to their basic counterparts and resulted in more accurate posterior distributions. We identified the best performing method, but caution that limitations related to the target statistics and available computing infrastructure may warrant the choice of one of the alternatives. Finally, we provide the code used to apply the calibration methods in our study as a primer to facilitate their application.
Open AccessCalibration of models to data: a comparison of methods
Complex models are often fitted to data using simulation-based calibration, a computationally challenging process. Several calibration methods to improve computational efficiency have been developed with no consensus on which methods perform best. We did a simulation study comparing the performance of 5 methods that differed in their Goodness-of-Fit (GOF) metrics and parameter search strategies. Posterior densities for two parameters of a simple Susceptible-Infectious-Recovered epidemic model were obtained for each calibration method under two scenarios. Scenario 1 (S1) allowed 60K model runs and provided two target statistics, whereas scenario 2 (S2) allowed 75K model runs and provided three target statistics. For both scenarios, we obtained reference posteriors against which we compare all other methods by running Rejection ABC for 5M parameter combinations, retaining the 0.1% best. We assessed performance by applying a 2D-grid to all posterior densities and quantifying the percentage overlap with the reference posterior. We considered basic and adaptive sampling calibration methods. Of the basic calibration methods, Bayesian calibration (Bc) Sampling Importance Resampling (S1: 34.8%, S2: 39.8%) outperformed Rejection Approximate Bayesian Computation (ABC) (S1: 2.3%, S2: 1.8%). Among the adaptive sampling methods, Bc Incremental Mixture Importance Sampling (S1: 72.7%, S2: 85.5%) outperformed sequential Monte Carlo ABC (AbcSmc) (S1: 53.9%, S2: 72.9%) and Sequential ABC (S1: 21.6%, S2: 62.7%). Basic methods led to sub-optimal calibration results. Methods using the surrogate Likelihood as a GOF outperformed methods using a distance measure. Adaptive sampling methods were more efficient compared to their basic counterparts and resulted in accurate posterior distributions. BcIMIS was the best performing method. When three rather than two target statistics were available, the difference in performance between the adaptive sampling methods was less pronounced. Although BcIMIS outperforms the other methods, limitations related to the target statistics and available computing infrastructure may warrant the choice of an alternative method. Author summary As mathematical models become more realistic, they tend to become more complex. Calibration, the process of tuning a model to better reproduce empirical data, can become dramatically more computationally intensive as model complexity increases. Researchers have responded by developing a range of more efficient, adaptive sampling calibration methods. However, the relative performance of these calibration methods remains unclear. To this end, we quantified the performance of five commonly used calibration methods. We found that adaptive sampling methods were more efficient compared to their basic counterparts and resulted in more accurate posterior distributions. We identified the best performing method, but caution that limitations related to the target statistics and available computing infrastructure may warrant the choice of one of the alternatives. Finally, we provide the code used to apply the calibration methods in our study as a primer to facilitate their application.
- Open Access
- 11 May 2020
- Published by Public Library of Science (PLoS) in PLOS Computational Biology
- Vol. 16 (5) , e1007893
Calibration of individual-based models to epidemiological data: A systematic review
Individual-based models (IBMs) informing public health policy should be calibrated to data and provide estimates of uncertainty. Two main components of model-calibration methods are the parameter-search strategy and the goodness-of-fit (GOF) measure; many options exist for each of these. This review provides an overview of calibration methods used in IBMs modelling infectious disease spread. We identified articles on PubMed employing simulation-based methods to calibrate IBMs informing public health policy in HIV, tuberculosis, and malaria epidemiology published between 1 January 2013 and 31 December 2018. Articles were included if models stored individual-specific information, and calibration involved comparing model output to population-level targets. We extracted information on parameter-search strategies, GOF measures, and model validation. The PubMed search identified 653 candidate articles, of which 84 met the review criteria. Of the included articles, 40 (48%) combined a quantitative GOF measure with an algorithmic parameter-search strategy–either an optimisation algorithm (14/40) or a sampling algorithm (26/40). These 40 articles varied widely in their choices of parameter-search strategies and GOF measures. For the remaining 44 (52%) articles, the parameter-search strategy could either not be identified (32/44) or was described as an informal, non-reproducible method (12/44). Of these 44 articles, the majority (25/44) were unclear about the GOF measure used; of the rest, only five quantitatively evaluated GOF. Only a minority of the included articles, 14 (17%) provided a rationale for their choice of model-calibration method. Model validation was reported in 31 (37%) articles. Reporting on calibration methods is far from optimal in epidemiological modelling studies of HIV, malaria and TB transmission dynamics. The adoption of better documented, algorithmic calibration methods could improve both reproducibility and the quality of inference in model-based epidemiology. There is a need for research comparing the performance of calibration methods to inform decisions about the parameter-search strategies and GOF measures. Calibration—that is, “fitting” the model to data—is a crucial part of using mathematical models to better forecast and control the population-level spread of infectious diseases. Evidence that the mathematical model is well-calibrated improves confidence that the model provides a realistic picture of the consequences of health policy decisions. To make informed decisions, Policymakers need information about uncertainty: i.e., what is the range of likely outcomes (rather than just a single prediction). Thus, modellers should also strive to provide accurate measurements of uncertainty, both for their model parameters and for their predictions. This systematic review provides an overview of the methods used to calibrate individual-based models (IBMs) of the spread of HIV, malaria, and tuberculosis. We found that less than half of the reviewed articles used reproducible, non-subjective calibration methods. For the remaining articles, the method could either not be identified or was described as an informal, non-reproducible method. Only one-third of the articles obtained estimates of parameter uncertainty. We conclude that the adoption of better-documented, algorithmic calibration methods could improve both reproducibility and the quality of inference in model-based epidemiology.
- 11 May 2020
- Published by Public Library of Science (PLoS) in PLOS Computational Biology
- Vol. 16 (5) , e1007893
Open AccessCalibration of individual-based models to epidemiological data: A systematic review
Individual-based models (IBMs) informing public health policy should be calibrated to data and provide estimates of uncertainty. Two main components of model-calibration methods are the parameter-search strategy and the goodness-of-fit (GOF) measure; many options exist for each of these. This review provides an overview of calibration methods used in IBMs modelling infectious disease spread. We identified articles on PubMed employing simulation-based methods to calibrate IBMs informing public health policy in HIV, tuberculosis, and malaria epidemiology published between 1 January 2013 and 31 December 2018. Articles were included if models stored individual-specific information, and calibration involved comparing model output to population-level targets. We extracted information on parameter-search strategies, GOF measures, and model validation. The PubMed search identified 653 candidate articles, of which 84 met the review criteria. Of the included articles, 40 (48%) combined a quantitative GOF measure with an algorithmic parameter-search strategy–either an optimisation algorithm (14/40) or a sampling algorithm (26/40). These 40 articles varied widely in their choices of parameter-search strategies and GOF measures. For the remaining 44 (52%) articles, the parameter-search strategy could either not be identified (32/44) or was described as an informal, non-reproducible method (12/44). Of these 44 articles, the majority (25/44) were unclear about the GOF measure used; of the rest, only five quantitatively evaluated GOF. Only a minority of the included articles, 14 (17%) provided a rationale for their choice of model-calibration method. Model validation was reported in 31 (37%) articles. Reporting on calibration methods is far from optimal in epidemiological modelling studies of HIV, malaria and TB transmission dynamics. The adoption of better documented, algorithmic calibration methods could improve both reproducibility and the quality of inference in model-based epidemiology. There is a need for research comparing the performance of calibration methods to inform decisions about the parameter-search strategies and GOF measures. Calibration—that is, “fitting” the model to data—is a crucial part of using mathematical models to better forecast and control the population-level spread of infectious diseases. Evidence that the mathematical model is well-calibrated improves confidence that the model provides a realistic picture of the consequences of health policy decisions. To make informed decisions, Policymakers need information about uncertainty: i.e., what is the range of likely outcomes (rather than just a single prediction). Thus, modellers should also strive to provide accurate measurements of uncertainty, both for their model parameters and for their predictions. This systematic review provides an overview of the methods used to calibrate individual-based models (IBMs) of the spread of HIV, malaria, and tuberculosis. We found that less than half of the reviewed articles used reproducible, non-subjective calibration methods. For the remaining articles, the method could either not be identified or was described as an informal, non-reproducible method. Only one-third of the articles obtained estimates of parameter uncertainty. We conclude that the adoption of better-documented, algorithmic calibration methods could improve both reproducibility and the quality of inference in model-based epidemiology.
- Open Access
- 17 December 2019
- Published by Springer Nature in Scientific Reports
- Vol. 9 (1) , 1-13
SimpactCyan 1.0: An Open-source Simulator for Individual-Based Models in HIV Epidemiology with R and Python Interfaces
SimpactCyan is an open-source simulator for individual-based models in HIV epidemiology. Its core algorithm is written in C++ for computational efficiency, while the R and Python interfaces aim to make the tool accessible to the fast-growing community of R and Python users. Transmission, treatment and prevention of HIV infections in dynamic sexual networks are simulated by discrete events. A generic “intervention” event allows model parameters to be changed over time, and can be used to model medical and behavioural HIV prevention programmes. First, we describe a more efficient variant of the modified Next Reaction Method that drives our continuous-time simulator. Next, we outline key built-in features and assumptions of individual-based models formulated in SimpactCyan, and provide code snippets for how to formulate, execute and analyse models in SimpactCyan through its R and Python interfaces. Lastly, we give two examples of applications in HIV epidemiology: the first demonstrates how the software can be used to estimate the impact of progressive changes to the eligibility criteria for HIV treatment on HIV incidence. The second example illustrates the use of SimpactCyan as a data-generating tool for assessing the performance of a phylodynamic inference framework.
- 17 December 2019
- Published by Springer Nature in Scientific Reports
- Vol. 9 (1) , 1-13
Open AccessSimpactCyan 1.0: An Open-source Simulator for Individual-Based Models in HIV Epidemiology with R and Python Interfaces
SimpactCyan is an open-source simulator for individual-based models in HIV epidemiology. Its core algorithm is written in C++ for computational efficiency, while the R and Python interfaces aim to make the tool accessible to the fast-growing community of R and Python users. Transmission, treatment and prevention of HIV infections in dynamic sexual networks are simulated by discrete events. A generic “intervention” event allows model parameters to be changed over time, and can be used to model medical and behavioural HIV prevention programmes. First, we describe a more efficient variant of the modified Next Reaction Method that drives our continuous-time simulator. Next, we outline key built-in features and assumptions of individual-based models formulated in SimpactCyan, and provide code snippets for how to formulate, execute and analyse models in SimpactCyan through its R and Python interfaces. Lastly, we give two examples of applications in HIV epidemiology: the first demonstrates how the software can be used to estimate the impact of progressive changes to the eligibility criteria for HIV treatment on HIV incidence. The second example illustrates the use of SimpactCyan as a data-generating tool for assessing the performance of a phylodynamic inference framework.
- Open Access
Influence of sexual risk behaviour and STI co-infection dynamics on the evolution of HIV set point viral load in MSM
HIV viral load (VL) is an important predictor of HIV progression and transmission. Anti-retroviral therapy (ART) has been reported to reduce HIV transmission by lowering VL. However, apart from this beneficial effect, increased levels of population mean set-point viral load (SPVL), an estimator for HIV virulence, have been observed in men who have sex with men (MSM) in the decade following the introduction of ART in the Netherlands. Several studies have been devoted to explain these counter-intuitive trends in SPVL. However, to our knowledge, none of these studies has investigated an explanation in which it arises as the result of a sexually transmitted infection (STI) co-factor in detail. In this study, we adapted an event-based, individual-based model to investigate how STI co-infection and sexual risk behaviour affect the evolution of HIV SPVL in MSM before and after the introduction of ART. The results suggest that sexual risk behaviour has an effect on SPVL and indicate that more data are needed to test the effect of STI co-factors on SPVL. Furthermore, the observed trends in SPVL cannot be explained by sexual risk behaviour and STI co-factors only. We recommend to develop mathematical models including also factors related to viral evolution as reported earlier in the literature. However, this requires more complex models, and the collection of more data for parameter estimation than what is currently available.
Open AccessInfluence of sexual risk behaviour and STI co-infection dynamics on the evolution of HIV set point viral load in MSM
HIV viral load (VL) is an important predictor of HIV progression and transmission. Anti-retroviral therapy (ART) has been reported to reduce HIV transmission by lowering VL. However, apart from this beneficial effect, increased levels of population mean set-point viral load (SPVL), an estimator for HIV virulence, have been observed in men who have sex with men (MSM) in the decade following the introduction of ART in the Netherlands. Several studies have been devoted to explain these counter-intuitive trends in SPVL. However, to our knowledge, none of these studies has investigated an explanation in which it arises as the result of a sexually transmitted infection (STI) co-factor in detail. In this study, we adapted an event-based, individual-based model to investigate how STI co-infection and sexual risk behaviour affect the evolution of HIV SPVL in MSM before and after the introduction of ART. The results suggest that sexual risk behaviour has an effect on SPVL and indicate that more data are needed to test the effect of STI co-factors on SPVL. Furthermore, the observed trends in SPVL cannot be explained by sexual risk behaviour and STI co-factors only. We recommend to develop mathematical models including also factors related to viral evolution as reported earlier in the literature. However, this requires more complex models, and the collection of more data for parameter estimation than what is currently available.
- Open Access
Comparison of two individual-based model simulators for HIV epidemiology in a population with HSV-2 using as case study Yaoundé-Cameroon, 1980-2005
Model comparisons have been widely used to guide intervention strategies to control infectious diseases. Agreement between different models is crucial for providing robust evidence for policy-makers because differences in model properties can influence their predictions. In this study, we compared models implemented by two individual-based model simulators for HIV epidemiology in a population with Herpes simplex virus type 2 (HSV-2). For each model simulator, we constructed four models, starting from a simplified basic model and stepwise including more model complexity. For the resulting eight models, the predictions of the impact of behavioural interventions on the HIV epidemic in Yaoundé (Cameroon) were compared. The results show that differences in model assumptions and model complexity can influence the size of the predicted impact of the intervention, as well as the predicted qualitative behaviour of the HIV epidemic after the intervention. Moreover, two models that agree in their predictions of the HIV epidemic in the absence of intervention can have different outputs when predicting the impact of interventions. Without additional data, it is impossible to determine which of these two models is the most reliable. These findings highlight the importance of making more data available for the calibration and validation of epidemiological models.
Open AccessComparison of two individual-based model simulators for HIV epidemiology in a population with HSV-2 using as case study Yaoundé-Cameroon, 1980-2005
Model comparisons have been widely used to guide intervention strategies to control infectious diseases. Agreement between different models is crucial for providing robust evidence for policy-makers because differences in model properties can influence their predictions. In this study, we compared models implemented by two individual-based model simulators for HIV epidemiology in a population with Herpes simplex virus type 2 (HSV-2). For each model simulator, we constructed four models, starting from a simplified basic model and stepwise including more model complexity. For the resulting eight models, the predictions of the impact of behavioural interventions on the HIV epidemic in Yaoundé (Cameroon) were compared. The results show that differences in model assumptions and model complexity can influence the size of the predicted impact of the intervention, as well as the predicted qualitative behaviour of the HIV epidemic after the intervention. Moreover, two models that agree in their predictions of the HIV epidemic in the absence of intervention can have different outputs when predicting the impact of interventions. Without additional data, it is impossible to determine which of these two models is the most reliable. These findings highlight the importance of making more data available for the calibration and validation of epidemiological models.