Elliot D. Hill
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Elliot D. Hill has a total of 49 co-authors. Here are the top co-authors by number of shared publications:
- Matthew M. Engelhard
Department of Biostatistics and Bioinformatics Duke University School of Medicine Durham North Carolina USA
4 shared publications - Geraldine Dawson
Duke Center for Autism and Brain Development, Duke University, Durham, North Carolina, USA
3 shared publications Matthew Engelhard
3 shared publications- Avshalom Caspi
Department of Psychology and Neuroscience, Duke University
2 shared publications - Benjamin Alan Goldstein
Duke University School of Medicine
2 shared publications Elizabeth Raffanello
Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine, Durham, USA
2 shared publications
Latest Publications
- Open Access
Improving the Performance of Models Trained on Small EHR-Derived Samples by Leveraging External Data with Continual Learning Methods
The performance of an EHR-based deep learning model trained on a small sample can be improved if more data is collected. Instead of collecting more data, the model can be trained on additional data from an analogous external source. However, this risks the model learning patterns in the external data that do not generalize to the target sample. Furthermore, data use agreements often prohibit combining datasets with medical records of different sources. We consider utilizing pre-existing methods in continual learning, namely the elastic weight consolidation (EWC) loss function and variational continual learning (VCL), both of which are regularization-based methods that we use to borrow external data and incorporate parameters from a model on external data into local model training. To investigate the utility of this modeling framework, we consider two binary classification tasks: (1) predicting which children will be diagnosed with autism spectrum disorder (ASD) from medical claims up to 18 months, and (2) predicting which patients with end-stage renal disease (ESRD) will be re-hospitalized within 30 days. Target datasets were derived from Duke University's EHR warehouse, and external datasets were sourced from either NC Medicaid claims for the ASD prediction task, or the United States Renal Data System (USRDS) for the rehospitalization prediction task. For both of these tasks, borrowing models - using either the EWC loss function or VCL - performed similarly to that of a model trained only on the full external data, when the sample size of target data used to train the model was small. That is, while a model that does not borrow using our methods performed poorly in low data regimes, the borrowing model instead matched the performance of a model trained on external data even when sample size of target data was small. In addition, an analysis of model predictions showed that models with small samples are better calibrated and more functionally similar to a model trained only on external data when the sample size is small.
Open AccessImproving the Performance of Models Trained on Small EHR-Derived Samples by Leveraging External Data with Continual Learning Methods
The performance of an EHR-based deep learning model trained on a small sample can be improved if more data is collected. Instead of collecting more data, the model can be trained on additional data from an analogous external source. However, this risks the model learning patterns in the external data that do not generalize to the target sample. Furthermore, data use agreements often prohibit combining datasets with medical records of different sources. We consider utilizing pre-existing methods in continual learning, namely the elastic weight consolidation (EWC) loss function and variational continual learning (VCL), both of which are regularization-based methods that we use to borrow external data and incorporate parameters from a model on external data into local model training. To investigate the utility of this modeling framework, we consider two binary classification tasks: (1) predicting which children will be diagnosed with autism spectrum disorder (ASD) from medical claims up to 18 months, and (2) predicting which patients with end-stage renal disease (ESRD) will be re-hospitalized within 30 days. Target datasets were derived from Duke University's EHR warehouse, and external datasets were sourced from either NC Medicaid claims for the ASD prediction task, or the United States Renal Data System (USRDS) for the rehospitalization prediction task. For both of these tasks, borrowing models - using either the EWC loss function or VCL - performed similarly to that of a model trained only on the full external data, when the sample size of target data used to train the model was small. That is, while a model that does not borrow using our methods performed poorly in low data regimes, the borrowing model instead matched the performance of a model trained on external data even when sample size of target data was small. In addition, an analysis of model predictions showed that models with small samples are better calibrated and more functionally similar to a model trained only on external data when the sample size is small.
- 27 April 2026
- Published by Springer Nature in Nature Mental Health
- Vol. 4 (5) , 806-815
Early attention deficit hyperactivity disorder prediction from longitudinal electronic health records
- 27 April 2026
- Published by Springer Nature in Nature Mental Health
- Vol. 4 (5) , 806-815
Early attention deficit hyperactivity disorder prediction from longitudinal electronic health records
- 1 April 2026
- Published by Wolters Kluwer Health in Epidemiology
- Vol. 37 (4) , 553-558
Validity of Military Service as Reported on U.S. Death Certificates
Background: Although death certificates are widely used to study mortality in the general population, their use to study mortality among individuals with military service has been limited in part by a lack of information on the validity of military service reported on death certificates. Our objective was to estimate bias parameters for the misclassification of military service to facilitate quantitative bias analysis in studies of mortality in military populations. Methods: We included 2014–2021 death certificates from decedents aged 18–64 years who resided and died in Alabama, Michigan, Minnesota, Montana, or Oregon. Death certificates were linked to military service records by the U.S. Defense Manpower Data Center using Social Security numbers. Military service was defined as any service in the Regular, Reserve, or National Guard components of the U.S. military in Defense Manpower Data Center records. We estimated sensitivity, specificity, and predictive values for military service reported on death certificates stratified by demographics, military characteristics, and manner of death. Results: Among the 467,075 death certificates included, 9.3% indicated military service; however, military records showed that 10.9% had served (sensitivity 81.5%, 95% confidence interval [CI] = 81.2%, 81.8%; specificity 99.5%, 95% CI = 99.5%, 99.5%). Sensitivity was lower among female (72.3%) compared with male (82.2%) servicemembers and was particularly low for those who had never served on active duty (63.5%). Conclusions: Military service was underreported on death certificates, especially for female servicemembers and those without active-duty service. The bias parameters we estimated can be used to account for this misclassification when analyzing death certificates.
- 1 April 2026
- Published by Wolters Kluwer Health in Epidemiology
- Vol. 37 (4) , 553-558
Validity of Military Service as Reported on U.S. Death Certificates
Background: Although death certificates are widely used to study mortality in the general population, their use to study mortality among individuals with military service has been limited in part by a lack of information on the validity of military service reported on death certificates. Our objective was to estimate bias parameters for the misclassification of military service to facilitate quantitative bias analysis in studies of mortality in military populations. Methods: We included 2014–2021 death certificates from decedents aged 18–64 years who resided and died in Alabama, Michigan, Minnesota, Montana, or Oregon. Death certificates were linked to military service records by the U.S. Defense Manpower Data Center using Social Security numbers. Military service was defined as any service in the Regular, Reserve, or National Guard components of the U.S. military in Defense Manpower Data Center records. We estimated sensitivity, specificity, and predictive values for military service reported on death certificates stratified by demographics, military characteristics, and manner of death. Results: Among the 467,075 death certificates included, 9.3% indicated military service; however, military records showed that 10.9% had served (sensitivity 81.5%, 95% confidence interval [CI] = 81.2%, 81.8%; specificity 99.5%, 95% CI = 99.5%, 99.5%). Sensitivity was lower among female (72.3%) compared with male (82.2%) servicemembers and was particularly low for those who had never served on active duty (63.5%). Conclusions: Military service was underreported on death certificates, especially for female servicemembers and those without active-duty service. The bias parameters we estimated can be used to account for this misclassification when analyzing death certificates.
- 19 May 2025
- Published by Springer Nature in Nature Medicine
- Vol. 31 (7) , 2453
Author Correction: Prediction of mental health risk in adolescents
- 19 May 2025
- Published by Springer Nature in Nature Medicine
- Vol. 31 (7) , 2453
Author Correction: Prediction of mental health risk in adolescents
- Open Access
- 27 March 2025
- Published by JMIR Publications Inc. in JMIR AI
- Vol. 4 (1) , e62985
Limitations of Binary Classification for Long-Horizon Diagnosis Prediction and Advantages of a Discrete-Time Time-to-Event Approach: Empirical Analysis
Background: A major challenge in using electronic health records (EHR) is the inconsistency of patient follow-up, resulting in right-censored outcomes. This becomes particularly problematic in long-horizon event predictions, such as autism and attention-deficit/hyperactivity disorder (ADHD) diagnoses, where a significant number of patients are lost to follow-up before the outcome can be observed. Consequently, fully supervised methods such as binary classification (BC), which are trained to predict observed diagnoses, are substantially affected by the probability of sufficient follow-up, leading to biased results. Objective: This empirical analysis aims to characterize BC’s inherent limitations for long-horizon diagnosis prediction from EHR; and quantify the benefits of a specific time-to-event (TTE) approach, the discrete-time neural network (DTNN). Methods: Records within the Duke University Health System EHR were analyzed, extracting features such as ICD-10 (International Classification of Diseases, Tenth Revision) diagnosis codes, medications, laboratories, and procedures. We compared a DTNN to 3 BC approaches and a deep Cox proportional hazards model across 4 clinical conditions to examine distributional patterns across various subgroups. Time-varying area under the receiving operating characteristic curve (AUCt) and time-varying average precision (APt) were our primary evaluation metrics. Results: TTE models consistently had comparable or higher AUCt and APt than BC for all conditions. At clinically relevant operating time points, the area under the receiving operating characteristic curve (AUC) values for DTNNYOB≤2020 (year-of-birth) and DCPHYOB≤2020 (deep Cox proportional hazard) were 0.70 (95% CI 0.66‐0.77) and 0.72 (95% CI 0.66‐0.78) at t=5 for autism, 0.72 (95% CI 0.65‐0.76) and 0.68 (95% CI 0.62‐0.74) at t=7 for ADHD, 0.72 (95% CI 0.70‐0.75) and 0.71 (95% CI 0.69‐0.74) at t=1 for recurrent otitis media, and 0.74 (95% CI 0.68‐0.82) and 0.71 (95% CI 0.63‐0.77) at t=1 for food allergy, compared to 0.6 (95% CI 0.55‐0.66), 0.47 (95% CI 0.40‐0.54), 0.73 (95% CI 0.70‐0.75), and 0.77 (95% CI 0.71‐0.82) for BCYOB≤2020, respectively. The probabilities predicted by BC models were positively correlated with censoring times, particularly for autism and ADHD prediction. Filtering strategies based on YOB or length of follow-up only partially corrected these biases. In subgroup analyses, only DTNN predicted diagnosis probabilities that accurately reflect actual clinical prevalence and temporal trends. Conclusions: BC models substantially underpredicted diagnosis likelihood and inappropriately assigned lower probability scores to individuals with earlier censoring. Common filtering strategies did not adequately address this limitation. TTE approaches, particularly DTNN, effectively mitigated bias from the censoring distribution, resulting in superior discrimination and calibration performance and more accurate prediction of clinical prevalence. Machine learning practitioners should recognize the limitations of BC for long-horizon diagnosis prediction and adopt TTE approaches. The DTNN in particular is well-suited to mitigate the effects of right-censoring and maximize prediction performance in this setting.
- 27 March 2025
- Published by JMIR Publications Inc. in JMIR AI
- Vol. 4 (1) , e62985
Open AccessLimitations of Binary Classification for Long-Horizon Diagnosis Prediction and Advantages of a Discrete-Time Time-to-Event Approach: Empirical Analysis
Background: A major challenge in using electronic health records (EHR) is the inconsistency of patient follow-up, resulting in right-censored outcomes. This becomes particularly problematic in long-horizon event predictions, such as autism and attention-deficit/hyperactivity disorder (ADHD) diagnoses, where a significant number of patients are lost to follow-up before the outcome can be observed. Consequently, fully supervised methods such as binary classification (BC), which are trained to predict observed diagnoses, are substantially affected by the probability of sufficient follow-up, leading to biased results. Objective: This empirical analysis aims to characterize BC’s inherent limitations for long-horizon diagnosis prediction from EHR; and quantify the benefits of a specific time-to-event (TTE) approach, the discrete-time neural network (DTNN). Methods: Records within the Duke University Health System EHR were analyzed, extracting features such as ICD-10 (International Classification of Diseases, Tenth Revision) diagnosis codes, medications, laboratories, and procedures. We compared a DTNN to 3 BC approaches and a deep Cox proportional hazards model across 4 clinical conditions to examine distributional patterns across various subgroups. Time-varying area under the receiving operating characteristic curve (AUCt) and time-varying average precision (APt) were our primary evaluation metrics. Results: TTE models consistently had comparable or higher AUCt and APt than BC for all conditions. At clinically relevant operating time points, the area under the receiving operating characteristic curve (AUC) values for DTNNYOB≤2020 (year-of-birth) and DCPHYOB≤2020 (deep Cox proportional hazard) were 0.70 (95% CI 0.66‐0.77) and 0.72 (95% CI 0.66‐0.78) at t=5 for autism, 0.72 (95% CI 0.65‐0.76) and 0.68 (95% CI 0.62‐0.74) at t=7 for ADHD, 0.72 (95% CI 0.70‐0.75) and 0.71 (95% CI 0.69‐0.74) at t=1 for recurrent otitis media, and 0.74 (95% CI 0.68‐0.82) and 0.71 (95% CI 0.63‐0.77) at t=1 for food allergy, compared to 0.6 (95% CI 0.55‐0.66), 0.47 (95% CI 0.40‐0.54), 0.73 (95% CI 0.70‐0.75), and 0.77 (95% CI 0.71‐0.82) for BCYOB≤2020, respectively. The probabilities predicted by BC models were positively correlated with censoring times, particularly for autism and ADHD prediction. Filtering strategies based on YOB or length of follow-up only partially corrected these biases. In subgroup analyses, only DTNN predicted diagnosis probabilities that accurately reflect actual clinical prevalence and temporal trends. Conclusions: BC models substantially underpredicted diagnosis likelihood and inappropriately assigned lower probability scores to individuals with earlier censoring. Common filtering strategies did not adequately address this limitation. TTE approaches, particularly DTNN, effectively mitigated bias from the censoring distribution, resulting in superior discrimination and calibration performance and more accurate prediction of clinical prevalence. Machine learning practitioners should recognize the limitations of BC for long-horizon diagnosis prediction and adopt TTE approaches. The DTNN in particular is well-suited to mitigate the effects of right-censoring and maximize prediction performance in this setting.
- Top Cited Papers
- 5 March 2025
- Published by Springer Nature in Nature Medicine
- Vol. 31 (6) , 1840-1846
Prediction of mental health risk in adolescents
Prospective prediction of mental health risk in adolescence can facilitate early preventive interventions. Here, using psychosocial questionnaires and neuroimaging measures from over 11,000 children in the Adolescent Brain and Cognitive Development Study, we trained neural network models to stratify general psychopathology risk. The model trained on current symptoms accurately predicted which participants would convert into the highest psychiatric illness risk group in the following year (area under the receiver operating characteristic curve = 0.84). The model trained solely on potential etiologies or disease mechanisms achieved an area under the receiver operating characteristic curve of 0.75 without relying on the child’s current symptom burden. Sleep disturbances emerged as the most influential predictor of high-risk status, surpassing adverse childhood experiences and family mental health history. Including neuroimaging measures did not enhance predictive performance. These findings suggest that artificial intelligence models trained on readily available psychosocial questionnaires can effectively predict future psychiatric risk while highlighting potential targets for intervention. This is a promising step toward artificial intelligence-based mental health screening for clinical decision support systems.
- 5 March 2025
- Published by Springer Nature in Nature Medicine
- Vol. 31 (6) , 1840-1846
Top Cited PapersPrediction of mental health risk in adolescents
Prospective prediction of mental health risk in adolescence can facilitate early preventive interventions. Here, using psychosocial questionnaires and neuroimaging measures from over 11,000 children in the Adolescent Brain and Cognitive Development Study, we trained neural network models to stratify general psychopathology risk. The model trained on current symptoms accurately predicted which participants would convert into the highest psychiatric illness risk group in the following year (area under the receiver operating characteristic curve = 0.84). The model trained solely on potential etiologies or disease mechanisms achieved an area under the receiver operating characteristic curve of 0.75 without relying on the child’s current symptom burden. Sleep disturbances emerged as the most influential predictor of high-risk status, surpassing adverse childhood experiences and family mental health history. Including neuroimaging measures did not enhance predictive performance. These findings suggest that artificial intelligence models trained on readily available psychosocial questionnaires can effectively predict future psychiatric risk while highlighting potential targets for intervention. This is a promising step toward artificial intelligence-based mental health screening for clinical decision support systems.
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL) in Proceedings of the conference. Association for Computational Linguistics. Meeting
- Vol. 2025, 30263-30283
IRIS: Interpretable Retrieval-Augmented Classification for Long Interspersed Document Sequences
Fengnan Li, Elliot D. Hill, Jiang Shu, Jiaxin Gao, Matthew M. Engelhard. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL) in Proceedings of the conference. Association for Computational Linguistics. Meeting
- Vol. 2025, 30263-30283
Open AccessIRIS: Interpretable Retrieval-Augmented Classification for Long Interspersed Document Sequences
Fengnan Li, Elliot D. Hill, Jiang Shu, Jiaxin Gao, Matthew M. Engelhard. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
- 30 August 2024
- Published by Elsevier in Journal of Biomedical Informatics
- Vol. 157, 104711
A conditional multi-label model to improve prediction of a rare outcome: An illustration predicting autism diagnosis
- 30 August 2024
- Published by Elsevier in Journal of Biomedical Informatics
- Vol. 157, 104711
A conditional multi-label model to improve prediction of a rare outcome: An illustration predicting autism diagnosis
- Open Access
ct2vl: A Robust Public Resource for Converting SARS-CoV-2 Ct Values to Viral Loads
The amount of SARS-CoV-2 in a sample is often measured using Ct values. However, the same Ct value may correspond to different viral loads on different platforms and assays, making them difficult to compare from study to study. To address this problem, we developed ct2vl, a Python package that converts Ct values to viral loads for any RT-qPCR assay/platform. The method is novel in that it is based on determining the maximum PCR replication efficiency, as opposed to fitting a sigmoid (S-shaped) curve relating signal to cycle number. We calibrated ct2vl on two FDA-approved platforms and validated its performance using reference-standard material, including sensitivity analysis. We found that ct2vl-predicted viral loads were highly accurate across five orders of magnitude, with 1.6-fold median error (for comparison, viral loads in clinical samples vary over 10 orders of magnitude). The package has 100% test coverage. We describe installation and usage both from the Unix command-line and from interactive Python environments. ct2vl is freely available via the Python Package Index (PyPI). It facilitates conversion of Ct values to viral loads for clinical investigators, basic researchers, and test developers for any RT-qPCR platform. It thus facilitates comparison among the many quantitative studies of SARS-CoV-2 by helping render observations in a natural, universal unit of measure.
Open Accessct2vl: A Robust Public Resource for Converting SARS-CoV-2 Ct Values to Viral Loads
The amount of SARS-CoV-2 in a sample is often measured using Ct values. However, the same Ct value may correspond to different viral loads on different platforms and assays, making them difficult to compare from study to study. To address this problem, we developed ct2vl, a Python package that converts Ct values to viral loads for any RT-qPCR assay/platform. The method is novel in that it is based on determining the maximum PCR replication efficiency, as opposed to fitting a sigmoid (S-shaped) curve relating signal to cycle number. We calibrated ct2vl on two FDA-approved platforms and validated its performance using reference-standard material, including sensitivity analysis. We found that ct2vl-predicted viral loads were highly accurate across five orders of magnitude, with 1.6-fold median error (for comparison, viral loads in clinical samples vary over 10 orders of magnitude). The package has 100% test coverage. We describe installation and usage both from the Unix command-line and from interactive Python environments. ct2vl is freely available via the Python Package Index (PyPI). It facilitates conversion of Ct values to viral loads for clinical investigators, basic researchers, and test developers for any RT-qPCR platform. It thus facilitates comparison among the many quantitative studies of SARS-CoV-2 by helping render observations in a natural, universal unit of measure.
- Open Access
Scaling Monte-Carlo-Based Inference on Antibody and TCR Repertoires
Previously, it has been shown that maximum-entropy models of immune-repertoire sequence can be used to determine a person's vaccination status. However, this approach has the drawback of requiring a computationally intensive method to compute each model's partition function ($Z$), the normalization constant required for calculating the probability that the model will generate a given sequence. Specifically, the method required generating approximately $10^{10}$ sequences via Monte-Carlo simulations for each model. This is impractical for large numbers of models. Here we propose an alternative method that requires estimating $Z$ this way for only a few models: it then uses these expensive estimates to estimate $Z$ more efficiently for the remaining models. We demonstrate that this new method enables the generation of accurate estimates for 27 models using only three expensive estimates, thereby reducing the computational cost by an order of magnitude. Importantly, this gain in efficiency is achieved with only minimal impact on classification accuracy. Thus, this new method enables larger-scale investigations in computational immunology and represents a useful contribution to energy-based modeling more generally.Open AccessScaling Monte-Carlo-Based Inference on Antibody and TCR Repertoires
Previously, it has been shown that maximum-entropy models of immune-repertoire sequence can be used to determine a person's vaccination status. However, this approach has the drawback of requiring a computationally intensive method to compute each model's partition function ($Z$), the normalization constant required for calculating the probability that the model will generate a given sequence. Specifically, the method required generating approximately $10^{10}$ sequences via Monte-Carlo simulations for each model. This is impractical for large numbers of models. Here we propose an alternative method that requires estimating $Z$ this way for only a few models: it then uses these expensive estimates to estimate $Z$ more efficiently for the remaining models. We demonstrate that this new method enables the generation of accurate estimates for 27 models using only three expensive estimates, thereby reducing the computational cost by an order of magnitude. Importantly, this gain in efficiency is achieved with only minimal impact on classification accuracy. Thus, this new method enables larger-scale investigations in computational immunology and represents a useful contribution to energy-based modeling more generally.