Raelene Goodwin
Research Subjects
Raelene Goodwin has a total of 5 co-authors. Here are the top co-authors by number of shared publications:
- Avijit Mitra
College of Information and Computer Sciences University of Massachusetts Amherst Amherst, MA United States
2 shared publications - Emily Druhl
Bedford Veterans Affairs Medical Center Center for Healthcare Organization and Implementation Research Bedford, MA United States
2 shared publications - Hong Yu
University of Massachusetts Lowell
2 shared publications - Jack Tsai
National Center on Homelessness Among Veterans, US Department of Veterans Affairs, Washington, DC, USA
2 shared publications - Matthew A Stimmel2 shared publications
Latest Publications
- Open Access
- 19 October 2026
- Published by JMIR Publications Inc. in JMIR AI
- Vol. 5, e104586
Large Language Model–Generated Synthetic Data for Parole and Probation Detection in Clinical Notes: Development and Evaluation Study
Background: Identifying individuals on parole or probation can help inform care pathways and ensure continuity of care in health care systems. However, this sensitive justice-related information is often absent from structured electronic health record fields and is instead documented in narrative clinical notes. Developing automated parole and probation detection (PPD) systems typically requires expert-annotated clinical data, but manual annotation is costly, time-consuming, and constrained by data sparsity and privacy concerns. Objective: This study aimed to develop and evaluate a synthetic-only training paradigm for PPD from clinical text that eliminates the need for expert-annotated training data while maintaining strong performance and cross-domain transfer to real-world data. Methods: We developed Synth-PPD, a synthetic dataset comprising 7579 labeled clinical notes generated using large language models (LLMs), redundancy filtering, and automated quality evaluation. Dataset quality was assessed through human review of 1517 synthetic notes by 2 annotators. The primary outcome was the macro–F1-score for PPD. We evaluated Synth-PPD across unsupervised and supervised learning settings using LLMs, traditional machine learning models, and transformer encoders. Performance was assessed on 293 expert-annotated real-world clinical notes from the US Department of Veterans Affairs. Results: Synth-PPD achieved 90.05% human-LLM alignment. In the unsupervised setting, Llama 3.3-70B-Instruct with chain-of-thought prompting and few-shot examples from Synth-PPD achieved a macro–F1-score of 87.22%, outperforming both the corresponding zero-shot model (80.10%) and the best rule-based baseline (77.55%; P<.001 for all comparisons). In the supervised setting, Llama 3.1-8B fine-tuned on Synth-PPD achieved a macro–F1-score of 81.43%, exceeding the best rule-based baseline (77.55%; P<.001) and achieving a numerically higher score than the substantially larger zero-shot Llama 3.3-70B-Instruct model (80.10%; P=.08). In contrast, traditional machine learning models and transformer encoders exhibited greater performance degradation under domain shift and derived limited benefit from increasing synthetic training data. Additional experiments showed that increasing synthetic training data improved cross-domain performance for LLM-based models but yielded comparatively smaller gains for traditional approaches. Conclusions: Synthetic data generated by LLMs can serve as a scalable, cost-effective, and privacy-preserving alternative to expert-annotated real-world clinical data for PPD from clinical text. The findings demonstrate successful cross-domain transfer to real-world US Department of Veterans Affairs clinical notes.
- 19 October 2026
- Published by JMIR Publications Inc. in JMIR AI
- Vol. 5, e104586
Open AccessLarge Language Model–Generated Synthetic Data for Parole and Probation Detection in Clinical Notes: Development and Evaluation Study
Background: Identifying individuals on parole or probation can help inform care pathways and ensure continuity of care in health care systems. However, this sensitive justice-related information is often absent from structured electronic health record fields and is instead documented in narrative clinical notes. Developing automated parole and probation detection (PPD) systems typically requires expert-annotated clinical data, but manual annotation is costly, time-consuming, and constrained by data sparsity and privacy concerns. Objective: This study aimed to develop and evaluate a synthetic-only training paradigm for PPD from clinical text that eliminates the need for expert-annotated training data while maintaining strong performance and cross-domain transfer to real-world data. Methods: We developed Synth-PPD, a synthetic dataset comprising 7579 labeled clinical notes generated using large language models (LLMs), redundancy filtering, and automated quality evaluation. Dataset quality was assessed through human review of 1517 synthetic notes by 2 annotators. The primary outcome was the macro–F1-score for PPD. We evaluated Synth-PPD across unsupervised and supervised learning settings using LLMs, traditional machine learning models, and transformer encoders. Performance was assessed on 293 expert-annotated real-world clinical notes from the US Department of Veterans Affairs. Results: Synth-PPD achieved 90.05% human-LLM alignment. In the unsupervised setting, Llama 3.3-70B-Instruct with chain-of-thought prompting and few-shot examples from Synth-PPD achieved a macro–F1-score of 87.22%, outperforming both the corresponding zero-shot model (80.10%) and the best rule-based baseline (77.55%; P<.001 for all comparisons). In the supervised setting, Llama 3.1-8B fine-tuned on Synth-PPD achieved a macro–F1-score of 81.43%, exceeding the best rule-based baseline (77.55%; P<.001) and achieving a numerically higher score than the substantially larger zero-shot Llama 3.3-70B-Instruct model (80.10%; P=.08). In contrast, traditional machine learning models and transformer encoders exhibited greater performance degradation under domain shift and derived limited benefit from increasing synthetic training data. Additional experiments showed that increasing synthetic training data improved cross-domain performance for LLM-based models but yielded comparatively smaller gains for traditional approaches. Conclusions: Synthetic data generated by LLMs can serve as a scalable, cost-effective, and privacy-preserving alternative to expert-annotated real-world clinical data for PPD from clinical text. The findings demonstrate successful cross-domain transfer to real-world US Department of Veterans Affairs clinical notes.
- Open Access
- 14 June 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Large Language Model–Generated Synthetic Data for Parole and Probation Detection in Clinical Notes: Development and Evaluation Study (Preprint)
BACKGROUND Identifying individuals on parole or probation can help inform care pathways and ensure continuity of care in health care systems. However, this sensitive justice-related information is often absent from structured electronic health record fields and is instead documented in narrative clinical notes. Developing automated parole and probation detection (PPD) systems typically requires expert-annotated clinical data, but manual annotation is costly, time-consuming, and constrained by data sparsity and privacy concerns. OBJECTIVE This study aimed to develop and evaluate a synthetic-only training paradigm for PPD from clinical text that eliminates the need for expert-annotated training data while maintaining strong performance and cross-domain transfer to real-world data. METHODS We developed Synth-PPD, a synthetic dataset comprising 7579 labeled clinical notes generated using large language models (LLMs), redundancy filtering, and automated quality evaluation. Dataset quality was assessed through human review of 1517 synthetic notes by 2 annotators. The primary outcome was the macro–F1-score for PPD. We evaluated Synth-PPD across unsupervised and supervised learning settings using LLMs, traditional machine learning models, and transformer encoders. Performance was assessed on 293 expert-annotated real-world clinical notes from the US Department of Veterans Affairs. RESULTS Synth-PPD achieved 90.05% human-LLM alignment. In the unsupervised setting, Llama 3.3-70B-Instruct with chain-of-thought prompting and few-shot examples from Synth-PPD achieved a macro–F1-score of 87.22%, outperforming both the corresponding zero-shot model (80.10%) and the best rule-based baseline (77.55%; P<.001 for all comparisons). In the supervised setting, Llama 3.1-8B fine-tuned on Synth-PPD achieved a macro–F1-score of 81.43%, exceeding the best rule-based baseline (77.55%; P<.001) and achieving a numerically higher score than the substantially larger zero-shot Llama 3.3-70B-Instruct model (80.10%; P=.08). In contrast, traditional machine learning models and transformer encoders exhibited greater performance degradation under domain shift and derived limited benefit from increasing synthetic training data. Additional experiments showed that increasing synthetic training data improved cross-domain performance for LLM-based models but yielded comparatively smaller gains for traditional approaches. CONCLUSIONS Synthetic data generated by LLMs can serve as a scalable, cost-effective, and privacy-preserving alternative to expert-annotated real-world clinical data for PPD from clinical text. The findings demonstrate successful cross-domain transfer to real-world US Department of Veterans Affairs clinical notes.
- 14 June 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessLarge Language Model–Generated Synthetic Data for Parole and Probation Detection in Clinical Notes: Development and Evaluation Study (Preprint)
BACKGROUND Identifying individuals on parole or probation can help inform care pathways and ensure continuity of care in health care systems. However, this sensitive justice-related information is often absent from structured electronic health record fields and is instead documented in narrative clinical notes. Developing automated parole and probation detection (PPD) systems typically requires expert-annotated clinical data, but manual annotation is costly, time-consuming, and constrained by data sparsity and privacy concerns. OBJECTIVE This study aimed to develop and evaluate a synthetic-only training paradigm for PPD from clinical text that eliminates the need for expert-annotated training data while maintaining strong performance and cross-domain transfer to real-world data. METHODS We developed Synth-PPD, a synthetic dataset comprising 7579 labeled clinical notes generated using large language models (LLMs), redundancy filtering, and automated quality evaluation. Dataset quality was assessed through human review of 1517 synthetic notes by 2 annotators. The primary outcome was the macro–F1-score for PPD. We evaluated Synth-PPD across unsupervised and supervised learning settings using LLMs, traditional machine learning models, and transformer encoders. Performance was assessed on 293 expert-annotated real-world clinical notes from the US Department of Veterans Affairs. RESULTS Synth-PPD achieved 90.05% human-LLM alignment. In the unsupervised setting, Llama 3.3-70B-Instruct with chain-of-thought prompting and few-shot examples from Synth-PPD achieved a macro–F1-score of 87.22%, outperforming both the corresponding zero-shot model (80.10%) and the best rule-based baseline (77.55%; P<.001 for all comparisons). In the supervised setting, Llama 3.1-8B fine-tuned on Synth-PPD achieved a macro–F1-score of 81.43%, exceeding the best rule-based baseline (77.55%; P<.001) and achieving a numerically higher score than the substantially larger zero-shot Llama 3.3-70B-Instruct model (80.10%; P=.08). In contrast, traditional machine learning models and transformer encoders exhibited greater performance degradation under domain shift and derived limited benefit from increasing synthetic training data. Additional experiments showed that increasing synthetic training data improved cross-domain performance for LLM-based models but yielded comparatively smaller gains for traditional approaches. CONCLUSIONS Synthetic data generated by LLMs can serve as a scalable, cost-effective, and privacy-preserving alternative to expert-annotated real-world clinical data for PPD from clinical text. The findings demonstrate successful cross-domain transfer to real-world US Department of Veterans Affairs clinical notes.