On the Fidelity versus Privacy and Utility Trade-Off of Synthetic Patient Data
Authors:
- Tim AdamsTATim Adams
- Colin BirkenbihlCBColin Birkenbihl
- Karen OtteKOKaren Otte
- Anatol-Fiete NaeherANAnatol-Fiete Naeher
- Fabian PrasserFPFabian Prasser
- Holger FroehlichHolger Froehlich
- Alzheimer’s Disease Neuroimaging InitiativeAIAlzheimer’s Disease Neuroimaging Initiative
Preprint
Abstract
Summary: The use of synthetic data is a widely discussed and promising solution for privacy-preserving medical research. Synthetic data may however not always be privacy preserving and can vary greatly in terms of data fidelity and utility. We systematically evaluate the trade-offs between privacy, fidelity, and utility across five synthetic data models and three patient-level datasets. We evaluate fidelity based on statistical similarity to the real data, utility on three machine learning use cases and privacy via membership inference, singling out, and attribute inference risks. Synthetic data without differential privacy (DP) maintained fidelity and utility without evident privacy breaches, whereas DP-enforced models significantly disrupted correlation structures. K-anonymity-based data sanitization, while preserving fidelity, introduced notable privacy risks. Our findings emphasize the need to advance methods that effectively balance privacy, fidelity, and utility in synthetic patient data generation. Highlights: Differential Privacy (DP) had a detrimental effect on feature correlations Models that did not implement DP showed good fidelity compared to real data Non-DP synthetic models showed no strong evidence of privacy breaches k-anonymization produced high fidelity data but showed notable privacy risksKeywords
All Related Versions
- Published version: iScience, 28 (5), 112382.
This publication has 47 references indexed in Scilit:
- Synthetic data as an enabler for machine learning applications in medicineiScience, 2022
- Using the Alzheimer's Disease Neuroimaging Initiative to improve early detection, diagnosis, and treatment of Alzheimer's diseaseAlzheimer's & Dementia, 2021
- Patterns of glioblastoma treatment and survival over a 16-years period: pooled data from the German Cancer RegistriesJournal of Cancer Research and Clinical Oncology, 2021
- Synthesis of diagnostic quality cancer pathology images by generative adversarial networksThe Journal of Pathology, 2020
- Synthetic data method to incorporate external information into a current studyCanadian Journal of Statistics, 2019
- Impact of the Alzheimer's Disease Neuroimaging Initiative, 2004 to 2014Alzheimer's & Dementia, 2015
- How Protective Are Synthetic Data?2008
- Differential Privacy2006
- Ways toward an early diagnosis in Alzheimer's disease: The Alzheimer's Disease Neuroimaging Initiative (ADNI)Alzheimer's & Dementia, 2005
- On the histogram as a density estimator:L 2 theoryProbability Theory and Related Fields, 1981