Enabling Synthetic Data adoption in regulated domains
Authors:
- 13 October 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-10
Abstract
The switch from a Model-Centric to a Data-Centric mindset is putting emphasis on data and its quality rather than algorithms, bringing forward new challenges. In particular, the sensitive nature of the information in highly regulated scenarios needs to be accounted for. Specific approaches to address the privacy issue have been developed, as Privacy Enhancing Technologies. However, they frequently cause loss of information, putting forward a crucial trade-off among data quality and privacy. A clever way to bypass such a conundrum relies on Synthetic Data: data obtained from a generative process, learning the real data properties. Both Academia and Industry realized the importance of evaluating synthetic data quality: without all-round reliable metrics, the innovative data generation task has no proper objective function to maximize. Despite that, the topic remains under-explored. For this reason, we systematically catalog the important traits of synthetic data quality and privacy, and devise a specific methodology to test them. The result is DAISYnt (aDoption of Artificial Intelligence SYnthesis): a comprehensive suite of advanced tests, which sets a de facto standard for synthetic data evaluation. As a practical use-case, a variety of generative algorithms have been trained on real-world Credit Bureau Data. The best model has been assessed, using DAISYnt on the different synthetic replicas. Further potential uses, among others, entail auditing and fine-tuning of generative models or ensuring high quality of a given synthetic dataset. From a prescriptive viewpoint, eventually, DAISYnt may pave the way to synthetic data adoption in highly regulated domains, ranging from Finance to Healthcare, through Insurance and Education.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- Monte Carlo and Reconstruction Membership Inference Attacks against Generative ModelsProceedings on Privacy Enhancing Technologies, 2019
- The European Union general data protection regulation: what it is and what it meansInformation & Communications Technology Law, 2019
- The Synthetic Data Vault2016
- XGBoost2016
- Weight of evidence as a tool for attribute transformation in the preprocessing stage of supervised learning algorithms2011
- Testing differences in proportionsAustralian Critical Care, 2011
- Privacy-Enhancing Technologies2009
- Conditional variable importance for random forestsBMC Bioinformatics, 2008
- Correlation Matrix Distance, a Meaningful Measure for Evaluation of Non-Stationary MIMO Channels2005
- Measuring Statistical Dependence with Hilbert-Schmidt Norms2005