Invasive or More Direct Measurements Can Provide an Objective Early-Stopping Ceiling for Training Deep Neural Networks on Non-invasive or Less-Direct Biomedical Data
Authors:
- Christopher W. BartlettCBChristopher W. Bartlett
- Jamie BossenbroekJBJamie Bossenbroek
- Yukie UeyamaYUYukie Ueyama
- Patricia McCallinhartPMPatricia McCallinhart
- Olivia A. PetersOPOlivia A. Peters
- Donna A. SantillanDSDonna A. Santillan
- Mark K. SantillanMSMark K. Santillan
- Aaron J. TraskATAaron J. Trask
- William C. RayWRWilliam C. Ray
Open Access
- 12 January 2023
- journal article
- research article
- Published by Springer Nature in SN Computer Science
- Vol. 4 (2) , 1-12
Abstract
Early stopping is an extremely common tool to minimize overfitting, which would otherwise be a cause of poor generalization of the model to novel data. However, early stopping is a heuristic that, while effective, primarily relies on ad hoc parameters and metrics. Optimizing when to stop remains a challenge. In this paper, we suggest that for some biomedical applications, a natural dichotomy of invasive/non-invasive measurements, or more generally proximal vs distal measurements of a biological system can be exploited to provide objective advice on early stopping. We discuss the conditions where invasive measurements of a biological process should provide better predictions than non-invasive measurements, or at best offer parity. Hence, if data from an invasive measurement are available locally, or from the literature, that information can be leveraged to know with high certainty whether a model of non-invasive data is overfitted. We present paired invasive/non-invasive cardiac and coronary artery measurements from two mouse strains, one of which spontaneously develops type 2 diabetes, posed as a classification problem. Examination of the various stopping rules shows that generalization is reduced with more training epochs and commonly applied stopping rules give widely different generalization error estimates. The use of an empirically derived training ceiling is demonstrated to be helpful as added information to leverage early stopping in order to reduce overfitting.Keywords
Funding Information
- National Institutes of Health (UL1TR002537, R00HL116769, R21EB026518, R01HD08994)
- National Institutes of Health (P50HD105556)
This publication has 25 references indexed in Scilit:
- Differential coronary resistance microvessel remodeling between type 1 and type 2 diabetic mice: Impact of exercise trainingVascular Pharmacology, 2012
- Dynamic micro- and macrovascular remodeling in coronary circulation of obese Ossabaw pigs with metabolic syndromeJournal of Applied Physiology, 2012
- Doppler assessment of the fetus with intrauterine growth restrictionAmerican Journal of Obstetrics and Gynecology, 2012
- Exercise Training Prevents Coronary Endothelial Dysfunction in Type 2 Diabetic MiceAmerican Journal of Biomedical Sciences, 2011
- Exercise training improves endothelial function via adiponectin-dependent and independent pathways in type 2 diabetic miceAmerican Journal of Physiology-Heart and Circulatory Physiology, 2011
- Coronary arterioles in type 2 diabetic (db/db) mice undergo a distinct pattern of remodeling associated with decreased vessel stiffnessBasic Research in Cardiology, 2011
- Effect of PAR2 in regulating TNF-α and NAD(P)H oxidase in coronary arterioles in type 2 diabetic miceBasic Research in Cardiology, 2010
- Intrauterine growth restriction: comparison of American College of Obstetricians and Gynecologists practice bulletin with other national guidelinesAmerican Journal of Obstetrics and Gynecology, 2009
- Role of EDHF in type 2 diabetes-induced endothelial dysfunctionAmerican Journal of Physiology-Heart and Circulatory Physiology, 2008
- Risk factors for intrauterine growth restriction in a socio-economically disadvantaged regionThe Journal of Maternal-Fetal & Neonatal Medicine, 2005