Transfer Learning with Multi-source Data: High-dimensional Inference for Group Distributionally Robust Models
Author:
Preprint
Abstract
The construction of generalizable and transferable models is a fundamental goal of statistical learning. Learning with the multi-source data helps improve model generalizability and is integral to many important statistical problems, including group distributionally robust optimization, minimax group fairness, and maximin projection. This paper considers multiple high-dimensional regression models for the multi-source data. We introduce the covariate shift maximin effect as a group distributionally robust model. This robust model helps transfer the information from the multi-source data to the unlabelled target population. Statistical inference for the covariate shift maximin effect is challenging since its point estimator may have a non-standard limiting distribution. We devise a novel {\it DenseNet} sampling method to construct valid confidence intervals for the high-dimensional maximin effect. We show that our proposed confidence interval achieves the desired coverage level and attains a parametric length. Our proposed DenseNet sampling method and the related theoretical analysis are of independent interest in addressing other non-regular or non-standard inference problems. We demonstrate the proposed method over a large-scale simulation and genetic data on yeast colony growth under multiple environments.Keywords
All Related Versions
This publication has 49 references indexed in Scilit:
- Individual Data Protected Integrative Regression Analysis of High-Dimensional Heterogeneous DataJournal of the American Statistical Association, 2021
- The Missing Diversity in Human Genetic StudiesCell, 2019
- Optimal Estimation of Genetic Relatedness in High-Dimensional Linear ModelsJournal of the American Statistical Association, 2018
- Linear Hypothesis Testing in Dense High-Dimensional Linear ModelsJournal of the American Statistical Association, 2018
- A study of generalizability of recurrent neural network-based predictive models for heart failure onset risk using a large and heterogeneous EHR data setJournal of Biomedical Informatics, 2018
- Generalized Fiducial Inference: A Review and New ResultsJournal of the American Statistical Association, 2016
- Robust inference on average treatment effects with possibly more covariates than observationsJournal of Econometrics, 2015
- Finding the sources of missing heritability in a yeast crossNature, 2013
- Square-root lasso: pivotal recovery of sparse signals via conic programmingBiometrika, 2011
- Improving predictive inference under covariate shift by weighting the log-likelihood functionJournal of Statistical Planning and Inference, 2000