Heterogeneous sets in dimensionality reduction and ensemble learning
Open Access
- 28 October 2022
- journal article
- research article
- Published by Springer Nature in Machine Learning
- Vol. 113 (4) , 1683-1704
Abstract
We present a general framework for dealing with set heterogeneity in data and learning problems, which is able to exploit low complexity components. The main ingredients are (i) A definition of complexity for elements of a convex union that takes into account the complexities of their individual composition – this is used to cover the heterogeneous convex union; and (ii) Upper bounds on the complexities of restricted subsets. We demonstrate this approach in two different application areas, highlighting their conceptual connection. (1) In random projection based dimensionality reduction, we obtain improved bounds on the uniform preservation of Euclidean norms and distances when low complexity components are present in the union. (2) In statistical learning, our generalisation bounds justify heterogeneous ensemble learning methods that were incompletely understood before. We exemplify empirical results with boosting type random subspace and random projection ensembles that implement our bounds.Keywords
Funding Information
- Engineering and Physical Sciences Research Council (EP/P004245/1)
This publication has 9 references indexed in Scilit:
- High-Dimensional Data Analysis with Low-Dimensional Models2022
- High-Dimensional Statistics2019
- Random-projection Ensemble ClassificationJournal of the Royal Statistical Society Series B: Statistical Methodology, 2017
- A Simple Tool for Bounding the Deviation of Random Matrices on Geometric Sets2017
- Fast random k-labELsets for large-scale multi-label classification2016
- Concentration Inequalities2013
- Classifier chains for multi-label classificationMachine Learning, 2011
- Empirical processes and random projectionsJournal of Functional Analysis, 2005
- Random projection in dimensionality reduction2001