Partitioned least squares
Open Access
- 15 July 2024
- journal article
- research article
- Published by Springer Nature in Machine Learning
- Vol. 113 (9) , 6839-6869
Abstract
Linear least squares is one of the most widely used regression methods in many fields. The simplicity of the model allows this method to be used when data is scarce and allows practitioners to gather some insight into the problem by inspecting the values of the learnt parameters. In this paper we propose a variant of the linear least squares model allowing practitioners to partition the input features into groups of variables that they require to contribute similarly to the final result. We show that the new formulation is not convex and provide two alternative methods to deal with the problem: one non-exact method based on an alternating least squares approach; and one exact method based on a reformulation of the problem. We show the correctness of the exact method and compare the two solutions showing that the exact solution provides better results in a fraction of the time required by the alternating least squares solution (when the number of partitions is small). We also provide a branch and bound algorithm that can be used in place of the exact method when the number of partitions is too large as well as a proof of NP-completeness of the optimization problem.Keywords
Funding Information
- Università degli Studi di Torino
This publication has 21 references indexed in Scilit:
- Partitioned Least SquaresLecture Notes in Computer Science, 2019
- A data-driven statistical model for predicting the critical temperature of a superconductorComputational Materials Science, 2018
- A Fast Chromatographic Method for Estimating Lipophilicity and Ionization in Nonpolar Membrane-Like EnvironmentMolecular Pharmaceutics, 2016
- Molecular interaction fields based descriptors to interpret and compare chromatographic indexesJournal of Chromatography A, 2012
- Partial least squares regression and projection on latent structure regression (PLS Regression)WIREs Computational Statistics, 2010
- Biconvex sets and optimization with biconvex functions: a survey and extensionsMathematical Methods of Operations Research, 2007
- Multicoil Dixon chemical species separation with an iterative least‐squares estimation methodMagnetic Resonance in Medicine, 2003
- Maximum likelihood fitting using ordinary least squares algorithmsJournal of Chemometrics, 2002
- PLS-regression: a basic tool of chemometricsChemometrics and Intelligent Laboratory Systems, 2001
- A tutorial history of least squares with applications to astronomy and geodesyJournal of Computational and Applied Mathematics, 2000