On the hinge-finding algorithm for hingeing hyperplanes
Authors:
- J. SjobergJSJ. Sjoberg
- 1 May 1998
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 44 (3) , 1310-1319
Abstract
This correspondence concerns the estimation algorithm for hinging hyperplane (HH) models, a piecewise-linear model for approximating functions of several variables, suggested in Breiman (1993). The estimation algorithm is analyzed and it is shown that it is a special case of a Newton algorithm applied to a sum of squared error criterion. This insight is then used to suggest possible improvements of the algorithm so that convergence to a local minimum can be guaranteed. In addition, the way of updating the parameters in the HH model is discussed. In Breiman, a stepwise updating procedure is proposed where only a subset of the parameters are changed in each step. This connects closely to some previously suggested greedy algorithms and these greedy algorithms are discussed and compared to a simultaneous updating of all parametersKeywords
This publication has 8 references indexed in Scilit:
- Some remarks on greedy algorithmsAdvances in Computational Mathematics, 1996
- Efficient agnostic learning of neural networks with bounded fan-inIEEE Transactions on Information Theory, 1996
- Overtraining, regularization and searching for a minimum, with application to neural networksInternational Journal of Control, 1995
- Minimisation methods for training feedforward neural networksNeural Networks, 1994
- Hinging hyperplanes for regression, classification, and function approximationIEEE Transactions on Information Theory, 1993
- Universal approximation bounds for superpositions of a sigmoidal functionIEEE Transactions on Information Theory, 1993
- A Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network TrainingThe Annals of Statistics, 1992
- Projection Pursuit RegressionJournal of the American Statistical Association, 1981