Characterizing Genetic Programming Error Through Extended Bias and Variance Decomposition
- 28 April 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Evolutionary Computation
- Vol. 24 (6) , 1164-1176
Abstract
An error function can be used to select between candidate models but it does not provide a thorough understanding of the behaviour of a model. A greater understanding of an algorithm can be obtained by performing a bias-variance decomposition. Splitting the error into bias and variance is effective for understanding a deterministic algorithm such as k-Nearest Neighbour, which provides the same predictions when performed multiple times using the same data. However, simply splitting the error into bias and variance is not sufficient for non-deterministic algorithms, such as genetic programming (GP), which potentially produces a different model each time it is run, even when using the same data. This paper presents an extended bias-variance decomposition that decomposes error into bias, external variance (error attributable to limited sampling of the problem) and internal variance (error due to random actions performed in the algorithm itself). This decomposition is applied to GP to expose the three components of error, providing a unique insight into the role of maximum tree depth, number of generations, size/complexity of function set and data standardisation in influencing predictive performance. The proposed tool can be used to inform targeted improvements for reducing specific components of model error.Keywords
Funding Information
- University of Otago Doctoral Scholarship
This publication has 24 references indexed in Scilit:
- Controlling Overfitting in Symbolic Regression Based on a Bias/Variance Error Decomposition2012
- Geometric Semantic Genetic Programming2012
- Open issues in genetic programmingGenetic Programming and Evolvable Machines, 2010
- Distilling Free-Form Natural Laws from Experimental DataScience, 2009
- Practical Bias Variance DecompositionLecture Notes in Computer Science, 2008
- Improving Symbolic Regression with Interval Arithmetic and Linear Scaling2003
- Genetic Programming, Ensemble Methods and the Bias/Variance Tradeoff – Introductory InvestigationsLecture Notes in Computer Science, 2000
- Boosting the margin: a new explanation for the effectiveness of voting methodsThe Annals of Statistics, 1998
- Bagging predictorsMachine Learning, 1996
- Neural Networks and the Bias/Variance DilemmaNeural Computation, 1992