Improving Generalisation of Genetic Programming for Symbolic Regression with Structural Risk Minimisation
Authors:
Open Access
- 20 July 2016
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 709-716
Abstract
Generalisation is one of the most important performance measures for any learning algorithm, no exception to Genetic Programming (GP). A number of works have been devoted to improve the generalisation ability of GP for symbolic regression. Methods based on a reliable estimation of generalisation error of models during evolutionary process are a sensible choice to enhance the generalisation of GP. Structural risk minimisation (SRM), which is based on the VC dimension in the learning theory, provides a powerful framework for estimating the difference between the generalisation error and the empirical error. Despite its solid theoretical foundation and reliability, SRM has seldom been applied to GP. The most important reason is the difficulty in measuring the VC dimension of GP models/programs. This paper introduces SRM, which is based on an empirical method to measure the VC dimension of models, into GP to improve its generalisation performance for symbolic regression. The results of a set of experiments confirm that GP with SRM has a dramatical generalisation gain while evolving more compact/less complex models than standard GP. Further analysis also shows that in most cases, GP with SRM has better generalisation performance than GP with bias-variance decomposition, which is one of the state-of-the-art methods to control overfitting.Keywords
Funding Information
- the University Research Fund at Victoria University of Wellington (210375/3557 209861/3580)
- the Marsden Fund of the New Zealand Government (VUW1209)
- State Key Laboratory for Novel Software Technology (Overseas Collaboration Grant) at Nanjing University (KFKT2014A28)
This publication has 24 references indexed in Scilit:
- Controlling Overfitting in Symbolic Regression Based on a Bias/Variance Error Decomposition2012
- Operator equalisation for bloat free genetic programming and a survey of bloat control methodsGenetic Programming and Evolvable Machines, 2011
- Penalty Functions for Genetic Programming AlgorithmsLecture Notes in Computer Science, 2011
- Improving the Generalisation Ability of Genetic Programming with Semantic Similarity based CrossoverLecture Notes in Computer Science, 2010
- Comparison of Model Selection for RegressionNeural Computation, 2003
- Improving Symbolic Regression with Interval Arithmetic and Linear Scaling2003
- Genetic Programming, Ensemble Methods and the Bias/Variance Tradeoff – Introductory InvestigationsLecture Notes in Computer Science, 2000
- Measuring the VC-Dimension of a Learning MachineNeural Computation, 1994
- Improving generalization with active learningMachine Learning, 1994
- Statistical predictor identificationAnnals of the Institute of Statistical Mathematics, 1970