Caitlin A. Owen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Caitlin A. Owen has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Grant Dick
School of Computing, University of Otago, Dunedin, New Zealand
7 shared publications - Peter Alexander Whigham
University of Otago
6 shared publications Peter A. Whigham
University of Otago, Dunedin, New Zealand
4 shared publicationsGrant Dick
University of Otago, Dunedin, New Zealand
2 shared publicationsBastin Tony Roy Savarimuthu
1 shared publication- Claudia Ott
University of Otago, Dunedin, New Zealand
1 shared publication
Latest Publications
- Open Access
- 2 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-8
Where are the Indigenous Usability Experts? — Guided Heuristic Evaluation with Non-Experts
- 2 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-8
Open AccessWhere are the Indigenous Usability Experts? — Guided Heuristic Evaluation with Non-Experts
- 20 November 2024
- Published by Springer Nature
- p. 162-173
Revisiting Bagging for Stochastic Algorithms
- Open Access
- 14 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 2050-2057
Characterising the Double Descent of Symbolic Regression
Recent work has argued that many machine learning techniques exhibit a 'double descent' in model risk, where increasing model complexity beyond an interpolation zone can overcome the bias-variance tradeoff to produce large, over-parameterised models that generalise well to unseen data. While the double descent characteristic has been identified in many learning methods, it has not been explored within symbolic regression research. This paper presents an initial exploration into the presence of double descent behaviour in symbolic regression over a range of parameter settings. Results suggest that symbolic regression via genetic programming does not exhibit a clear double descent risk curve relative to model size or function set. Unlike other methods, models evolved through symbolic regression do not appear to strongly interpolate training data, which promotes a degree of robustness towards noise in training data. However, models evolved by symbolic regression can still be large and do not present a strong overfitting characteristic. Given that a prime motivation for symbolic regression is to produce compact interpretable models, these results suggest that methods aimed at regularising evolved models should be a key feature of all symbolic regression methods.
- 14 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 2050-2057
Open AccessCharacterising the Double Descent of Symbolic Regression
Recent work has argued that many machine learning techniques exhibit a 'double descent' in model risk, where increasing model complexity beyond an interpolation zone can overcome the bias-variance tradeoff to produce large, over-parameterised models that generalise well to unseen data. While the double descent characteristic has been identified in many learning methods, it has not been explored within symbolic regression research. This paper presents an initial exploration into the presence of double descent behaviour in symbolic regression over a range of parameter settings. Results suggest that symbolic regression via genetic programming does not exhibit a clear double descent risk curve relative to model size or function set. Unlike other methods, models evolved through symbolic regression do not appear to strongly interpolate training data, which promotes a degree of robustness towards noise in training data. However, models evolved by symbolic regression can still be large and do not present a strong overfitting characteristic. Given that a prime motivation for symbolic regression is to produce compact interpretable models, these results suggest that methods aimed at regularising evolved models should be a key feature of all symbolic regression methods.
- 27 February 2024
- Published by MIT Press in Evolutionary Computation
- Vol. 32 (1) , 49-68
Using Decomposed Error for Reproducing Implicit Understanding of Algorithms
Reproducibility is important for having confidence in evolutionary machine learning algorithms. Although the focus of reproducibility is usually to recreate an aggregate prediction error score using fixed random seeds, this is not sufficient. Firstly, multiple runs of an algorithm, without a fixed random seed, should ideally return statistically equivalent results. Secondly, it should be confirmed whether the expected behaviour of an algorithm matches its actual behaviour, in terms of how an algorithm targets a reduction in prediction error. Confirming the behaviour of an algorithm is not possible when using a total error aggregate score. Using an error decomposition framework as a methodology for improving the reproducibility of results in evolutionary computation addresses both of these factors. By estimating decomposed error using multiple runs of an algorithm and multiple training sets, the framework provides a greater degree of certainty about the prediction error. Also, decomposing error into bias, variance due to the algorithm (internal variance) and variance due to the training data (external variance) more fully characterises evolutionary algorithms. This allows the behaviour of an algorithm to be confirmed. Applying the framework to a number of evolutionary algorithms shows that their expected behaviour can be different to their actual behaviour. Identifying a behaviour mismatch is important in terms of understanding how to further refine an algorithm as well as how to effectively apply an algorithm to a problem.
- 27 February 2024
- Published by MIT Press in Evolutionary Computation
- Vol. 32 (1) , 49-68
Using Decomposed Error for Reproducing Implicit Understanding of Algorithms
Reproducibility is important for having confidence in evolutionary machine learning algorithms. Although the focus of reproducibility is usually to recreate an aggregate prediction error score using fixed random seeds, this is not sufficient. Firstly, multiple runs of an algorithm, without a fixed random seed, should ideally return statistically equivalent results. Secondly, it should be confirmed whether the expected behaviour of an algorithm matches its actual behaviour, in terms of how an algorithm targets a reduction in prediction error. Confirming the behaviour of an algorithm is not possible when using a total error aggregate score. Using an error decomposition framework as a methodology for improving the reproducibility of results in evolutionary computation addresses both of these factors. By estimating decomposed error using multiple runs of an algorithm and multiple training sets, the framework provides a greater degree of certainty about the prediction error. Also, decomposing error into bias, variance due to the algorithm (internal variance) and variance due to the training data (external variance) more fully characterises evolutionary algorithms. This allows the behaviour of an algorithm to be confirmed. Applying the framework to a number of evolutionary algorithms shows that their expected behaviour can be different to their actual behaviour. Identifying a behaviour mismatch is important in terms of understanding how to further refine an algorithm as well as how to effectively apply an algorithm to a problem.
- 3 December 2022
- Published by Springer Nature
- p. 177-190
Towards Explainable AutoML Using Error Decomposition
- 3 December 2022
- Published by Springer Nature
- p. 177-190
Towards Explainable AutoML Using Error Decomposition
- 17 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Evolutionary Computation
- Vol. 26 (6) , 1596-1608
Standardization and Data Augmentation in Genetic Programming
Genetic programming (GP) is a common method for performing symbolic regression that relies on the use of ephemeral random constants in order to adequately scale predictions. Suitable values for these constants must be drawn from appropriate, but typically unknown, distributions for the problem being modelled. While rarely used with GP, Z-score standardisation of feature and response spaces often significantly improves the predictive performance of GP by removing scale issues and reducing error due to bias. However, in some cases it is also associated with erratic error due to variance. This paper demonstrates that this variance component increases in the presence of gaps at the boundaries of the training data explanatory variable intervals. An initial solution to this problem is proposed that augments training data with pseudo instances located at the boundaries of the intervals. When applied to benchmark problems, particularly with small training samples, this solution reduces error due to variance and therefore total error. Augmentation is shown to also stabilise error in larger problems, however results suggest that standardised GP works well on such problems with little need for training data augmentation.
- 17 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Evolutionary Computation
- Vol. 26 (6) , 1596-1608
Standardization and Data Augmentation in Genetic Programming
Genetic programming (GP) is a common method for performing symbolic regression that relies on the use of ephemeral random constants in order to adequately scale predictions. Suitable values for these constants must be drawn from appropriate, but typically unknown, distributions for the problem being modelled. While rarely used with GP, Z-score standardisation of feature and response spaces often significantly improves the predictive performance of GP by removing scale issues and reducing error due to bias. However, in some cases it is also associated with erratic error due to variance. This paper demonstrates that this variance component increases in the presence of gaps at the boundaries of the training data explanatory variable intervals. An initial solution to this problem is proposed that augments training data with pseudo instances located at the boundaries of the intervals. When applied to benchmark problems, particularly with small training samples, this solution reduces error due to variance and therefore total error. Augmentation is shown to also stabilise error in larger problems, however results suggest that standardised GP works well on such problems with little need for training data augmentation.
- 4 November 2020
- Published by Taylor & Francis in Journal of the Operational Research Society
- Vol. 73 (2) , 285-300
Simulation evidence on Herfindahl-Hirschman measures of competitive balance in professional sports leagues
Measurement of the degree of competitive balance, how evenly teams are matched, is central to the analysis of professional sports leagues. A common problem with within-season competitive balance measures, however, is their sensitivity to the number of teams and the number of matches played by each team, i.e. season length. This paper uses simulation methods to examine the effects of changes in season length on the distributions of several widely used variants of the Herfindahl-Hirschman index applied to wins in a season. Accounting for both the lower and upper bounds of the index improves its robustness to variation in the numbers of teams or games played per season, but does not completely remove overestimation of imbalance associated with shorter season lengths. Overall, McGee’s balance measure significantly mitigates the dependence on the number of matches and, of the measures considered, is recommended as the most robust to variation in season length.
- 4 November 2020
- Published by Taylor & Francis in Journal of the Operational Research Society
- Vol. 73 (2) , 285-300
Simulation evidence on Herfindahl-Hirschman measures of competitive balance in professional sports leagues
Measurement of the degree of competitive balance, how evenly teams are matched, is central to the analysis of professional sports leagues. A common problem with within-season competitive balance measures, however, is their sensitivity to the number of teams and the number of matches played by each team, i.e. season length. This paper uses simulation methods to examine the effects of changes in season length on the distributions of several widely used variants of the Herfindahl-Hirschman index applied to wins in a season. Accounting for both the lower and upper bounds of the index improves its robustness to variation in the numbers of teams or games played per season, but does not completely remove overestimation of imbalance associated with shorter season lengths. Overall, McGee’s balance measure significantly mitigates the dependence on the number of matches and, of the measures considered, is recommended as the most robust to variation in season length.
- 1 November 2020
- Published by Elsevier in Science of Computer Programming
- Vol. 199
Understanding stack overflow code quality: A recommendation of caution
- 1 November 2020
- Published by Elsevier in Science of Computer Programming
- Vol. 199
Understanding stack overflow code quality: A recommendation of caution
- 28 April 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Evolutionary Computation
- Vol. 24 (6) , 1164-1176
Characterizing Genetic Programming Error Through Extended Bias and Variance Decomposition
An error function can be used to select between candidate models but it does not provide a thorough understanding of the behaviour of a model. A greater understanding of an algorithm can be obtained by performing a bias-variance decomposition. Splitting the error into bias and variance is effective for understanding a deterministic algorithm such as k-Nearest Neighbour, which provides the same predictions when performed multiple times using the same data. However, simply splitting the error into bias and variance is not sufficient for non-deterministic algorithms, such as genetic programming (GP), which potentially produces a different model each time it is run, even when using the same data. This paper presents an extended bias-variance decomposition that decomposes error into bias, external variance (error attributable to limited sampling of the problem) and internal variance (error due to random actions performed in the algorithm itself). This decomposition is applied to GP to expose the three components of error, providing a unique insight into the role of maximum tree depth, number of generations, size/complexity of function set and data standardisation in influencing predictive performance. The proposed tool can be used to inform targeted improvements for reducing specific components of model error.
- 28 April 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Evolutionary Computation
- Vol. 24 (6) , 1164-1176
Characterizing Genetic Programming Error Through Extended Bias and Variance Decomposition
An error function can be used to select between candidate models but it does not provide a thorough understanding of the behaviour of a model. A greater understanding of an algorithm can be obtained by performing a bias-variance decomposition. Splitting the error into bias and variance is effective for understanding a deterministic algorithm such as k-Nearest Neighbour, which provides the same predictions when performed multiple times using the same data. However, simply splitting the error into bias and variance is not sufficient for non-deterministic algorithms, such as genetic programming (GP), which potentially produces a different model each time it is run, even when using the same data. This paper presents an extended bias-variance decomposition that decomposes error into bias, external variance (error attributable to limited sampling of the problem) and internal variance (error due to random actions performed in the algorithm itself). This decomposition is applied to GP to expose the three components of error, providing a unique insight into the role of maximum tree depth, number of generations, size/complexity of function set and data standardisation in influencing predictive performance. The proposed tool can be used to inform targeted improvements for reducing specific components of model error.
- 10 November 2018
- Published by Springer Nature
- p. 565-576
Feature Standardisation in Symbolic Regression
- 10 November 2018
- Published by Springer Nature
- p. 565-576
Feature Standardisation in Symbolic Regression