A new definition for feature selection stability analysis
Authors:
Open Access
- 1 March 2024
- journal article
- research article
- Published by Springer Nature in Annals of Mathematics and Artificial Intelligence
- Vol. 92 (3) , 753-770
Abstract
Feature selection (FS) stability is an important topic of recent interest. Finding stable features is important for creating reliable, non-overfitted feature sets, which in turn can be used to generate machine learning models with better accuracy and explanations and are less prone to adversarial attacks. There are currently several definitions of FS stability that are widely used. In this paper, we demonstrate that existing stability metrics fail to quantify certain key elements of many datasets such as resilience to data drift or non-uniformly distributed missing values. To address this shortcoming, we propose a new definition for FS stability inspired by Lyapunov stability in dynamic systems. We show the proposed definition is statistically different from the classical record-stability on ( $$n=90$$ ) datasets. We present the advantages and disadvantages of using Lyapunov and other stability definitions and demonstrate three scenarios in which each one of the three proposed stability metrics is best suited.
Keywords
This publication has 53 references indexed in Scilit:
- An algorithm to optimize explainability using feature ensemblesApplied Intelligence, 2024
- Implementing Machine Learning Methods in Estimating the Size of the Non-observed EconomyComputational Economics, 2023
- Changes in characteristics and outcomes of critically ill COVID-19 patients in Tyrol (Austria) over 1 yearWiener klinische Wochenschrift, 2021
- Daily Activity Feature Selection in Smart Homes Based on Pearson Correlation CoefficientNeural Processing Letters, 2020
- Dealing with missing data, small sample sizes, and heterogeneity in machine learning studies of brain disorders2020
- Explainability in human–agent systemsAutonomous Agents and Multi-Agent Systems, 2019
- User Behavior Shift Detection in Intelligent Environments2012
- Missing data imputation using statistical and machine learning methods in a real breast cancer problemArtificial Intelligence in Medicine, 2010
- Stability of feature selection algorithms: a study on high-dimensional spacesKnowledge and Information Systems, 2006
- AUC: A Better Measure than Accuracy in Comparing Learning Algorithms2003