Enhancing Explainable Machine Learning by Reconsidering Initially Unselected Items in Feature Selection for Classification
Open Access
- 12 December 2022
- journal article
- research article
- Published by MDPI AG in BioMedInformatics
- Vol. 2 (4) , 701-714
Abstract
Feature selection is a common step in data preprocessing that precedes machine learning to reduce data space and the computational cost of processing or obtaining the data. Filtering out uninformative variables is also important for knowledge discovery. By reducing the data space to only those components that are informative to the class structure, feature selection can simplify models so that they can be more easily interpreted by researchers in the field, reminiscent of explainable artificial intelligence. Knowledge discovery in complex data thus benefits from feature selection that aims to understand feature sets in the thematic context from which the data set originates. However, a single variable selected from a very small number of variables that are technically sufficient for AI training may make little immediate thematic sense, whereas the additional consideration of a variable discarded during feature selection could make scientific discovery very explicit. In this report, we propose an approach to explainable feature selection (XFS) based on a systematic reconsideration of unselected features. The difference between the respective classifications when training the algorithms with the selected features or with the unselected features provides a valid estimate of whether the relevant features in a data set have been selected and uninformative or trivial information was filtered out. It is shown that revisiting originally unselected variables in multivariate data sets allows for the detection of pathologies and errors in the feature selection that occasionally resulted in the failure to identify the most appropriate variables.Keywords
Funding Information
- Deutsche Forschungsgemeinschaft (DFG LO 612/16-1)
This publication has 25 references indexed in Scilit:
- Computed ABC Analysis for Rational Selection of Most Informative Variables in Multivariate DataPLOS ONE, 2015
- Modeling Wine Preferences from Physicochemical Properties using Fuzzy Techniques2015
- Bagging, Boosting, and Random Forests Using R2013
- Feature Selection with theBorutaPackage2010
- A review of feature selection techniques in bioinformaticsBioinformatics, 2007
- Bias in random forest variable importance measures: Illustrations, sources and a solutionBMC Bioinformatics, 2007
- Random decision forests2002
- R: A Language for Data Analysis and GraphicsJournal of Computational and Graphical Statistics, 1996
- Linear Inversion of Band-Limited Reflection SeismogramsSIAM Journal on Scientific and Statistical Computing, 1986
- THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMSAnnals of Eugenics, 1936