Interpreting Deep Forest through Feature Contribution and MDI Feature Importance
Open Access
- 30 November 2025
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Knowledge Discovery From Data
- Vol. 20 (1) , 1-21
Abstract
Deep forest is a non-differentiable deep model that has achieved impressive empirical success across a wide variety of applications, especially on categorical/symbolic or mixed modeling tasks. Many of the application fields prefer explainable models, such as random forests with feature contributions that can provide a local explanation for each prediction, and Mean Decrease Impurity (MDI) that can provide global feature importance. However, deep forest, as a cascade of random forests, possesses interpretability only at the first layer. From the second layer on, many of the tree splits occur on the new features generated by the previous layer, which makes existing explaining tools for random forests inapplicable. To disclose the impact of the original features in the deep layers, we design a calculation method with an estimation step followed by a calibration step for each layer, and propose our feature contribution and MDI feature importance calculation tools for deep forest. Experimental results on both simulated data and real-world data verify the effectiveness of our methods.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62176117,62306104)
- jiangsu science foundation (BK20230949)
- China Postdoctoral Science Foundation (2023TQ0104)
- Jiangsu Excellent Postdoctoral Program (2023ZB140)
- Collaborative Innovation Center of Novel Software Technology and Industrialization
This publication has 24 references indexed in Scilit:
- A computationally fast variable importance test for random forests for high-dimensional dataAdvances in Data Analysis and Classification, 2016
- XGBoost2016
- Consistency of random forestsThe Annals of Statistics, 2015
- Random Forests2012
- Letter to the Editor: On the stability and ranking of predictors from random forest variable importance measuresBriefings in Bioinformatics, 2011
- Predictor correlation impacts machine learning algorithms: implications for genomic studiesBioinformatics, 2009
- Conditional variable importance for random forestsBMC Bioinformatics, 2008
- Bias in random forest variable importance measures: Illustrations, sources and a solutionBMC Bioinformatics, 2007
- Extremely randomized treesMachine Learning, 2006
- Greedy function approximation: A gradient boosting machine.The Annals of Statistics, 2001