TabNet: Attentive Interpretable Tabular Learning
Preprint
Abstract
We propose a novel high-performance and interpretable canonical deep tabular data learning architecture, TabNet. TabNet uses sequential attention to choose which features to reason from at each decision step, enabling interpretability and more efficient learning as the learning capacity is used for the most salient features. We demonstrate that TabNet outperforms other neural network and decision tree variants on a wide range of non-performance-saturated tabular datasets and yields interpretable feature attributions plus insights into the global model behavior. Finally, for the first time to our knowledge, we demonstrate self-supervised learning for tabular data, significantly improving performance with unsupervised representation learning when unlabeled data is abundant.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Deep Neural Network Initialization With Decision TreesIEEE Transactions on Neural Networks and Learning Systems, 2018
- Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network scienceNature Communications, 2018
- Deep Residual Learning for Image Recognition2016
- “Why Should I Trust You?”: Explaining the Predictions of Any Classifier2016
- Deep Neural Decision Forests2015
- Feature selection with decision tree criterion2005
- The random subspace method for constructing decision forestsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998