Learning With Interpretable Structure From Gated RNN
- 13 February 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. PP (2162237X) , 1-13
Abstract
The interpretability of deep learning models has raised extended attention these years. It will be beneficial if we can learn an interpretable structure from deep learning models. In this article, we focus on recurrent neural networks (RNNs), especially gated RNNs whose inner mechanism is still not clearly understood. We find that finite-state automaton (FSA) that processes sequential data have a more interpretable inner mechanism according to the definition of interpretability and can be learned from RNNs as the interpretable structure. We propose two methods to learn FSA from RNN based on two different clustering methods. With the learned FSA and via experiments on artificial and real data sets, we find that FSA is more trustable than the RNN from which it learned, which gives FSA a chance to substitute RNNs in applications involving humans' lives or dangerous facilities. Besides, we analyze how the number of gates affects the performance of RNN. Our result suggests that gate in RNN is important but the less the better, which could be a guidance to design other RNNs. Finally, we observe that the FSA learned from RNN gives semantic aggregated states, and its transition graph shows us a very interesting vision of how RNNs intrinsically handle text classification tasks.Keywords
All Related Versions
Funding Information
- National Basic Research Program of China (2018YFB1004300)
- National Natural Science Foundation of China (61751306, 61921006)
- Collaborative Innovation Center of Novel Software Technology and Industrialization
This publication has 36 references indexed in Scilit:
- NeC4.5: neural ensemble based C4.5IEEE Transactions on Knowledge and Data Engineering, 2004
- Rule extraction: Using neural networks or for neural networks?Journal of Computer Science and Technology, 2004
- Learning to Forget: Continual Prediction with LSTMNeural Computation, 2000
- Text categorization with Support Vector Machines: Learning with many relevant features1998
- Long Short-Term MemoryNeural Computation, 1997
- Extraction of rules from discrete-time recurrent neural networksNeural Networks, 1996
- Learning Finite State Machines With Self-Clustering Recurrent NetworksNeural Computation, 1993
- Finding structure in timeCognitive Science, 1990
- Induction of decision treesMachine Learning, 1986
- Algorithm AS 136: A K-Means Clustering AlgorithmJournal of the Royal Statistical Society Series C: Applied Statistics, 1979