Accretionary Learning With Deep Neural Networks With Applications
- 15 December 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive Communications and Networking
- Vol. 10 (2) , 660-673
Abstract
One of the fundamental limitations of Deep Neural Networks (DNN) is their inability to acquire and accumulate new cognitive capabilities in an incremental or progressive manner. When data appear from object classes not among the learned ones, a conventional DNN would not be able to recognize them due to the fundamental formulation that it assumes. A typical solution is to re-design and re-learn a new network, most likely an expanded one, for the expanded set of object classes. This process is quite different from that of a human learner. In this paper, we propose a new learning method named Accretionary Learning (AL) to emulate human learning, in that the set of object classes to be recognized need not be fixed, meaning it can grow as the situation arises without requiring an entire redesign of the system. The proposed learning structure is modularized, and can dynamically expand to learn and register new knowledge, as the set of objects grows in size. AL does not forget previous knowledge when learning new data classes. We show that the structure and its learning methodology lead to a system that can grow to cope with increased cognitive complexity while providing stable and superior overall performance.Keywords
This publication has 29 references indexed in Scilit:
- Overcoming Catastrophic Forgetting Using Sparse Coding and Meta LearningIEEE Access, 2021
- Incremental Learning in Deep Convolutional Neural Networks Using Partial Network SharingIEEE Access, 2019
- Learning Without Memorizing2019
- Continual lifelong learning with neural networks: A reviewNeural Networks, 2019
- Deep Learning in Physical Layer CommunicationsIEEE Wireless Communications, 2019
- iCaRL: Incremental Classifier and Representation Learning2017
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences, 2017
- Deep neural networks – a developmental perspectiveAPSIPA Transactions on Signal and Information Processing, 2016
- Online Gradient Descent Learning AlgorithmsFoundations of Computational Mathematics, 2007
- An introduction to ROC analysisPattern Recognition Letters, 2006