On Recurrent Neural Networks for Sequence-based Processing in Communications
Preprint
Abstract
In this work, we analyze the capabilities and practical limitations of neural networks (NNs) for sequence-based signal processing which can be seen as an omnipresent property in almost any modern communication systems. In particular, we train multiple state-of-the-art recurrent neural network (RNN) structures to learn how to decode convolutional codes allowing a clear benchmarking with the corresponding maximum likelihood (ML) Viterbi decoder. We examine the decoding performance for various kinds of NN architectures, beginning with classical types like feedforward layers and gated recurrent unit (GRU)-layers, up to more recently introduced architectures such as temporal convolutional networks (TCNs) and differentiable neural computers (DNCs) with external memory. As a key limitation, it turns out that the training complexity increases exponentially with the length of the encoding memory $\nu$ and, thus, practically limits the achievable bit error rate (BER) performance. To overcome this limitation, we introduce a new training-method by gradually increasing the number of ones within the training sequences, i.e., we constrain the amount of possible training sequences in the beginning until first convergence. By consecutively adding more and more possible sequences to the training set, we finally achieve training success in cases that did not converge before via naive training. Further, we show that our network can learn to jointly detect and decode a quadrature phase shift keying (QPSK) modulated code with sub-optimal (anti-Gray) labeling in one-shot at a performance that would require iterations between demapper and decoder in classic detection schemes.
Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- LEARN Codes: Inventing Low-Latency Codes via Recurrent Neural NetworksIEEE Journal on Selected Areas in Information Theory, 2020
- MIND: Model Independent Neural Decoder2019
- DEEPTURBO: Deep Turbo Decoder2019
- Optimal Training Channel Statistics for Neural-based Decoders2018
- Performance Evaluation of Channel Decoding with Deep Neural Networks2018
- An Introduction to Deep Learning for the Physical LayerIEEE Transactions on Cognitive Communications and Networking, 2017
- On deep learning-based channel decoding2017
- Iterative demapping for QPSK modulationElectronics Letters, 1998
- Multilayer feedforward networks are universal approximatorsNeural Networks, 1989
- Convolutional codes II. Maximum-likelihood decodingInformation and Control, 1974