Temporal difference learning applied to sequential detection
Authors:
- Chengan GuoCGChengan Guo
- A. KuhAKA. Kuh
- 1 March 1997
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 8 (2) , 278-287
Abstract
This paper proposes a novel neural-network method for sequential detection, We first examine the optimal parametric sequential probability ratio test (SPRT) and make a simple equivalent transformation of the SPRT that makes it suitable for neural-network architectures. We then discuss how neural networks can learn the SPRT decision functions from observation data and labels. Conventional supervised learning algorithms have difficulties handling the variable length observation sequences, but a reinforcement learning algorithm, the temporal difference (TD) learning algorithm works ideally in training the neural network. The entire neural network is composed of context units followed by a feedforward neural network. The context units are necessary to store dynamic information that is needed to make good decisions. For an appropriate neural-network architecture, trained with independent and identically distributed (iid) observations by the TD learning algorithm, we show that the neural-network sequential detector can closely approximate the optimal SPRT with similar performance. The neural-network sequential detector has the additional advantage that it is a nonparametric detector that does not require probability density functions. Simulations demonstrated on iid Gaussian data show that the neural network and the SPRT have similar performance.Keywords
This publication has 9 references indexed in Scilit:
- A sequential procedure for multihypothesis testingIEEE Transactions on Information Theory, 1994
- Decentralized sequential detection with a fusion center performing the sequential testIEEE Transactions on Information Theory, 1993
- The convergence of TD(?) for general ?Machine Learning, 1992
- An optimum multilayer perceptron neural receiver for signal detectionIEEE Transactions on Neural Networks, 1990
- Decentralized sequential detectionIEEE Transactions on Information Theory, 1989
- Learning to predict by the methods of temporal differencesMachine Learning, 1988
- Strategy Learning with Multilayer Connectionist Representations1987
- Optimum Character of the Sequential Probability Ratio TestThe Annals of Mathematical Statistics, 1948
- Sequential Tests of Statistical HypothesesThe Annals of Mathematical Statistics, 1945