Mean-Square Convergence Analysis of ADALINE Training With Minimum Error Entropy Criterion
Authors:
- 21 June 2010
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 21 (7) , 1168-1179
Abstract
Recently, the minimum error entropy (MEE) criterion has been used as an information theoretic alternative to traditional mean-square error criterion in supervised learning systems. MEE yields nonquadratic, nonconvex performance surface even for adaptive linear neuron (ADALINE) training, which complicates the theoretical analysis of the method. In this paper, we develop a unified approach for mean-square convergence analysis for ADALINE training under MEE criterion. The weight update equation is formulated in the form of block-data. Based on a block version of energy conservation relation, and under several assumptions, we carry out the mean-square convergence analysis of this class of adaptation algorithm, including mean-square stability, mean-square evolution (transient behavior) and the mean-square steady-state performance. Simulation experimental results agree with the theoretical predictions very well.Keywords
This publication has 28 references indexed in Scilit:
- Mean-Square Performance of a Family of Affine Projection AlgorithmsIEEE Transactions on Signal Processing, 2004
- A unified approach to the steady-state and tracking analyses of adaptive filtersIEEE Transactions on Signal Processing, 2001
- Adaptive Filters with Error Nonlinearities: Mean-Square Analysis and Optimum DesignEURASIP Journal on Advances in Signal Processing, 2001
- Convergence and steady-state properties of the least-mean mixed-norm (LMMN) adaptive algorithmIEE Proceedings - Vision, Image, and Signal Processing, 1996
- Analysis of the multiple-error and block least-mean-square adaptive algorithmsIEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 1995
- Least mean p-power error criterion for adaptive FIR filterIEEE Journal on Selected Areas in Communications, 1994
- Stochastic gradient adaptation under general error criteriaIEEE Transactions on Signal Processing, 1994
- Saturation effects in LMS adaptive echo cancellation for binary dataIEEE Transactions on Acoustics, Speech, and Signal Processing, 1990
- Fast algorithms for block FIR adaptive digital filteringIEEE Transactions on Circuits and Systems, 1987
- Block implementation of adaptive digital filtersIEEE Transactions on Acoustics, Speech, and Signal Processing, 1981