Is a Complex-Valued Stepsize Advantageous in Complex-Valued Gradient Learning Algorithms?
Authors:
- 5 November 2015
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 27 (12) , 2730-2735
Abstract
Complex gradient methods have been widely used in learning theory, and typically aim to optimize real-valued functions of complex variables. The stepsize of complex gradient learning methods (CGLMs) is a positive number, and little is known about how a complex stepsize would affect the learning process. To this end, we undertake a comprehensive analysis of CGLMs with a complex stepsize, including the search space, convergence properties, and the dynamics near critical points. Furthermore, several adaptive stepsizes are derived by extending the Barzilai-Borwein method to the complex domain, in order to show that the complex stepsize is superior to the corresponding real one in approximating the information in the Hessian. A numerical example is presented to support the analysis.Keywords
Funding Information
- National Natural Science Foundation of China (61101228, 61402071)
- Liaoning Provincial Natural Science Foundation of China (2015020011)
- Fundamental Research Funds for the Central Universities of China (3132015157)
This publication has 30 references indexed in Scilit:
- Feasibility of Constrained Receding Horizon Control Implementation in Adaptive OpticsIEEE Transactions on Control Systems Technology, 2014
- A complex-valued neuro-fuzzy inference system and its learning mechanismNeurocomputing, 2014
- Boundedness and Convergence of Split-Complex Back-Propagation Algorithm with Momentum and PenaltyNeural Processing Letters, 2013
- Complex Blind Source Extraction From Noisy Mixtures Using Second-Order StatisticsIEEE Transactions on Circuits and Systems I: Regular Papers, 2010
- Joint Video Summarization and Transmission Adaptation for Energy-Efficient Wireless Video StreamingEURASIP Journal on Advances in Signal Processing, 2008
- A Complex-Valued RTRL Algorithm for Recurrent Neural NetworksNeural Computation, 2004
- Approximation by Fully Complex Multilayer PerceptronsNeural Computation, 2003
- Fully Complex Multi-Layer Perceptron Network for Nonlinear Signal ProcessingJournal of Signal Processing Systems, 2002
- A complex gradient operator and its application in adaptive array theoryIEE Proceedings F Communications, Radar and Signal Processing, 1983
- Zur formalen Theorie der Funktionen von mehr komplexen Ver nderlichenMathematische Annalen, 1927