Is a Complex-Valued Stepsize Advantageous in Complex-Valued Gradient Learning Algorithms?

Authors:

Abstract
Complex gradient methods have been widely used in learning theory, and typically aim to optimize real-valued functions of complex variables. The stepsize of complex gradient learning methods (CGLMs) is a positive number, and little is known about how a complex stepsize would affect the learning process. To this end, we undertake a comprehensive analysis of CGLMs with a complex stepsize, including the search space, convergence properties, and the dynamics near critical points. Furthermore, several adaptive stepsizes are derived by extending the Barzilai-Borwein method to the complex domain, in order to show that the complex stepsize is superior to the corresponding real one in approximating the information in the Hessian. A numerical example is presented to support the analysis.
Funding Information
  • National Natural Science Foundation of China (61101228, 61402071)
  • Liaoning Provincial Natural Science Foundation of China (2015020011)
  • Fundamental Research Funds for the Central Universities of China (3132015157)

This publication has 30 references indexed in Scilit: