A New Perspective of Accelerated Gradient Methods: The Controlled Invariant Manifold Approach

Authors:
  • RGRevati Gunjal
    Revati Gunjal
  • SWSushama Wagh
    Sushama Wagh
  • SNSyed Shadab Nayyer
    Syed Shadab Nayyer
  • ASAlex Stankovic
    Alex Stankovic
  • NSNavdeep M. Singh
    Navdeep M. Singh

Abstract
Gradient Descent (GD) is a ubiquitous algorithm for finding the optimal solution to an optimization problem. For reduced computational complexity, the optimal solution $\mathrm{x^*}$ of the optimization problem must be attained in a minimum number of iterations. For this objective, the paper proposes a genesis of an accelerated gradient algorithm through the controlled dynamical system perspective. The objective of optimally reaching the optimal solution $\mathrm{x^*}$ where $\mathrm{\nabla f(x^*)=0}$ with a given initial condition $\mathrm{x(0)}$ is achieved through control.

This publication has 9 references indexed in Scilit: