An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization
Preprint
Abstract
This paper focuses on the minimization of a sum of a twice continuously differentiable function $f$ and a nonsmooth convex function. We propose an inexact regularized proximal Newton method by an approximation of the Hessian $\nabla^2\!f(x)$ involving the $\varrho$th power of the KKT residual. For $\varrho=0$, we demonstrate the global convergence of the iterate sequence for the KL objective function and its $R$-linear convergence rate for the KL objective function of exponent $1/2$. For $\varrho\in(0,1)$, we establish the global convergence of the iterate sequence and its superlinear convergence rate of order $q(1\!+\!\varrho)$ under an assumption that cluster points satisfy a local H\"{o}lderian local error bound of order $q\in(\max(\varrho,\frac{1}{1+\varrho}),1]$ on the strong stationary point set; and when cluster points satisfy a local error bound of order $q>1+\varrho$ on the common stationary point set, we also obtain the global convergence of the iterate sequence, and its superlinear convergence rate of order $\frac{(q-\varrho)^2}{q}$ if $q>\frac{2\varrho+1+\sqrt{4\varrho+1}}{2}$. A dual semismooth Newton augmented Lagrangian method is developed for seeking an inexact minimizer of subproblem. Numerical comparisons with two state-of-the-art methods on $\ell_1$-regularized Student's $t$-regression, group penalized Student's $t$-regression, and nonconvex image restoration confirm the efficiency of the proposed method.
Keywords
All Related Versions
This publication has 39 references indexed in Scilit:
- Inexact proximal Newton methods for self-concordant functionsMathematical Methods of Operations Research, 2016
- Proximal alternating linearized minimization for nonconvex and nonsmooth problemsMathematical Programming, 2013
- Robust inversion, dimensionality reduction, and randomized samplingMathematical Programming, 2012
- Convergence Properties of the Regularized Newton Method for the Unconstrained Nonconvex OptimizationApplied Mathematics & Optimization, 2009
- A coordinate gradient descent method for nonsmooth separable minimizationMathematical Programming, 2007
- Error bounds for systems of lower semicontinuous functions in Asplund spacesMathematical Programming, 2007
- On the convergence of the proximal algorithm for nonsmooth functions involving analytic featuresMathematical Programming, 2007
- Local behavior of an iterative framework for generalized equations with nonisolated solutionsMathematical Programming, 2002
- Error bounds and convergence analysis of feasible descent methods: a general approachAnnals of Operations Research, 1993
- A nonsmooth version of Newton's methodMathematical Programming, 1993