Zheng Peng
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Zheng Peng has a total of 53 co-authors. Here are the top co-authors by number of shared publications:
- Kangkang Deng
Department of Mathematics National University of Defense Technology 410073 Changsha CHINA
5 shared publications - Wenxing Zhu
Fuzhou University
5 shared publications Kangkang Deng
Beijing International Center for Mathematical Research, Peking University , Beijing 100871, China
4 shared publications- Qingsong Wang
School of Mathematics and Computational Science, Xiangtan University, Xiangtan, China
4 shared publications Chengcheng Yan
School of Mathematics and Computational Science Xiangtan University 411105 Xiangtan China
3 shared publications- Donghua Wu
Shanghai University
3 shared publications
Latest Publications
- 28 February 2026
- Published by Elsevier in Journal of Computational and Applied Mathematics
- Vol. 485
Adaptive accelerated memoryless quasi-Newton projection method for nonlinear equations with applications
- 28 February 2026
- Published by Elsevier in Journal of Computational and Applied Mathematics
- Vol. 485
Adaptive accelerated memoryless quasi-Newton projection method for nonlinear equations with applications
- 1 January 2026
- Published by Wilmington Scientific Publisher, LLC in Journal of Applied Analysis & Computation
- Vol. 16 (1) , 209-228
A NOVEL RIEMANNIAN CONJUGATE GRADIENT METHOD WITH ITERATION COMPLEXITY GUARANTEES
Conjugate gradient methods are important first-order optimization algorithms both in Euclidean spaces and on Riemannian manifolds. However, while various types of Riemannian conjugate gradient methods have been studied, the iteration complexity analysis remains unknown. This paper proposes a novel Riemannian conjugate gradient method for nonconvex problems with an iteration complexity guarantee. In particular, we introduce a novel restart condition, leading to a function decrease at each iteration. Our method converges to an $ \epsilon $-stationary point with an iteration complexity of $ \mathcal{O}(\epsilon^{-2}) $. To the best of our knowledge, this is the first Riemannian conjugate gradient method with iteration complexity guarantees. Numerical experiments on Rayleigh-quotient and Brockett-cost-function minimization problems demonstrate the efficiency and practical applicability of the proposed method.- 1 January 2026
- Published by Wilmington Scientific Publisher, LLC in Journal of Applied Analysis & Computation
- Vol. 16 (1) , 209-228
A NOVEL RIEMANNIAN CONJUGATE GRADIENT METHOD WITH ITERATION COMPLEXITY GUARANTEES
Conjugate gradient methods are important first-order optimization algorithms both in Euclidean spaces and on Riemannian manifolds. However, while various types of Riemannian conjugate gradient methods have been studied, the iteration complexity analysis remains unknown. This paper proposes a novel Riemannian conjugate gradient method for nonconvex problems with an iteration complexity guarantee. In particular, we introduce a novel restart condition, leading to a function decrease at each iteration. Our method converges to an $ \epsilon $-stationary point with an iteration complexity of $ \mathcal{O}(\epsilon^{-2}) $. To the best of our knowledge, this is the first Riemannian conjugate gradient method with iteration complexity guarantees. Numerical experiments on Rayleigh-quotient and Brockett-cost-function minimization problems demonstrate the efficiency and practical applicability of the proposed method.- Open Access
Accelerated Tensor Completion via Trace-Regularized Fully-Connected Tensor Network
The fully-connected tensor network (FCTN) decomposition has gained prominence in the field of tensor completion owing to its powerful capacity to capture the low-rank characteristics of tensors. Nevertheless, the recovery of local details in the reconstructed tensor still leaves scope for enhancement. In this paper, we propose efficient tensor completion model that incorporates trace regularization within the FCTN decomposition framework. The trace regularization is constructed based on the mode-$k$ unfolding of the FCTN factors combined with periodically modified negative laplacian. The trace regularization promotes the smoothness of the FCTN factors through discrete second-order derivative penalties, thereby enhancing the continuity and local recovery performance of the reconstructed tensor. To solve the proposed model, we develop an efficient algorithm within the proximal alternating minimization (PAM) framework and theoretically prove its convergence. To reduce the runtime of the proposed algorithm, we design an intermediate tensor reuse mechanism that can decrease runtime by 10\%-30\% without affecting image recovery, with more significant improvements for larger-scale data. A comprehensive complexity analysis reveals that the mechanism attains a reduced computational complexity. Numerical experiments demonstrate that the proposed method outperforms existing approaches.
Open AccessAccelerated Tensor Completion via Trace-Regularized Fully-Connected Tensor Network
The fully-connected tensor network (FCTN) decomposition has gained prominence in the field of tensor completion owing to its powerful capacity to capture the low-rank characteristics of tensors. Nevertheless, the recovery of local details in the reconstructed tensor still leaves scope for enhancement. In this paper, we propose efficient tensor completion model that incorporates trace regularization within the FCTN decomposition framework. The trace regularization is constructed based on the mode-$k$ unfolding of the FCTN factors combined with periodically modified negative laplacian. The trace regularization promotes the smoothness of the FCTN factors through discrete second-order derivative penalties, thereby enhancing the continuity and local recovery performance of the reconstructed tensor. To solve the proposed model, we develop an efficient algorithm within the proximal alternating minimization (PAM) framework and theoretically prove its convergence. To reduce the runtime of the proposed algorithm, we design an intermediate tensor reuse mechanism that can decrease runtime by 10\%-30\% without affecting image recovery, with more significant improvements for larger-scale data. A comprehensive complexity analysis reveals that the mechanism attains a reduced computational complexity. Numerical experiments demonstrate that the proposed method outperforms existing approaches.
- Open Access
Accelerated Proximal Dogleg Majorization for Sparse Regularized Quadratic Optimization Problem
This paper addresses the problems of minimizing the sum of a quadratic function and a proximal-friendly nonconvex nonsmooth function. While the existing Proximal Dogleg Opportunistic Majorization (PDOM) algorithm for these problems offers computational efficiency by minimizing opportunistic majorization subproblems along mixed Newton directions and requiring only a single Hessian inversion, its convergence rate is limited due to the nonconvex nonsmooth regularization term, and its theoretical analysis is restricted to local convergence. To overcome these limitations, we firstly propose a novel algorithm named PDOM with extrapolation (PDOME). Its core innovations lie in two key aspects: (1) the integration of an extrapolation strategy into the construction of the hybrid Newton direction, and (2) the enhancement of the line search mechanism. Furthermore, we establish the global convergence of the entire sequence generated by PDOME to a critical point and derive its convergence rate under the Kurdyka-Lojasiewicz (KL) property. Numerical experiments demonstrate that PDOME achieves faster convergence and tends to converge to a better local optimum compared to the original PDOM.
Open AccessAccelerated Proximal Dogleg Majorization for Sparse Regularized Quadratic Optimization Problem
This paper addresses the problems of minimizing the sum of a quadratic function and a proximal-friendly nonconvex nonsmooth function. While the existing Proximal Dogleg Opportunistic Majorization (PDOM) algorithm for these problems offers computational efficiency by minimizing opportunistic majorization subproblems along mixed Newton directions and requiring only a single Hessian inversion, its convergence rate is limited due to the nonconvex nonsmooth regularization term, and its theoretical analysis is restricted to local convergence. To overcome these limitations, we firstly propose a novel algorithm named PDOM with extrapolation (PDOME). Its core innovations lie in two key aspects: (1) the integration of an extrapolation strategy into the construction of the hybrid Newton direction, and (2) the enhancement of the line search mechanism. Furthermore, we establish the global convergence of the entire sequence generated by PDOME to a critical point and derive its convergence rate under the Kurdyka-Lojasiewicz (KL) property. Numerical experiments demonstrate that PDOME achieves faster convergence and tends to converge to a better local optimum compared to the original PDOM.
- Open Access
Neural Network Training via Stochastic Alternating Minimization with Trainable Step Sizes
The training of deep neural networks is inherently a nonconvex optimization problem, yet standard approaches such as stochastic gradient descent (SGD) require simultaneous updates to all parameters, often leading to unstable convergence and high computational cost. To address these issues, we propose a novel method, Stochastic Alternating Minimization with Trainable Step Sizes (SAMT), which updates network parameters in an alternating manner by treating the weights of each layer as a block. By decomposing the overall optimization into sub-problems corresponding to different blocks, this block-wise alternating strategy reduces per-step computational overhead and enhances training stability in nonconvex settings. To fully leverage these benefits, inspired by meta-learning, we proposed a novel adaptive step size strategy to incorporate into the sub-problem solving steps of alternating updates. It supports different types of trainable step sizes, including but not limited to scalar, element-wise, row-wise, and column-wise, enabling adaptive step size selection tailored to each block via meta-learning. We further provide a theoretical convergence guarantee for the proposed algorithm, establishing its optimization soundness. Extensive experiments for multiple benchmarks demonstrate that SAMT achieves better generalization performance with fewer parameter updates compared to state-of-the-art methods, highlighting its effectiveness and potential in neural network optimization.
Open AccessNeural Network Training via Stochastic Alternating Minimization with Trainable Step Sizes
The training of deep neural networks is inherently a nonconvex optimization problem, yet standard approaches such as stochastic gradient descent (SGD) require simultaneous updates to all parameters, often leading to unstable convergence and high computational cost. To address these issues, we propose a novel method, Stochastic Alternating Minimization with Trainable Step Sizes (SAMT), which updates network parameters in an alternating manner by treating the weights of each layer as a block. By decomposing the overall optimization into sub-problems corresponding to different blocks, this block-wise alternating strategy reduces per-step computational overhead and enhances training stability in nonconvex settings. To fully leverage these benefits, inspired by meta-learning, we proposed a novel adaptive step size strategy to incorporate into the sub-problem solving steps of alternating updates. It supports different types of trainable step sizes, including but not limited to scalar, element-wise, row-wise, and column-wise, enabling adaptive step size selection tailored to each block via meta-learning. We further provide a theoretical convergence guarantee for the proposed algorithm, establishing its optimization soundness. Extensive experiments for multiple benchmarks demonstrate that SAMT achieves better generalization performance with fewer parameter updates compared to state-of-the-art methods, highlighting its effectiveness and potential in neural network optimization.
- 28 July 2025
- Published by Springer Nature in Communications on Applied Mathematics and Computation
- Vol. 8 (4) , 1307-1323
A Nonconvex Nonsmooth PADM for the VLSI Placement Problem
- 28 July 2025
- Published by Springer Nature in Communications on Applied Mathematics and Computation
- Vol. 8 (4) , 1307-1323
A Nonconvex Nonsmooth PADM for the VLSI Placement Problem
- Open Access
Faster stochastic cubic regularized Newton methods with momentum
Cubic regularized Newton (CRN) methods have attracted signiffcant research interest because they offer stronger solution guarantees and lower iteration complexity. With the rise of the big-data era, there is growing interest in developing stochastic cubic regularized Newton (SCRN) methods that do not require exact gradient and Hessian evaluations. In this paper, we propose faster SCRN methods that incorporate gradient estimation with small, controlled errors and Hessian estimation with momentum-based variance reduction. These methods are particularly effective for problems where the gradient can be estimated accurately and at low cost, whereas accurate estimation of the Hessian is expensive. Under mild assumptions, we establish the iteration complexity of our SCRN methods by analyzing the descent of a novel potential sequence. Finally, numerical experiments show that our SCRN methods can achieve comparable performance to deterministic CRN methods and vastly outperform ffrst-order methods in terms of both iteration counts and solution quality.
Open AccessFaster stochastic cubic regularized Newton methods with momentum
Cubic regularized Newton (CRN) methods have attracted signiffcant research interest because they offer stronger solution guarantees and lower iteration complexity. With the rise of the big-data era, there is growing interest in developing stochastic cubic regularized Newton (SCRN) methods that do not require exact gradient and Hessian evaluations. In this paper, we propose faster SCRN methods that incorporate gradient estimation with small, controlled errors and Hessian estimation with momentum-based variance reduction. These methods are particularly effective for problems where the gradient can be estimated accurately and at low cost, whereas accurate estimation of the Hessian is expensive. Under mild assumptions, we establish the iteration complexity of our SCRN methods by analyzing the descent of a novel potential sequence. Finally, numerical experiments show that our SCRN methods can achieve comparable performance to deterministic CRN methods and vastly outperform ffrst-order methods in terms of both iteration counts and solution quality.
- 13 May 2025
- Published by Springer Nature in Journal of Scientific Computing
- Vol. 103 (3) , 1-24
Inexact Riemannian Gradient Descent Method for Nonconvex Optimization with Strong Convergence
- 13 May 2025
- Published by Springer Nature in Journal of Scientific Computing
- Vol. 103 (3) , 1-24
Inexact Riemannian Gradient Descent Method for Nonconvex Optimization with Strong Convergence
- Open Access
A Triple-Inertial Accelerated Alternating Optimization Method for Deep Learning Training
The stochastic gradient descent (SGD) algorithm has achieved remarkable success in training deep learning models. However, it has several limitations, including susceptibility to vanishing gradients, sensitivity to input data, and a lack of robust theoretical guarantees. In recent years, alternating minimization (AM) methods have emerged as a promising alternative for model training by employing gradient-free approaches to iteratively update model parameters. Despite their potential, these methods often exhibit slow convergence rates. To address this challenge, we propose a novel Triple-Inertial Accelerated Alternating Minimization (TIAM) framework for neural network training. The TIAM approach incorporates a triple-inertial acceleration strategy with a specialized approximation method, facilitating targeted acceleration of different terms in each sub-problem optimization. This integration improves the efficiency of convergence, achieving superior performance with fewer iterations. Additionally, we provide a convergence analysis of the TIAM algorithm, including its global convergence properties and convergence rate. Extensive experiments validate the effectiveness of the TIAM method, showing significant improvements in generalization capability and computational efficiency compared to existing approaches, particularly when applied to the rectified linear unit (ReLU) and its variants.
Open AccessA Triple-Inertial Accelerated Alternating Optimization Method for Deep Learning Training
The stochastic gradient descent (SGD) algorithm has achieved remarkable success in training deep learning models. However, it has several limitations, including susceptibility to vanishing gradients, sensitivity to input data, and a lack of robust theoretical guarantees. In recent years, alternating minimization (AM) methods have emerged as a promising alternative for model training by employing gradient-free approaches to iteratively update model parameters. Despite their potential, these methods often exhibit slow convergence rates. To address this challenge, we propose a novel Triple-Inertial Accelerated Alternating Minimization (TIAM) framework for neural network training. The TIAM approach incorporates a triple-inertial acceleration strategy with a specialized approximation method, facilitating targeted acceleration of different terms in each sub-problem optimization. This integration improves the efficiency of convergence, achieving superior performance with fewer iterations. Additionally, we provide a convergence analysis of the TIAM algorithm, including its global convergence properties and convergence rate. Extensive experiments validate the effectiveness of the TIAM method, showing significant improvements in generalization capability and computational efficiency compared to existing approaches, particularly when applied to the rectified linear unit (ReLU) and its variants.