ZPZheng Peng

Zheng Peng

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Zheng Peng has a total of 53 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • A NOVEL RIEMANNIAN CONJUGATE GRADIENT METHOD WITH ITERATION COMPLEXITY GUARANTEES

    by Juan Zhou, Kangkang Deng, Hongxia Wang, Zheng Peng

    Conjugate gradient methods are important first-order optimization algorithms both in Euclidean spaces and on Riemannian manifolds. However, while various types of Riemannian conjugate gradient methods have been studied, the iteration complexity analysis remains unknown. This paper proposes a novel Riemannian conjugate gradient method for nonconvex problems with an iteration complexity guarantee. In particular, we introduce a novel restart condition, leading to a function decrease at each iteration. Our method converges to an $ \epsilon $-stationary point with an iteration complexity of $ \mathcal{O}(\epsilon^{-2}) $. To the best of our knowledge, this is the first Riemannian conjugate gradient method with iteration complexity guarantees. Numerical experiments on Rayleigh-quotient and Brockett-cost-function minimization problems demonstrate the efficiency and practical applicability of the proposed method.


  • Open Access

    Accelerated Tensor Completion via Trace-Regularized Fully-Connected Tensor Network

    by Wenchao Xie, Qingsong Wang, Chengcheng Yan, Zheng Peng

    The fully-connected tensor network (FCTN) decomposition has gained prominence in the field of tensor completion owing to its powerful capacity to capture the low-rank characteristics of tensors. Nevertheless, the recovery of local details in the reconstructed tensor still leaves scope for enhancement. In this paper, we propose efficient tensor completion model that incorporates trace regularization within the FCTN decomposition framework. The trace regularization is constructed based on the mode-$k$ unfolding of the FCTN factors combined with periodically modified negative laplacian. The trace regularization promotes the smoothness of the FCTN factors through discrete second-order derivative penalties, thereby enhancing the continuity and local recovery performance of the reconstructed tensor. To solve the proposed model, we develop an efficient algorithm within the proximal alternating minimization (PAM) framework and theoretically prove its convergence. To reduce the runtime of the proposed algorithm, we design an intermediate tensor reuse mechanism that can decrease runtime by 10\%-30\% without affecting image recovery, with more significant improvements for larger-scale data. A comprehensive complexity analysis reveals that the mechanism attains a reduced computational complexity. Numerical experiments demonstrate that the proposed method outperforms existing approaches.

  • Open Access

    Accelerated Proximal Dogleg Majorization for Sparse Regularized Quadratic Optimization Problem

    by Feifei Zhao, Qingsong Wang, Mingcai Ding, Zheng Peng

    This paper addresses the problems of minimizing the sum of a quadratic function and a proximal-friendly nonconvex nonsmooth function. While the existing Proximal Dogleg Opportunistic Majorization (PDOM) algorithm for these problems offers computational efficiency by minimizing opportunistic majorization subproblems along mixed Newton directions and requiring only a single Hessian inversion, its convergence rate is limited due to the nonconvex nonsmooth regularization term, and its theoretical analysis is restricted to local convergence. To overcome these limitations, we firstly propose a novel algorithm named PDOM with extrapolation (PDOME). Its core innovations lie in two key aspects: (1) the integration of an extrapolation strategy into the construction of the hybrid Newton direction, and (2) the enhancement of the line search mechanism. Furthermore, we establish the global convergence of the entire sequence generated by PDOME to a critical point and derive its convergence rate under the Kurdyka-Lojasiewicz (KL) property. Numerical experiments demonstrate that PDOME achieves faster convergence and tends to converge to a better local optimum compared to the original PDOM.

  • Open Access

    Neural Network Training via Stochastic Alternating Minimization with Trainable Step Sizes

    by Chengcheng Yan, Jiawei Xu, Zheng Peng, Qingsong Wang

    The training of deep neural networks is inherently a nonconvex optimization problem, yet standard approaches such as stochastic gradient descent (SGD) require simultaneous updates to all parameters, often leading to unstable convergence and high computational cost. To address these issues, we propose a novel method, Stochastic Alternating Minimization with Trainable Step Sizes (SAMT), which updates network parameters in an alternating manner by treating the weights of each layer as a block. By decomposing the overall optimization into sub-problems corresponding to different blocks, this block-wise alternating strategy reduces per-step computational overhead and enhances training stability in nonconvex settings. To fully leverage these benefits, inspired by meta-learning, we proposed a novel adaptive step size strategy to incorporate into the sub-problem solving steps of alternating updates. It supports different types of trainable step sizes, including but not limited to scalar, element-wise, row-wise, and column-wise, enabling adaptive step size selection tailored to each block via meta-learning. We further provide a theoretical convergence guarantee for the proposed algorithm, establishing its optimization soundness. Extensive experiments for multiple benchmarks demonstrate that SAMT achieves better generalization performance with fewer parameter updates compared to state-of-the-art methods, highlighting its effectiveness and potential in neural network optimization.

  • Open Access

    Faster stochastic cubic regularized Newton methods with momentum

    by Yiming Yang, Chuan He, Xiao Wang, Zheng Peng

    Cubic regularized Newton (CRN) methods have attracted signiffcant research interest because they offer stronger solution guarantees and lower iteration complexity. With the rise of the big-data era, there is growing interest in developing stochastic cubic regularized Newton (SCRN) methods that do not require exact gradient and Hessian evaluations. In this paper, we propose faster SCRN methods that incorporate gradient estimation with small, controlled errors and Hessian estimation with momentum-based variance reduction. These methods are particularly effective for problems where the gradient can be estimated accurately and at low cost, whereas accurate estimation of the Hessian is expensive. Under mild assumptions, we establish the iteration complexity of our SCRN methods by analyzing the descent of a novel potential sequence. Finally, numerical experiments show that our SCRN methods can achieve comparable performance to deterministic CRN methods and vastly outperform ffrst-order methods in terms of both iteration counts and solution quality.

  • Open Access

    A Triple-Inertial Accelerated Alternating Optimization Method for Deep Learning Training

    by Chengcheng Yan, Jiawei Xu, Qingsong Wang, Zheng Peng

    The stochastic gradient descent (SGD) algorithm has achieved remarkable success in training deep learning models. However, it has several limitations, including susceptibility to vanishing gradients, sensitivity to input data, and a lack of robust theoretical guarantees. In recent years, alternating minimization (AM) methods have emerged as a promising alternative for model training by employing gradient-free approaches to iteratively update model parameters. Despite their potential, these methods often exhibit slow convergence rates. To address this challenge, we propose a novel Triple-Inertial Accelerated Alternating Minimization (TIAM) framework for neural network training. The TIAM approach incorporates a triple-inertial acceleration strategy with a specialized approximation method, facilitating targeted acceleration of different terms in each sub-problem optimization. This integration improves the efficiency of convergence, achieving superior performance with fewer iterations. Additionally, we provide a convergence analysis of the TIAM algorithm, including its global convergence properties and convergence rate. Extensive experiments validate the effectiveness of the TIAM method, showing significant improvements in generalization capability and computational efficiency compared to existing approaches, particularly when applied to the rectified linear unit (ReLU) and its variants.