DYDeming Yuan

Deming Yuan

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Deming Yuan has a total of 64 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Generic Frameworks for Distributed Functional Optimization and Learning Over Time-Varying Networks

    by Zhan Yu, Zhongjie Shi, Deming Yuan, Daniel W. C. Ho

    In this paper, we establish a distributed functional optimization (DFO) theory over time-varying networks. The vast majority of existing distributed optimization theories are developed based on Euclidean decision variables. However, for many scenarios in machine learning and statistical learning, such as reproducing kernel spaces or probability measure spaces where functions or probability measures serve as the fundamental variables, the development of existing distributed optimization theories exhibit obvious theoretical and technical deficiencies. To address these issues, we develop a novel general DFO theory on Banach spaces, allowing functional learning problems in the aforementioned scenarios to be incorporated into our framework for resolution. We study both convex and nonconvex DFO problems. We propose a distributed functional mirror descent algorithm and a distributed functional gradient descent algorithm to solve them. Comprehensive convergence theory of these methods is rigorously established and satisfactory convergence rates are fully derived. The work provides generic analytical frameworks for DFO. The established theory is shown to have crucial application value in kernel-based distributed learning theory over time-varying networks.

  • Distributed Online Stochastic Convex-Concave Optimization: Dynamic Regret Analyses Under Single and Multiple Consensus Steps

    by Wentao Zhang, Baoyong Zhang, Deming Yuan, Shengyuan Xu, Vincent K. N. Lau

    This paper considers the distributed online convex-concave optimization with constraint sets over a multiagent network, in which each agent autonomously generates a series of decision pairs through a designable mechanism to cooperatively minimize the global loss function. To this end, under no-Euclidean distance metrics, we propose a distributed online stochastic mirror descent convex-concave optimization algorithm with timevarying predictive mappings. Taking dynamic saddle point regret as a performance metric, it is proved that the proposed algorithm achieves the regret upper-bound in $O({\rm max}\{T^{θ1} , T^{θ2}(1 + V_T )\})$ for the general convex-concave loss function, where θ1, θ2 ∈ (0, 1) are the tuning parameters, T is the total iteration time, and VT is the path-variation. Surely, this algorithm guarantees the sublinear convergence, provided that VT is sublinear. Moreover, aiming to achieve better convergence, we further investigate a variant of this algorithm by employing the multiple consensus technique. The obtained results show that the appropriate setting can effectively tighten the regret bound to a certain extent. Finally, the efficacy of the proposed algorithms is validated and compared through the simulation example of a target tracking problem.


  • High-Probability Convergence Theory for Distributed Composite Optimization with Sub-Weibull Noises

    by Zhan Yu, Zhongjie Shi, Deming Yuan

    With the rapid development of distributed optimization (DO) theory, the distributed stochastic gradient methods (DSGMs) occupy an important position. Although the theory of different DSGMs has been widely established, the main-stream results of existing work are still derived under the condition of light-tailed stochastic gradient noises. Increasing examples from various fields, indicate that, the light-tailed noise model is overly idealized in many practical instances, failing to capture the complexity and variability of noises in real-world scenarios, such as the presence of outliers or extreme values from data science and statistical learning. To address this issue, we propose a new DO framework that incorporates stochastic gradients under sub-Weibull randomness. We study a distributed composite stochastic mirror descent scheme with sub-Weibull gradient noise (DCSMD-SW) for solving a convex distributed composite optimization (DCO) problem over the time-varying multi-agent network. By investigating sub-Weibull randomness in DCSMD for the first time, we show that the algorithm is applicable in some common heavier-tailed noise environments while also guaranteeing good convergence properties. We comprehensively study the convergence performance of DCSMD-SW. Satisfactory high-probability convergence rates are derived for DCSMD-SW without any smoothness requirement. The work also offers a unified analytical framework for several critical cases of both algorithms and noise environments.

  • Distributed Optimization by Network Flows With Spatio-Temporal Compression

    by Zihao Ren, Lei Wang, Xinlei Yi, Xi Wang, Deming Yuan, Tao Yang, Zhengguang Wu, Guodong Shi

    Several data compressors have been proposed in distributed optimization frameworks of network systems to reduce communication overhead in large-scale applications. In this paper, we demonstrate that effective information compression may occur over time or space during sequences of node communications in distributed algorithms, leading to the concept of spatio-temporal compressors. This abstraction classifies existing compressors and inspires new compressors as spatio-temporal compressors, with their effectiveness described by constructive stability criteria from nonlinear system theory. Subsequently, we incorporate these spatio-temporal compressors directly into standard continuous-time consensus flows and distributed primal-dual flows, establishing conditions ensuring exponential convergence. Additionally, we introduce a novel observer-based distributed primal-dual continuous flow integrated with spatio-temporal compressors, which provides broader convergence conditions. These continuous flows achieve exponential convergence to the global optimum when the objective function is strongly convex and can be discretized using Euler approximations. Finally, numerical simulations illustrate the versatility of the proposed spatio-temporal compressors and verify the convergence of algorithms.


  • Distributed Composite Optimization With Distributed Composite Block Mirror Descent

    by Zhan Yu, Daniel W. C. Ho, Deming Yuan, Zhongjie Shi

    This article investigates a class of large-scale distributed nonsmooth composite optimization problems over time-varying multiagent networks. Specifically, the decision space, which can be split into several blocks of convex set, is considered. Each node, endowed with a private nonsmooth cost function and a regularization function, aims to minimize the sum of all local functions across the network. We propose a novel distributed composite block mirror descent (DCBMD) method, where each node performs information communication with other agents and executes a block regularized mirror descent in each iteration. In contrast to existing work on distributed composite optimization, for the decision space with block structure, we do not require the projection to be operated on the whole decision space. Instead, in each step, a distributed projection procedure induced by a composite mirror descent scheme is performed on only one randomly selected block, significantly saving the iteration cost. The explicit formulation of the convergence bound depending on random projection probabilities and network parameters is achieved. An optimal convergence rate $\mathcal {O}(1/\sqrt{T})$ is rigorously derived. The DCBMD provides a generic framework for different projection-based distributed algorithms.


  • Distributed Online Optimization With Differential Privacy Over Time-Varying Unbalanced Digraphs

    by Mingcheng Dai, Baoyong Zhang, Deming Yuan, Xianju Fang

    This article focuses on the online distributed optimization with privacy protection in time-varying unbalanced networks. We consider the case where there exist potential passive attackers in the network, having access to all communication channels. The attackers attempt to deduce the privacy of participating nodes. In this case, the differential privacy approach is leveraged to enable privacy protection. Based on this privacy-preserving approach and the stochastic subgradient method, a novel differentially private distributed online stochastic subgradient descent (DP-DOSSD) algorithm is devised for addressing the considered problem. Unlike the existing algorithms that require the weight matrices to be doubly stochastic, our algorithm only depends on two different weight matrices that are column-stochastic and row-stochastic, respectively. Moreover, a stochastic subgradient rescaling technique is adopted to tackle the unbalancedness of time-varying directed networks. It proves that our algorithm not only guarantees ϵ-differential privacy but also establishes an expected regret of order O(√T) in convex settings, where ϵ and T are the privacy level and the time horizon, respectively. The established result matches the optimal regret bound derived by state-of-the-art algorithms. The fundamental tradeoff between convergence performance and privacy level is also studied. Finally, simulation results for the sensor network-based online distributed estimation problem and the distributed online ridge regression problem are provided to confirm the effectiveness of our approach.


  • Static and Dynamic Regrets for Online Distributed Composite Optimization: Edge-Based Dynamic Event-Triggered Algorithms

    by Menghui Xiong, Chunyu Yang, Jianguo Zhao, Baoyong Zhang, Deming Yuan

    This article addresses the online distributed convex-constrained composite optimization problem on an unbalanced directed multiagent network with limited bandwidth. The distinctive feature is that each agent's objective function is composed of two components: a loss function that changes over iterations and a regularizer. To cope with potential network bandwidth constraints, an edge-based dynamic event-triggered communication protocol with a control factor is employed in the link edges between agents. Based on the proximal operator, we propose two algorithms that only require the weight matrix to be row-stochastic: an edge-based dynamic event-triggered online distributed proximal gradient descent algorithm and a variant that provides an approximate solution for the proximal step. In addition, we analyze the corresponding static and dynamic regret bounds for two algorithms. Under reasonable assumptions and parameter settings, both algorithms achieve a static regret bound of order $\mathcal {O}(\sqrt{T})$, and a dynamic regret bound of order $\mathcal {O}(\sqrt{T}(1+{\mathcal {D}_{T}}))$, respectively (where $T$ is the time horizon and $\mathcal {D}_{T}$ is path-variation level). Furthermore, if $\mathcal {D}_{T}$ can be estimated in advance, two algorithms attain a dynamic regret bound of order $\mathcal {O}(\sqrt{T(1+{\mathcal {D}_{T}})})$. Finally, simulations of the online distributed least absolute shrinkage and selection operator problem are presented to validate the effectiveness of our developed algorithms.