Stability and Performance Limits of Adaptive Primal-Dual Networks
Authors:
- 23 March 2015
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Signal Processing
- Vol. 63 (11) , 2888-2903
Abstract
This paper studies distributed primal-dual strategies for adaptation and learning over networks from streaming data. Two first-order methods are considered based on the Arrow-Hurwicz (AH) and augmented Lagrangian (AL) techniques. Several revealing results are discovered in relation to the performance and stability of these strategies when employed over adaptive networks. The conclusions establish that the advantages that these methods exhibit for deterministic optimization problems do not necessarily carry over to stochastic optimization problems. It is found that they have narrower stability ranges and worse steady-state mean-square-error performance than primal methods of the consensus and diffusion type. It is also found that the AH technique can become unstable under a partial observation model, while the other techniques are able to recover the unknown under this scenario. A method to enhance the performance of AL strategies is proposed by tying the selection of the step-size to their regularization parameter. It is shown that this method allows the AL algorithm to approach the performance of consensus and diffusion strategies but that it remains less stable than these other strategies.Keywords
All Related Versions
Funding Information
- NSF (CCF-1011918, ECCS-1407712)
This publication has 43 references indexed in Scilit:
- Distributed computation of the Fiedler vector with application to topology inference in ad hoc networksSignal Processing, 2012
- Distributed Optimization and Statistical Learning via the Alternating Direction Method of MultipliersFoundations and Trends in Machine Learning, 2011
- Distributed Stochastic Subgradient Projection Algorithms for Convex OptimizationJournal of Optimization Theory and Applications, 2010
- Performance Analysis of the Consensus-Based Distributed LMS AlgorithmEURASIP Journal on Advances in Signal Processing, 2009
- A new incremental affine projection-based adaptive algorithm for distributed networksSignal Processing, 2008
- Parallel algorithms for variational inequalities over the Cartesian product of the intersections of the fixed point sets of nonexpansive mappingsJournal of Approximation Theory, 2008
- Fastest Mixing Markov Chain on a GraphSIAM Review, 2004
- The Laplacian Spectrum of a Graph IISIAM Journal on Discrete Mathematics, 1994
- Eigenvalues of the Laplacian of a graph∗Linear and Multilinear Algebra, 1985
- A new matrix product and its applications in partitioned matrix differentiationStatistica Neerlandica, 1972