Decentralized RLS with Data-Adaptive Censoring for Regressions over Large-Scale Networks
Authors:
- Zifeng WangZWZifeng Wang
- Zheng YuZYZheng Yu
- Qing LingQLQing Ling
- Dimitris BerberidisDBDimitris Berberidis
- Georgios B. GiannakisGGGeorgios B. Giannakis
Preprint
Abstract
The deluge of networked data motivates the development of algorithms for computation- and communication-efficient information processing. In this context, three data-adaptive censoring strategies are introduced to considerably reduce the computation and communication overhead of decentralized recursive least-squares (D-RLS) solvers. The first relies on alternating minimization and the stochastic Newton iteration to minimize a network-wide cost, which discards observations with small innovations. In the resultant algorithm, each node performs local data-adaptive censoring to reduce computations, while exchanging its local estimate with neighbors so as to consent on a network-wide solution. The communication cost is further reduced by the second strategy, which prevents a node from transmitting its local estimate to neighbors when the innovation it induces to incoming data is minimal. In the third strategy, not only transmitting, but also receiving estimates from neighbors is prohibited when data-adaptive censoring is in effect. For all strategies, a simple criterion is provided for selecting the threshold of innovation to reach a prescribed average data reduction. The novel censoring-based (C)D-RLS algorithms are proved convergent to the optimal argument in the mean-square deviation sense. Numerical experiments validate the effectiveness of the proposed algorithms in reducing computation and communication overhead.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Stochastic Approximation vis-a-vis Online Learning for Big Data Analytics [Lecture Notes]IEEE Signal Processing Magazine, 2014
- Convex Optimization for Big Data: Scalable, randomized, and parallel algorithms for big data analyticsIEEE Signal Processing Magazine, 2014
- Resource allocation in a network-based cloud computing environment: design challengesIEEE Communications Magazine, 2013
- Performance of a Distributed Stochastic Approximation AlgorithmIEEE Transactions on Information Theory, 2013
- Distributed Recursive Least-Squares: Stability and Performance AnalysisIEEE Transactions on Signal Processing, 2012
- Sensor-Centric Data Reduction for Estimation With WSNs via Censoring and QuantizationIEEE Transactions on Signal Processing, 2011
- Distributed Recursive Least-Squares for Consensus-Based In-Network Adaptive EstimationIEEE Transactions on Signal Processing, 2009
- Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance AnalysisIEEE Transactions on Signal Processing, 2008
- Decentralized Detection With Censoring SensorsIEEE Transactions on Signal Processing, 2008
- Censoring sensors: a low-communication-rate scheme for distributed detectionIEEE Transactions on Aerospace and Electronic Systems, 1996