Robustness of Iteratively Pre-Conditioned Gradient-Descent Method: The Case of Distributed Linear Regression Problem
- 17 December 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Control Systems Letters
- Vol. 5 (6) , 2180-2185
Abstract
This paper considers the problem of multi-agent distributed linear regression in the presence of system noises. In this problem, the system comprises multiple agents wherein each agent locally observes a set of data points, and the agents’ goal is to compute a linear model that best fits the collective data points observed by all the agents. We consider a server-based distributed architecture where the agents interact with a common server to solve the problem; however, the server cannot access the agents’ data points. We consider a practical scenario wherein the system either has observation noise, i.e., the data points observed by the agents are corrupted, or has process noise, i.e., the computations performed by the server and the agents are corrupted. In noise-free systems, the recently proposed distributed linear regression algorithm, named the Iteratively Pre-conditioned Gradient-descent (IPG) method, has been claimed to converge faster than related methods. In this paper, we study the robustness of the IPG method, against both the observation noise and the process noise. We empirically show that the robustness of the IPG method compares favorably to the state-of-the-art algorithms.Keywords
All Related Versions
Funding Information
- Petroleum Institute, Khalifa University of Science and Technology, Abu Dhabi, UAE through the PSIM Project
- ARL (W911NF-17-2-0196)
This publication has 14 references indexed in Scilit:
- Linear Hypothesis Testing in Dense High-Dimensional Linear ModelsJournal of the American Statistical Association, 2018
- Scaling Distributed Machine Learning with the Parameter Server2014
- First-order methods of smooth convex optimization with inexact oracleMathematical Programming, 2013
- Matrix Analysis2012
- Application of machine learning techniques for supply chain demand forecastingEuropean Journal of Operational Research, 2008
- Class Noise vs. Attribute Noise: A Quantitative StudyArtificial Intelligence Review, 2004
- Iterative Methods for Optimization1999
- Finite precision error analysis of neural network hardware implementationsIEEE Transactions on Computers, 1993
- Effects of quantization noise in digital filters1966
- Some methods of speeding up the convergence of iteration methodsUSSR Computational Mathematics and Mathematical Physics, 1964