Decentralized Kernel Ridge Regression Based on Data-Dependent Random Feature
- 12 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (5) , 7945-7954
Abstract
Random feature (RF) has been widely used for node consistency in decentralized kernel ridge regression (KRR). Currently, the consistency is guaranteed by imposing constraints on coefficients of features, necessitating that the RFs on different nodes are identical. However, in many applications, data on different nodes vary significantly on the number or distribution, which calls for adaptive and data-dependent methods that generate different RFs. To tackle the essential difficulty, we propose a new decentralized KRR algorithm that pursues consensus on decision functions, which allows great flexibility and well adapts data on nodes. The convergence is rigorously given, and the effectiveness is numerically verified: by capturing the characteristics of the data on each node, while maintaining the same communication costs as other methods, we achieved an average regression accuracy improvement of 25.5% across six real-world datasets.Keywords
All Related Versions
Funding Information
- Shanghai Municipal Science and Technology Research Program (22511105600)
- National Natural Science Foundation of China (62376155, 62376153)
- Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102)
This publication has 31 references indexed in Scilit:
- BFOD: Blockchain-Based Privacy Protection and Security Sharing Scheme of Flight Operation DataIEEE Internet of Things Journal, 2023
- Decentralized learning over a network with Nyström approximation using SGDApplied and Computational Harmonic Analysis, 2023
- Distributed Online One-Class Support Vector Machine for Anomaly Detection Over NetworksIEEE Transactions on Cybernetics, 2018
- A Fundamental Tradeoff Between Computation and Communication in Distributed ComputingIEEE Transactions on Information Theory, 2017
- A Scalable Framework for Wireless Distributed ComputingIEEE/ACM Transactions on Networking, 2017
- Big Data Privacy in Biomedical ResearchIEEE Transactions on Big Data, 2016
- Design of a low-cost Wireless Sensor Network with UAV mobile node for agricultural applicationsComputers and Electronics in Agriculture, 2015
- Random Laplace Feature Maps for Semigroup Kernels on Histograms2014
- Finding the Missing Link for Big Biomedical DataJAMA, 2014
- iMapReduce: A Distributed Computing Framework for Iterative ComputationJournal of Grid Computing, 2012