Over-the-Air Split Machine Learning in Wireless MIMO Networks
- 6 February 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Selected Areas in Communications
- Vol. 41 (4) , 1007-1022
Abstract
In split machine learning (ML), different partitions of a neural network (NN) are executed by different computing nodes, requiring a large amount of communication cost. As over-the-air computation (OAC) can efficiently implement all or part of the computation at the same time of communication, thus by substituting the wireless transmission in the traditional split ML framework with OAC, the communication load can be eased. In this paper, we propose to deploy split ML in a wireless multiple-input multiple-output (MIMO) communication network utilizing the intricate interplay between MIMO-based OAC and NN. The basic procedure of OAC split ML system is first provided, and we show that the inter-layer connection in a NN of any size can be mathematically decomposed into a set of linear precoding and combining transformations over a MIMO channel carrying out multi-stream analog communication. The precoding and combining matrices which are regarded as trainable parameters, and the MIMO channel matrix which are regarded as unknown (implicit) parameters, jointly serve as a fully connected layer of the NN. Most interestingly, the channel estimation procedure can be eliminated by exploiting the MIMO channel reciprocity of the forward and backward propagation, thus greatly saving the system costs and/or further improving its overall efficiency. The generalization of the proposed scheme to the conventional NNs is also introduced, i.e., the widely used convolutional neural networks. We demonstrate its effectiveness under both the static and quasi-static memory channel conditions with comprehensive simulations.Keywords
All Related Versions
Funding Information
- National Key Research and Development Program of China (2020YFB1807100, 2018YFB1801104)
- National Natural Science Foundation of China (U20A20158, 61725104)
- Provincial Key Research and Development Program of Zhejiang (2023C01021)
This publication has 23 references indexed in Scilit:
- BottleNet: A Deep Learning Architecture for Intelligent Mobile Cloud Computing Services2019
- Dynamic Adaptive DNN Surgery for Inference Acceleration on the Edge2019
- JALAD: Joint Accuracy-And Latency-Aware Deep Structure Decoupling for Edge-Cloud Execution2018
- MIMO Over-the-Air Computation for High-Mobility Multimodal SensingIEEE Internet of Things Journal, 2018
- Deep Convolutional Neural Networks and Noisy Images2018
- Deep Learning Based Communication Over the AirIEEE Journal of Selected Topics in Signal Processing, 2017
- NeurosurgeonACM SIGARCH Computer Architecture News, 2017
- Robust Analog Function Computation via Wireless Multiple-Access ChannelsIEEE Transactions on Communications, 2013
- Uncoded Transmission Is Exactly Optimal for a Simple Gaussian “Sensor” NetworkIEEE Transactions on Information Theory, 2008
- A Stochastic Approximation MethodThe Annals of Mathematical Statistics, 1951