Tian Ding
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Tian Ding has a total of 8 co-authors. Here are the top co-authors by number of shared publications:
- Xiaojun Yuan
University of Electronic Science and Technology of China
5 shared publications - Ruoyu Sun
Department of ISE Coordinated Science Laboratory University of Illinois at Urbana- Champaign Urbana IL
2 shared publications - Soung Chang Liew
The Chinese University of Hong Kong, Hong Kong, Hong Kong
2 shared publications Dawei Li
1 shared publication- Dawei Li
Department of Industrial and Enterprise Systems Engineering and Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801
1 shared publication Feifei Gao
Tsinghua National Laboratory for Information Science and Technology, Beijing, 100084, China
1 shared publication
Latest Publications
- 19 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Communications
- Vol. 71 (8) , 4561-4576
Federated Learning With Lossy Distributed Source Coding: Analysis and Optimization
Recently, federated learning (FL), which replaces data sharing with model sharing, has emerged as an efficient and privacy-friendly machine learning (ML) paradigm. One of the main challenges in FL is the huge communication cost for model aggregation. Many compression/quantization schemes have been proposed to reduce the communication cost for model aggregation. However, the following question remains unanswered: What is the fundamental trade-off between the communication cost and the FL convergence performance? In this paper, we manage to answer this question. Specifically, we first put forth a general framework for model aggregation performance analysis based on the rate-distortion theory. Under the proposed analysis framework, we derive an inner bound of the rate-distortion region of model aggregation. We then conduct an FL convergence analysis to connect the aggregation distortion and the FL convergence performance. We formulate an aggregation distortion minimization problem to improve the FL convergence performance. Two algorithms are developed to solve the above problem. Numerical results on aggregation distortion, convergence performance, and communication cost demonstrate that the baseline model aggregation schemes still have great potential for further improvement.
- 19 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Communications
- Vol. 71 (8) , 4561-4576
Federated Learning With Lossy Distributed Source Coding: Analysis and Optimization
Recently, federated learning (FL), which replaces data sharing with model sharing, has emerged as an efficient and privacy-friendly machine learning (ML) paradigm. One of the main challenges in FL is the huge communication cost for model aggregation. Many compression/quantization schemes have been proposed to reduce the communication cost for model aggregation. However, the following question remains unanswered: What is the fundamental trade-off between the communication cost and the FL convergence performance? In this paper, we manage to answer this question. Specifically, we first put forth a general framework for model aggregation performance analysis based on the rate-distortion theory. Under the proposed analysis framework, we derive an inner bound of the rate-distortion region of model aggregation. We then conduct an FL convergence analysis to connect the aggregation distortion and the FL convergence performance. We formulate an aggregation distortion minimization problem to improve the FL convergence performance. Two algorithms are developed to solve the above problem. Numerical results on aggregation distortion, convergence performance, and communication cost demonstrate that the baseline model aggregation schemes still have great potential for further improvement.
- 1 November 2022
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Mathematics of Operations Research
- Vol. 47 (4) , 2784-2814
Suboptimal Local Minima Exist for Wide Neural Networks with Smooth Activations
Does a large width eliminate all suboptimal local minima for neural nets? An affirmative answer was given by a classic result published in 1995 for one-hidden-layer wide neural nets with a sigmoid activation function, but this result has not been extended to the multilayer case. Recently, it was shown that, with piecewise linear activations, suboptimal local minima exist even for wide nets. Given the classic positive result on smooth activation and the negative result on nonsmooth activations, an interesting open question is: Does a large width eliminate all suboptimal local minima for deep neural nets with smooth activation? In this paper, we give a largely negative answer to this question. Specifically, we prove that, for neural networks with generic input data and smooth nonlinear activation functions, suboptimal local minima can exist no matter how wide the network is (as long as the last hidden layer has at least two neurons). Therefore, the classic result of no suboptimal local minimum for a one-hidden-layer network does not hold. Whereas this classic result assumes sigmoid activation, our counterexample covers a large set of activation functions (dense in the set of continuous functions), indicating that the limitation is not a result of the specific activation. Together with recent progress on piecewise linear activations, our result indicates that suboptimal local minima are common for wide neural nets.
- 1 November 2022
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Mathematics of Operations Research
- Vol. 47 (4) , 2784-2814
Suboptimal Local Minima Exist for Wide Neural Networks with Smooth Activations
Does a large width eliminate all suboptimal local minima for neural nets? An affirmative answer was given by a classic result published in 1995 for one-hidden-layer wide neural nets with a sigmoid activation function, but this result has not been extended to the multilayer case. Recently, it was shown that, with piecewise linear activations, suboptimal local minima exist even for wide nets. Given the classic positive result on smooth activation and the negative result on nonsmooth activations, an interesting open question is: Does a large width eliminate all suboptimal local minima for deep neural nets with smooth activation? In this paper, we give a largely negative answer to this question. Specifically, we prove that, for neural networks with generic input data and smooth nonlinear activation functions, suboptimal local minima can exist no matter how wide the network is (as long as the last hidden layer has at least two neurons). Therefore, the classic result of no suboptimal local minimum for a one-hidden-layer network does not hold. Whereas this classic result assumes sigmoid activation, our counterexample covers a large set of activation functions (dense in the set of continuous functions), indicating that the limitation is not a result of the specific activation. Together with recent progress on piecewise linear activations, our result indicates that suboptimal local minima are common for wide neural nets.
- 1 August 2022
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Optimization
- Vol. 32 (3) , 1728-1758
On the Benefit of Width for Neural Networks: Disappearance of Basins
Wide networks are often believed to have a nice optimization landscape, but what rigorous results can we prove? To understand the benefit of width, it is important to identify the difference betwee...
- 1 August 2022
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Optimization
- Vol. 32 (3) , 1728-1758
On the Benefit of Width for Neural Networks: Disappearance of Basins
Wide networks are often believed to have a nice optimization landscape, but what rigorous results can we prove? To understand the benefit of width, it is important to identify the difference betwee...
- 16 May 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 18 (7) , 3569-3582
Sparsity Learning-Based Multiuser Detection in Grant-Free Massive-Device Multiple Access
In this work, we study the multiuser detection (MUD) problem for a grant-free massive-device multiple access (MaDMA) system, where a large number of single-antenna user devices transmit sporadic data to a multi-antenna base station (BS). Specifically, we put forth two MUD schemes, termed random sparsity learning multiuser detection (RSL-MUD) and structured sparsity learning multiuser detection (SSL-MUD) for the time-slotted and non-time-slotted grant-free MaDMA systems, respectively. In RSL-MUD, active users generate and transmit data packets with random sparsity. In SSL-MUD, we introduce a sliding-window-based detection framework, and the user signals in each observation window naturally exhibit structured sparsity. We show that by exploiting the sparsity embedded in the user signals, we can recover the user activity state, the channel, and the user data in a single phase, without using pilot signals for channel estimation and/or active user identification. To this end, we develop a message-passing-based statistical inference framework for the BS to blindly detect the user data without any prior knowledge of the identities and the channel state information (CSI) of active users. Simulation results show that our RSL-MUD and SSL-MUD schemes significantly outperform their counterpart schemes in both reducing the transmission overhead and improving the error behavior of the system.
- 16 May 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 18 (7) , 3569-3582
Sparsity Learning-Based Multiuser Detection in Grant-Free Massive-Device Multiple Access
In this work, we study the multiuser detection (MUD) problem for a grant-free massive-device multiple access (MaDMA) system, where a large number of single-antenna user devices transmit sporadic data to a multi-antenna base station (BS). Specifically, we put forth two MUD schemes, termed random sparsity learning multiuser detection (RSL-MUD) and structured sparsity learning multiuser detection (SSL-MUD) for the time-slotted and non-time-slotted grant-free MaDMA systems, respectively. In RSL-MUD, active users generate and transmit data packets with random sparsity. In SSL-MUD, we introduce a sliding-window-based detection framework, and the user signals in each observation window naturally exhibit structured sparsity. We show that by exploiting the sparsity embedded in the user signals, we can recover the user activity state, the channel, and the user data in a single phase, without using pilot signals for channel estimation and/or active user identification. To this end, we develop a message-passing-based statistical inference framework for the BS to blindly detect the user data without any prior knowledge of the identities and the channel state information (CSI) of active users. Simulation results show that our RSL-MUD and SSL-MUD schemes significantly outperform their counterpart schemes in both reducing the transmission overhead and improving the error behavior of the system.
- 24 July 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 67 (10) , 10081-10086
Algorithmic Beamforming Design for MIMO Multiway Relay Channel With Clustered Full Data Exchange
This paper studies the beamforming design for the MIMO multiway relay channel with clustered full data exchange. We formulate the linear signal alignment in the model under a rank-constrained rank-minimization (RCRM) framework. Our contribution is two-fold: 1) we decompose the RCRM problem into independent rank minimization subproblems and put forth an iterative algorithm for the beamforming design; 2) for symmetric antenna setups, our approach advances the state of the art by expanding the achievable DoF region, and for asymmetric antenna setups, our approach can be directly applied while prior approaches are not readily applicable.
- 24 July 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 67 (10) , 10081-10086
Algorithmic Beamforming Design for MIMO Multiway Relay Channel With Clustered Full Data Exchange
This paper studies the beamforming design for the MIMO multiway relay channel with clustered full data exchange. We formulate the linear signal alignment in the model under a rank-constrained rank-minimization (RCRM) framework. Our contribution is two-fold: 1) we decompose the RCRM problem into independent rank minimization subproblems and put forth an iterative algorithm for the beamforming design; 2) for symmetric antenna setups, our approach advances the state of the art by expanding the achievable DoF region, and for asymmetric antenna setups, our approach can be directly applied while prior approaches are not readily applicable.
- 1 June 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1182-1186
Network-coded fronthaul transmission for cache-aided C-RAN
In this paper, we study the cache-aided cloud radio access network (C-RAN) with wireless fronthaul, where multiple cache-enabled users are served by multiple cache-enabled transmitters that are connected to a cloud processor through a wireless fronthaul link. We put forth a caching-and-delivery scheme that combines network-coded fronthaul transmission with cache-aided interference management. By broadcasting network-coded messages, the cloud processor provides additional information of the requested files to the transmitters, so as to reduce the edge delivery time. Based on our scheme, an achievable normalized delivery time (NDT) is derived with respect to the cache sizes and the fronthaul capacity.
- 1 June 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1182-1186
Network-coded fronthaul transmission for cache-aided C-RAN
In this paper, we study the cache-aided cloud radio access network (C-RAN) with wireless fronthaul, where multiple cache-enabled users are served by multiple cache-enabled transmitters that are connected to a cloud processor through a wireless fronthaul link. We put forth a caching-and-delivery scheme that combines network-coded fronthaul transmission with cache-aided interference management. By broadcasting network-coded messages, the cloud processor provides additional information of the requested files to the transmitters, so as to reduce the edge delivery time. Based on our scheme, an achievable normalized delivery time (NDT) is derived with respect to the cache sizes and the fronthaul capacity.
- 1 June 2015
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2338-2343
Lattice-based cooperative communications for two-path relay channels with direct link
In this paper two-path relay channels with direct link between the source and the destination are considered. We present a decode-and-forward cooperative transmission scheme based on nested lattice codes. The proposed scheme performs complete inter-relay interference (IRI) cancellation at the relays as well as successive decoding at the destination, which efficiently combines the signals from the source and the relays. We derive the achievable rates of the proposed scheme, and show that our scheme considerably outperforms the existing schemes in the literature.
- 1 June 2015
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2338-2343
Lattice-based cooperative communications for two-path relay channels with direct link
In this paper two-path relay channels with direct link between the source and the destination are considered. We present a decode-and-forward cooperative transmission scheme based on nested lattice codes. The proposed scheme performs complete inter-relay interference (IRI) cancellation at the relays as well as successive decoding at the destination, which efficiently combines the signals from the source and the relays. We derive the achievable rates of the proposed scheme, and show that our scheme considerably outperforms the existing schemes in the literature.