Chao Huang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Chao Huang has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
- Xin Liu
Department of Computer Science, University of California, Davis, CA 95616, USA
5 shared publications - Jianwei Huang
The Chinese University of Hong Kong
4 shared publications - Haoran Yu
Beijing Institute of Technology
2 shared publications - Justin Dachille
King's College London, London, United Kingdom
2 shared publications - Randall Berry
Northwestern University, Evanston, USA
2 shared publications - Biying Shou
Department of Management Science City University of Hong Kong Hong Kong China
1 shared publication
Latest Publications
- Open Access
- 19 February 2025
- Published by Springer Nature in Machine Learning
- Vol. 114 (4) , 1-17
An empirical study on impact of label noise on synthetic tabular data generation
Synthetic data has been actively used for various machine learning-based tasks due to its benefits such as massive reproducibility and privacy enhancement compared to using the original data. The quality of the generated synthetic dataset crucially depends on the quality of the original data, and the latter is often corrupted by label noise. While there have been studies on feature noise, how label noise affects synthetic data generation is under-explored. In this paper, we evaluate the impact of the noisy label on synthetic data generation with a focus on tabular data. One challenge is how to evaluate the quality of synthetic data under label noise. To this end, we design comprehensive experiments to measure the impact of label noise on synthetic data generation in different aspects: synthetic data quality, data utility, and convergence for training synthesizers and machine learning models for downstream tasks. The empirical results cover wide aspects of synthetic data generation under label noise and they show quality and utility degrades with higher noise levels while there is no significant effect on the synthesizer convergence observed.
- 19 February 2025
- Published by Springer Nature in Machine Learning
- Vol. 114 (4) , 1-17
Open AccessAn empirical study on impact of label noise on synthetic tabular data generation
Synthetic data has been actively used for various machine learning-based tasks due to its benefits such as massive reproducibility and privacy enhancement compared to using the original data. The quality of the generated synthetic dataset crucially depends on the quality of the original data, and the latter is often corrupted by label noise. While there have been studies on feature noise, how label noise affects synthetic data generation is under-explored. In this paper, we evaluate the impact of the noisy label on synthetic data generation with a focus on tabular data. One challenge is how to evaluate the quality of synthetic data under label noise. To this end, we design comprehensive experiments to measure the impact of label noise on synthetic data generation in different aspects: synthetic data quality, data utility, and convergence for training synthesizers and machine learning models for downstream tasks. The empirical results cover wide aspects of synthetic data generation under label noise and they show quality and utility degrades with higher noise levels while there is no significant effect on the synthesizer convergence observed.
- 17 September 2024
- Published by Springer Nature
- p. 143-158
An Accuracy-Shaping Mechanism for Competitive Distributed Learning
- 17 September 2024
- Published by Springer Nature
- p. 143-158
An Accuracy-Shaping Mechanism for Competitive Distributed Learning
- 14 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 11 (23) , 38012-38022
Incentivizing Efficient Label Denoising in Federated Learning
Federated learning (FL) is a distributed machine learning scheme that enables clients to train a shared global model without exchanging local data. In FL, the presence of label noise can severely reduce the accuracy of the global model. Although some recent works have focused on designing algorithms for label denoising, they ignored the important issue that clients may not apply costly label denoising strategies due to them being self-interested and having heterogeneous valuations on the model accuracy. To fill this gap, we model the clients’ strategic interactions as a novel label denoising game and determine the clients’ equilibrium strategies. We prove that the equilibrium outcome always leads to a lower global model accuracy than the socially optimal solution does. To motivate the clients’ efficient label denosing behaviors, we propose a penalty-based incentive mechanism and design the degree of penalty for punishing the clients’ undesired denoising behaviors, addressing the inaccurate noise rate detection in FL. We prove that our mechanism can achieve social efficiency, individual rationality, and weak budget balance. Numerical experiments on MNIST and CIFAR-10 show that as clients’ data become noisier, the gap between the equilibrium outcome and the socially optimal solution increases, verifying the necessity of an incentive mechanism. We empirically show that our proposed mechanism improves the model accuracy by up to 4.4% and incentivizes clients to achieve equilibrium strategies that are close to the socially optimal solution.
- 14 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 11 (23) , 38012-38022
Incentivizing Efficient Label Denoising in Federated Learning
Federated learning (FL) is a distributed machine learning scheme that enables clients to train a shared global model without exchanging local data. In FL, the presence of label noise can severely reduce the accuracy of the global model. Although some recent works have focused on designing algorithms for label denoising, they ignored the important issue that clients may not apply costly label denoising strategies due to them being self-interested and having heterogeneous valuations on the model accuracy. To fill this gap, we model the clients’ strategic interactions as a novel label denoising game and determine the clients’ equilibrium strategies. We prove that the equilibrium outcome always leads to a lower global model accuracy than the socially optimal solution does. To motivate the clients’ efficient label denosing behaviors, we propose a penalty-based incentive mechanism and design the degree of penalty for punishing the clients’ undesired denoising behaviors, addressing the inaccurate noise rate detection in FL. We prove that our mechanism can achieve social efficiency, individual rationality, and weak budget balance. Numerical experiments on MNIST and CIFAR-10 show that as clients’ data become noisier, the gap between the equilibrium outcome and the socially optimal solution increases, verifying the necessity of an incentive mechanism. We empirically show that our proposed mechanism improves the model accuracy by up to 4.4% and incentivizes clients to achieve equilibrium strategies that are close to the socially optimal solution.
- 13 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 11 (16) , 27409-27420
When Federated Learning Meets Oligopoly Competition: Stability and Model Differentiation
Federated learning (FL) is decentralized machine learning framework that finds various applications in health, finance, and the Internet of things. This article studies the under-explored business competition in FL, where organizations are both collaborators in training a shared model and competitors in providing model-based services to a continuum of customers. We focus on an oligopoly case with three organizations. To understand how competition affects FL collaboration, we start with a benchmark case where organizations are not competitors, and show that they have an incentive to collaborate. However, in the presence of competition, organizations may prefer to train local models instead of collaborating via FL (even if FL incurs zero training costs). The reason is that FL intensifies price competition by improving organizations’ model performance to a similar level. To address this issue, we devise a model differentiation mechanism in which organizations adaptively adjust their model performance, enabling differentiated model-based services to customers. We prove that the adaptive mechanism converges in polynomial time and is incentive compatible. Perhaps surprisingly, numerical experiments on CIFAR-10 show that the mechanism can simultaneously improve the model performance, organizations’ revenues, and social welfare. The improvement is up to 22.31%, 14.42%, and 19.50%, respectively.
- 13 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 11 (16) , 27409-27420
When Federated Learning Meets Oligopoly Competition: Stability and Model Differentiation
Federated learning (FL) is decentralized machine learning framework that finds various applications in health, finance, and the Internet of things. This article studies the under-explored business competition in FL, where organizations are both collaborators in training a shared model and competitors in providing model-based services to a continuum of customers. We focus on an oligopoly case with three organizations. To understand how competition affects FL collaboration, we start with a benchmark case where organizations are not competitors, and show that they have an incentive to collaborate. However, in the presence of competition, organizations may prefer to train local models instead of collaborating via FL (even if FL incurs zero training costs). The reason is that FL intensifies price competition by improving organizations’ model performance to a similar level. To address this issue, we devise a model differentiation mechanism in which organizations adaptively adjust their model performance, enabling differentiated model-based services to customers. We prove that the adaptive mechanism converges in polynomial time and is incentive compatible. Perhaps surprisingly, numerical experiments on CIFAR-10 show that the mechanism can simultaneously improve the model performance, organizations’ revenues, and social welfare. The improvement is up to 22.31%, 14.42%, and 19.50%, respectively.
- Open Access
Federated Learning in Competitive EV Charging Market
Federated Learning (FL) has demonstrated a significant potential to improve the quality of service (QoS) of EV charging stations. While existing studies have primarily focused on developing FL algorithms, the effect of FL on the charging stations' operation in terms of price competition has yet to be fully understood. This paper aims to fill this gap by modeling the strategic interactions between two charging stations and EV owners as a multi-stage game. Each station first decides its FL participation strategy and charging price, and then individual EV owners decide their charging strategies. The game analysis involves solving a non-concave problem and by decomposing it into a piece-wise concave program we manage to fully characterize the equilibrium. Based on real-world datasets, our numerical results reveal an interesting insight: even if FL improves QoS, it can lead to smaller profits for both stations. The key reason is that FL intensifies the price competition between charging stations by improving stations' QoS to a similar level. We further show that the stations will participate in FL when their data distributions are mildly dissimilar.
Open AccessFederated Learning in Competitive EV Charging Market
Federated Learning (FL) has demonstrated a significant potential to improve the quality of service (QoS) of EV charging stations. While existing studies have primarily focused on developing FL algorithms, the effect of FL on the charging stations' operation in terms of price competition has yet to be fully understood. This paper aims to fill this gap by modeling the strategic interactions between two charging stations and EV owners as a multi-stage game. Each station first decides its FL participation strategy and charging price, and then individual EV owners decide their charging strategies. The game analysis involves solving a non-concave problem and by decomposing it into a piece-wise concave program we manage to fully characterize the equilibrium. Based on real-world datasets, our numerical results reveal an interesting insight: even if FL improves QoS, it can lead to smaller profits for both stations. The key reason is that FL intensifies the price competition between charging stations by improving stations' QoS to a similar level. We further show that the stations will participate in FL when their data distributions are mildly dissimilar.
- Open Access
Incentive Mechanism Design for Distributed Ensemble Learning
Distributed ensemble learning (DEL) involves training multiple models at distributed learners, and then combining their predictions to improve performance. Existing related studies focus on DEL algorithm design and optimization but ignore the important issue of incentives, without which self-interested learners may be unwilling to participate in DEL. We aim to fill this gap by presenting a first study on the incentive mechanism design for DEL. Our proposed mechanism specifies both the amount of training data and reward for learners with heterogeneous computation and communication costs. One design challenge is to have an accurate understanding regarding how learners' diversity (in terms of training data) affects the ensemble accuracy. To this end, we decompose the ensemble accuracy into a diversity-precision tradeoff to guide the mechanism design. Another challenge is that the mechanism design involves solving a mixed-integer program with a large search space. To this end, we propose an alternating algorithm that iteratively updates each learner's training data size and reward. We prove that under mild conditions, the algorithm converges. Numerical results using MNIST dataset show an interesting result: our proposed mechanism may prefer a lower level of learner diversity to achieve a higher ensemble accuracy.
Open AccessIncentive Mechanism Design for Distributed Ensemble Learning
Distributed ensemble learning (DEL) involves training multiple models at distributed learners, and then combining their predictions to improve performance. Existing related studies focus on DEL algorithm design and optimization but ignore the important issue of incentives, without which self-interested learners may be unwilling to participate in DEL. We aim to fill this gap by presenting a first study on the incentive mechanism design for DEL. Our proposed mechanism specifies both the amount of training data and reward for learners with heterogeneous computation and communication costs. One design challenge is to have an accurate understanding regarding how learners' diversity (in terms of training data) affects the ensemble accuracy. To this end, we decompose the ensemble accuracy into a diversity-precision tradeoff to guide the mechanism design. Another challenge is that the mechanism design involves solving a mixed-integer program with a large search space. To this end, we propose an alternating algorithm that iteratively updates each learner's training data size and reward. We prove that under mild conditions, the algorithm converges. Numerical results using MNIST dataset show an interesting result: our proposed mechanism may prefer a lower level of learner diversity to achieve a higher ensemble accuracy.
- 31 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Network Science and Engineering
- Vol. 11 (1) , 340-351
Duopoly Business Competition in Cross-Silo Federated Learning
In cross-silo federated learning, clients (e.g., organizations) collaboratively train a global model using private local data. In practice, clients may be not only collaborators but also business competitors. This paper studies the overlooked but practically important problem of business competition in cross-silo FL. We formulate the clients' market competition as a three-stage game, where the clients decide FL training strategies in Stage I and the pricing strategies in Stage II, and then heterogeneous customers decide purchasing strategies in Stage III. The game analysis is highly challenging, as clients' collaborations and competitions are complexly coupled. We manage to characterize the equilibrium properties and find that market competition always reduces the clients' profits and can further lead to a worse global model when clients' costs are high. To mitigate this issue, we propose a general framework that enables proper revenue (profit plus cost) sharing among clients. Both theoretical and numerical results with MNIST and CIFAR-10 show that revenue sharing can greatly improve the global model accuracy and clients' profits. Counter-intuitively, even if market competition limits clients' profits, it can lead to a better global model when clients' costs are low, as clients strive to survive in the market by contributing more training data.
- 31 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Network Science and Engineering
- Vol. 11 (1) , 340-351
Duopoly Business Competition in Cross-Silo Federated Learning
In cross-silo federated learning, clients (e.g., organizations) collaboratively train a global model using private local data. In practice, clients may be not only collaborators but also business competitors. This paper studies the overlooked but practically important problem of business competition in cross-silo FL. We formulate the clients' market competition as a three-stage game, where the clients decide FL training strategies in Stage I and the pricing strategies in Stage II, and then heterogeneous customers decide purchasing strategies in Stage III. The game analysis is highly challenging, as clients' collaborations and competitions are complexly coupled. We manage to characterize the equilibrium properties and find that market competition always reduces the clients' profits and can further lead to a worse global model when clients' costs are high. To mitigate this issue, we propose a general framework that enables proper revenue (profit plus cost) sharing among clients. Both theoretical and numerical results with MNIST and CIFAR-10 show that revenue sharing can greatly improve the global model accuracy and clients' profits. Counter-intuitively, even if market competition limits clients' profits, it can lead to a better global model when clients' costs are low, as clients strive to survive in the market by contributing more training data.
- Open Access
Quantifying the Impact of Label Noise on Federated Learning
Federated Learning (FL) is a distributed machine learning paradigm where clients collaboratively train a model using their local (human-generated) datasets. While existing studies focus on FL algorithm development to tackle data heterogeneity across clients, the important issue of data quality (e.g., label noise) in FL is overlooked. This paper aims to fill this gap by providing a quantitative study on the impact of label noise on FL. We derive an upper bound for the generalization error that is linear in the clients' label noise level. Then we conduct experiments on MNIST and CIFAR-10 datasets using various FL algorithms. Our empirical results show that the global model accuracy linearly decreases as the noise level increases, which is consistent with our theoretical analysis. We further find that label noise slows down the convergence of FL training, and the global model tends to overfit when the noise level is high.
Open AccessQuantifying the Impact of Label Noise on Federated Learning
Federated Learning (FL) is a distributed machine learning paradigm where clients collaboratively train a model using their local (human-generated) datasets. While existing studies focus on FL algorithm development to tackle data heterogeneity across clients, the important issue of data quality (e.g., label noise) in FL is overlooked. This paper aims to fill this gap by providing a quantitative study on the impact of label noise on FL. We derive an upper bound for the generalization error that is linear in the clients' label noise level. Then we conduct experiments on MNIST and CIFAR-10 datasets using various FL algorithms. Our empirical results show that the global model accuracy linearly decreases as the noise level increases, which is consistent with our theoretical analysis. We further find that label noise slows down the convergence of FL training, and the global model tends to overfit when the noise level is high.
- 6 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Selected Areas in Communications
- Vol. 41 (4) , 1167-1185
An Online Inference-Aided Incentive Framework for Information Elicitation Without Verification
We study the design of incentive mechanisms for the problem of information elicitation without verification (IEWV). In IEWV, a data requester seeks to design proper incentives to optimize the tradeoff between the quality of information (collected from distributed crowd workers) and the total cost of incentives (provided to crowd workers) without verifiable ground truth. While prior work often relies on sufficient knowledge of worker information, we study a scenario where the data requester cannot access workers’ heterogeneous information quality and costs ex-ante. We propose a continuum-armed bandit-based incentive mechanism that dynamically learns the optimal reward level from workers’ reported information. A key challenge is that the data requester cannot evaluate the workers’ information quality without verification, which motivates the design of an inference algorithm. The inference problem is non-convex, yet we reformulate it as a bi-convex problem and derive an approximate solution with a performance guarantee, which ensures the effectiveness of our online reward design. We further enhance the inference algorithm using part of the workers’ historical reports. We also propose a novel rule for the data requester to aggregate workers’ solutions more effectively. We show that our mechanism achieves a sub-linear regret $\tilde {O}(T^{1/2})$ and outperforms several celebrated benchmarks.
- 6 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Selected Areas in Communications
- Vol. 41 (4) , 1167-1185
An Online Inference-Aided Incentive Framework for Information Elicitation Without Verification
We study the design of incentive mechanisms for the problem of information elicitation without verification (IEWV). In IEWV, a data requester seeks to design proper incentives to optimize the tradeoff between the quality of information (collected from distributed crowd workers) and the total cost of incentives (provided to crowd workers) without verifiable ground truth. While prior work often relies on sufficient knowledge of worker information, we study a scenario where the data requester cannot access workers’ heterogeneous information quality and costs ex-ante. We propose a continuum-armed bandit-based incentive mechanism that dynamically learns the optimal reward level from workers’ reported information. A key challenge is that the data requester cannot evaluate the workers’ information quality without verification, which motivates the design of an inference algorithm. The inference problem is non-convex, yet we reformulate it as a bi-convex problem and derive an approximate solution with a performance guarantee, which ensures the effectiveness of our online reward design. We further enhance the inference algorithm using part of the workers’ historical reports. We also propose a novel rule for the data requester to aggregate workers’ solutions more effectively. We show that our mechanism achieves a sub-linear regret $\tilde {O}(T^{1/2})$ and outperforms several celebrated benchmarks.
- 9 May 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. PP (15361233) , 1
Online Crowd Learning Through Strategic Worker Reports
When it is difficult to verify contributed solutions in mobile crowdsourcing, the majority voting mechanism is widely utilized to incentivize distributed workers to provide high-quality and truthful solutions. In the majority voting mechanism, a worker is rewarded based on whether his solution is consistent with the majority. However, most prior related work relies on a strong assumption that workers solution accuracy levels are public knowledge, which may not hold in many practical scenarios. We relax such an assumption and propose an online mechanism, which allows the platform to learn the distribution of the workers solution accuracy levels via asking workers to report their private accuracy levels (which do not need to be the true values), in addition to deciding their effort levels and solution reporting strategies. The mechanism design is challenging, as neither the workers task solutions nor their accuracy reports can be verified. We devise a randomized reward mechanism that computes the workers rewards based on their reported accuracy levels, under which the workers obtain rewards if their reported solutions match the majority. Our mechanism induces workers to truthfully report their solution accuracy levels in the long run, and the empirical accuracy distribution converges to the actual accuracy distribution.
- 9 May 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. PP (15361233) , 1
Online Crowd Learning Through Strategic Worker Reports
When it is difficult to verify contributed solutions in mobile crowdsourcing, the majority voting mechanism is widely utilized to incentivize distributed workers to provide high-quality and truthful solutions. In the majority voting mechanism, a worker is rewarded based on whether his solution is consistent with the majority. However, most prior related work relies on a strong assumption that workers solution accuracy levels are public knowledge, which may not hold in many practical scenarios. We relax such an assumption and propose an online mechanism, which allows the platform to learn the distribution of the workers solution accuracy levels via asking workers to report their private accuracy levels (which do not need to be the true values), in addition to deciding their effort levels and solution reporting strategies. The mechanism design is challenging, as neither the workers task solutions nor their accuracy reports can be verified. We devise a randomized reward mechanism that computes the workers rewards based on their reported accuracy levels, under which the workers obtain rewards if their reported solutions match the majority. Our mechanism induces workers to truthfully report their solution accuracy levels in the long run, and the empirical accuracy distribution converges to the actual accuracy distribution.