Gang Chen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Gang Chen has a total of 4 co-authors. Here are the top co-authors by number of shared publications:
- Fuli Feng
National University of Singapore, Singapore, Singapore
1 shared publication - Jiawei Chen
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China and Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Hangzhou, China
1 shared publication - Sheng Zhou
Zhejiang Key Lab of Accessible Perception & Intelligent Systems, Zhejiang University, Hangzhou, China
1 shared publication - Xiangnan He
University of Science and Technology of China, Hefei, China
1 shared publication
Latest Publications
- Open Access
- 27 February 2023
- Published by Association for Computing Machinery (ACM)
- p. 976-984
Unbiased Knowledge Distillation for Recommendation
As a promising solution for model compression, knowledge distillation (KD) has been applied in recommender systems (RS) to reduce inference latency. Traditional solutions first train a full teacher model from the training data, and then transfer its knowledge (\iesoft labels ) to supervise the learning of a compact student model. However, we find such a standard distillation paradigm would incur serious bias issue --- popular items are more heavily recommended after the distillation. This effect prevents the student model from making accurate and fair recommendations, decreasing the effectiveness of RS. In this work, we identify the origin of the bias in KD --- it roots in the biased soft labels from the teacher, and is further propagated and intensified during the distillation. To rectify this, we propose a new KD method with a stratified distillation strategy. It first partitions items into multiple groups according to their popularity, and then extracts the ranking knowledge within each group to supervise the learning of the student. Our method is simple and teacher-agnostic --- it works on distillation stage without affecting the training of the teacher model. We conduct extensive theoretical and empirical studies to validate the effectiveness of our proposal. We release our code at: https://github.com/chengang95/UnKD.
- 27 February 2023
- Published by Association for Computing Machinery (ACM)
- p. 976-984
Open AccessUnbiased Knowledge Distillation for Recommendation
As a promising solution for model compression, knowledge distillation (KD) has been applied in recommender systems (RS) to reduce inference latency. Traditional solutions first train a full teacher model from the training data, and then transfer its knowledge (\iesoft labels ) to supervise the learning of a compact student model. However, we find such a standard distillation paradigm would incur serious bias issue --- popular items are more heavily recommended after the distillation. This effect prevents the student model from making accurate and fair recommendations, decreasing the effectiveness of RS. In this work, we identify the origin of the bias in KD --- it roots in the biased soft labels from the teacher, and is further propagated and intensified during the distillation. To rectify this, we propose a new KD method with a stratified distillation strategy. It first partitions items into multiple groups according to their popularity, and then extracts the ranking knowledge within each group to supervise the learning of the student. Our method is simple and teacher-agnostic --- it works on distillation stage without affecting the training of the teacher model. We conduct extensive theoretical and empirical studies to validate the effectiveness of our proposal. We release our code at: https://github.com/chengang95/UnKD.