Jingru Li
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Jingru Li has a total of 17 co-authors. Here are the top co-authors by number of shared publications:
- Jiajun Bu
Zhejiang Key Lab of Accessible Perception & Intelligent Systems, Zhejiang University, Hangzhou, China
2 shared publications - Liangcheng Li
Zhejiang University, Hangzhou, China
2 shared publications - Zhi Yu
Zhejiang University
2 shared publications - Arunava Roy
Department of Computer Science, The University of Memphis, Memphis, TN, USA
1 shared publication - Bo Wang
School of Mathematics and Statistics Key Laboratory of Operations Research and Control of Universities in Fujian Fuzhou University 350116 Fuzhou China
1 shared publication Haishuai Wang
1 shared publication
Latest Publications
- 10 May 2024
- Published by Elsevier in Neural Networks
- Vol. 177, 106386
AdaDFKD: Exploring adaptive inter-sample relationship in data-free knowledge distillation
- 10 May 2024
- Published by Elsevier in Neural Networks
- Vol. 177, 106386
AdaDFKD: Exploring adaptive inter-sample relationship in data-free knowledge distillation
- 24 May 2023
- Published by Elsevier in Information Sciences
- Vol. 642
Dynamic data-free knowledge distillation by easy-to-hard learning strategy
- 24 May 2023
- Published by Elsevier in Information Sciences
- Vol. 642
Dynamic data-free knowledge distillation by easy-to-hard learning strategy
- Open Access
- 8 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 46441-46453
Deep Generative Knowledge Distillation by Likelihood Finetuning
Knowledge Distillation (KD) is designed to train smaller student models using a larger pretrained teacher model. However, in decentralized data systems such as blockchain, privacy concerns may arise, making the data inaccessible. To address this issue, Data-Free KD (DFKD) methods have been proposed, which extract prior knowledge from teacher networks and use it to synthesize data for KD. Previous DFKD methods faced challenges due to the large search space of data generation. Recently, deep generative models (DGMs) have been proposed to learn data distribution using deep networks, which provides an efficient way to reduce the search space by generating a set of pseudo data. In this paper, we explore the performance of KD trained using pseudo samples generated by pretrained DGMs and find that the correlation with image quality is not always positive. Based on this observation, we propose a new DFKD framework called Generative Knowledge Distillation (GenKD) that reduces the search space by constructing a prior distribution modeled by DGMs for their power of likelihood estimation. Specifically, we use energy-based models (EBM) to generate data from the Maximum Likelihood Estimation (MLE) of the EBM and gradients from downstream KD tasks by policy gradient. We then train the student model using the pretrained teacher model and pseudo samples. We also implement our GenKD framework on several widely-used benchmarks, including CIFAR100, CIFAR10, and SVHN. Our experiments demonstrate that we can generate high-quality pseudo samples quantitatively and qualitatively using GenKD. Additionally, the top-1 accuracy of the student network can approach state-of-the-art (SOTA) DFKD methods trained using fewer pseudo samples and less generation time.
- 8 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 46441-46453
Open AccessDeep Generative Knowledge Distillation by Likelihood Finetuning
Knowledge Distillation (KD) is designed to train smaller student models using a larger pretrained teacher model. However, in decentralized data systems such as blockchain, privacy concerns may arise, making the data inaccessible. To address this issue, Data-Free KD (DFKD) methods have been proposed, which extract prior knowledge from teacher networks and use it to synthesize data for KD. Previous DFKD methods faced challenges due to the large search space of data generation. Recently, deep generative models (DGMs) have been proposed to learn data distribution using deep networks, which provides an efficient way to reduce the search space by generating a set of pseudo data. In this paper, we explore the performance of KD trained using pseudo samples generated by pretrained DGMs and find that the correlation with image quality is not always positive. Based on this observation, we propose a new DFKD framework called Generative Knowledge Distillation (GenKD) that reduces the search space by constructing a prior distribution modeled by DGMs for their power of likelihood estimation. Specifically, we use energy-based models (EBM) to generate data from the Maximum Likelihood Estimation (MLE) of the EBM and gradients from downstream KD tasks by policy gradient. We then train the student model using the pretrained teacher model and pseudo samples. We also implement our GenKD framework on several widely-used benchmarks, including CIFAR100, CIFAR10, and SVHN. Our experiments demonstrate that we can generate high-quality pseudo samples quantitatively and qualitatively using GenKD. Additionally, the top-1 accuracy of the student network can approach state-of-the-art (SOTA) DFKD methods trained using fewer pseudo samples and less generation time.
- 1 September 2020
- Published by Elsevier in Neurocomputing
- Vol. 407, 136-154
A Dual Recurrent Neural Network-based Hybrid Approach for Solving Convex Quadratic Bi-Level Programming Problem
- 1 September 2020
- Published by Elsevier in Neurocomputing
- Vol. 407, 136-154
A Dual Recurrent Neural Network-based Hybrid Approach for Solving Convex Quadratic Bi-Level Programming Problem
- Open Access
- 20 January 2011
- Published by China Science Publishing & Media Ltd. in Journal of Electronics & Information Technology
- Vol. 33 (1) , 175-179
Semiring Trust Model Based on Adaptive Forgetting Scheme
- 20 January 2011
- Published by China Science Publishing & Media Ltd. in Journal of Electronics & Information Technology
- Vol. 33 (1) , 175-179
Open AccessSemiring Trust Model Based on Adaptive Forgetting Scheme