Han Li
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Han Li has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Jing Xie
School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400044, China
2 shared publications - Yingcheng Lin
School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400044, China
2 shared publications - Chunhua Tang
State Key Laboratory of Intelligent Vehicle Safety Technology, Chongqing 401133, China
1 shared publication - Jianchao Wang
Seres Group Co., Ltd., Chongqing 400038, China
1 shared publication - Li Wang
State Key Laboratory of Intelligent Vehicle Safety Technology, Chongqing 401133, China
1 shared publication - Lin Zhou
School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400044, China
1 shared publication
Latest Publications
- Open Access
Reconstruction and Consolidation Pruning with Feature Reload Mechanism for Efficient Deep CNNs
Channel pruning enables model acceleration by removing channels from convolutional neural networks (CNNs). However, many existing methods adopt a “hard removal” strategy that directly removing low-importance channels, leading to severe feature loss and accuracy degradation. To address this issue, we propose Reconstruction and Consolidation Pruning (RCP), a pruning framework that decouples the pruning process into a pruning-training phase and an inference phase. During pruning training, RCP generates a pruning strategy based on channel importance under a global pruning rate constraint, and constructs a feature reloading mechanism. This mechanism utilizes a learnable compensation convolution to adaptively transfer and fuse discriminative features hidden in the pruned channels into the retained channels. In the inference phase, RCP adopts a linear reparameterization strategy to seamlessly consolidate the compensation branches into the main network branch without loss of performance, ensuring zero additional operator overhead during inference. This reversible structural transformation ensures that the training-time augmented architecture and the inference-time compact architecture are functionally identical under linear consolidation. Experimental results show that at 50% FLOPs reduction, RCP incurs only a 0.84% accuracy drop on ResNet-50 (ImageNet-1K), while at 53% FLOPs reduction it achieves a 0.07% accuracy improvement for ResNet-56 (CIFAR-10), validating the proposed method’s effectiveness and superiority under high compression rates.Open AccessReconstruction and Consolidation Pruning with Feature Reload Mechanism for Efficient Deep CNNs
Channel pruning enables model acceleration by removing channels from convolutional neural networks (CNNs). However, many existing methods adopt a “hard removal” strategy that directly removing low-importance channels, leading to severe feature loss and accuracy degradation. To address this issue, we propose Reconstruction and Consolidation Pruning (RCP), a pruning framework that decouples the pruning process into a pruning-training phase and an inference phase. During pruning training, RCP generates a pruning strategy based on channel importance under a global pruning rate constraint, and constructs a feature reloading mechanism. This mechanism utilizes a learnable compensation convolution to adaptively transfer and fuse discriminative features hidden in the pruned channels into the retained channels. In the inference phase, RCP adopts a linear reparameterization strategy to seamlessly consolidate the compensation branches into the main network branch without loss of performance, ensuring zero additional operator overhead during inference. This reversible structural transformation ensures that the training-time augmented architecture and the inference-time compact architecture are functionally identical under linear consolidation. Experimental results show that at 50% FLOPs reduction, RCP incurs only a 0.84% accuracy drop on ResNet-50 (ImageNet-1K), while at 53% FLOPs reduction it achieves a 0.07% accuracy improvement for ResNet-56 (CIFAR-10), validating the proposed method’s effectiveness and superiority under high compression rates. - Open Access
Scale Calibration and Pressure-Driven Knowledge Distillation for Image Classification
Knowledge distillation achieves model compression by training a lightweight student network to mimic the output distribution of a larger teacher network. However, when the teacher becomes overconfident, its sharply peaked logits break the scale symmetry of supervision and induce high-variance gradients, leading to unstable optimization. Meanwhile, research that focuses only on final-logit alignment often fails to utilize intermediate semantic structure effectively. This causes weak discrimination of student representations, especially under class imbalance. To address these issues, we propose Scale Calibration and Pressure-Driven Knowledge Distillation (SPKD): a one-stage framework comprising two lightweight, complementary mechanisms. First, a dynamic scale calibration module normalizes the teacher’s logits to a consistent magnitude, reducing gradient variance. Secondly, an adaptive pressure-driven mechanism refines student learning by preventing feature collapse and promoting intra-class compactness and inter-class separability. Extensive experiments on CIFAR-100 and ImageNet demonstrate that SPKD achieves superior performance to distillation baselines across various teacher–student combinations. For example, SPKD achieves a score of 74.84% on CIFAR-100 for the homogeneous architecture VGG13-VGG8. Additional evidence from logit norm and gradient variance statistics, as well as representation analyses, proves the fact that SPKD stabilizes optimization while learning more discriminative and well-structured features.
Open AccessScale Calibration and Pressure-Driven Knowledge Distillation for Image Classification
Knowledge distillation achieves model compression by training a lightweight student network to mimic the output distribution of a larger teacher network. However, when the teacher becomes overconfident, its sharply peaked logits break the scale symmetry of supervision and induce high-variance gradients, leading to unstable optimization. Meanwhile, research that focuses only on final-logit alignment often fails to utilize intermediate semantic structure effectively. This causes weak discrimination of student representations, especially under class imbalance. To address these issues, we propose Scale Calibration and Pressure-Driven Knowledge Distillation (SPKD): a one-stage framework comprising two lightweight, complementary mechanisms. First, a dynamic scale calibration module normalizes the teacher’s logits to a consistent magnitude, reducing gradient variance. Secondly, an adaptive pressure-driven mechanism refines student learning by preventing feature collapse and promoting intra-class compactness and inter-class separability. Extensive experiments on CIFAR-100 and ImageNet demonstrate that SPKD achieves superior performance to distillation baselines across various teacher–student combinations. For example, SPKD achieves a score of 74.84% on CIFAR-100 for the homogeneous architecture VGG13-VGG8. Additional evidence from logit norm and gradient variance statistics, as well as representation analyses, proves the fact that SPKD stabilizes optimization while learning more discriminative and well-structured features.