Hee-Jun Jung
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Hee-Jun Jung has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
- Kangil Kim
AI Graduate School Gwangju Institute of Science and Technology
2 shared publications - Seung-Hoon Na
Graduate School of Artificial Intelligence, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea
2 shared publications Doyeon Kim
AI graduate school of Gwangju Institute of Science and Technology (GIST) 123 Cheomdangwagi-ro Buk-gu 61005 Gwangju South Korea
1 shared publication- Doyeon Kim1 shared publication
- Jin-Hwi Park
Chung-Ang University
1 shared publication - Kangil Kim
Gwangju Institute of Science and Technology (GIST)
1 shared publication
Latest Publications
- 1 October 2026
- Published by Springer Nature
- p. 187-206
Disentangling Rotation and Translation from SE(3)-Equivariant Features for Shape Assembly
- 1 October 2026
- Published by Springer Nature
- p. 187-206
Disentangling Rotation and Translation from SE(3)-Equivariant Features for Shape Assembly
- 14 July 2023
- Published by Elsevier in Expert Systems with Applications
- Vol. 234
Feature structure distillation with Centered Kernel Alignment in BERT transferring
- 14 July 2023
- Published by Elsevier in Expert Systems with Applications
- Vol. 234
Feature structure distillation with Centered Kernel Alignment in BERT transferring
- Open Access
Feature Structure Distillation with Centered Kernel Alignment in BERT Transferring
Knowledge distillation is an approach to transfer information on representations from a teacher to a student by reducing their difference. A challenge of this approach is to reduce the flexibility of the student's representations inducing inaccurate learning of the teacher's knowledge. To resolve it in transferring, we investigate distillation of structures of representations specified to three types: intra-feature, local inter-feature, global inter-feature structures. To transfer them, we introduce feature structure distillation methods based on the Centered Kernel Alignment, which assigns a consistent value to similar features structures and reveals more informative relations. In particular, a memory-augmented transfer method with clustering is implemented for the global structures. The methods are empirically analyzed on the nine tasks for language understanding of the GLUE dataset with Bidirectional Encoder Representations from Transformers (BERT), which is a representative neural language model. In the results, the proposed methods effectively transfer the three types of structures and improve performance compared to state-of-the-art distillation methods. Indeed, the code for the methods is available in https://github.com/maroo-sky/FSD.
Open AccessFeature Structure Distillation with Centered Kernel Alignment in BERT Transferring
Knowledge distillation is an approach to transfer information on representations from a teacher to a student by reducing their difference. A challenge of this approach is to reduce the flexibility of the student's representations inducing inaccurate learning of the teacher's knowledge. To resolve it in transferring, we investigate distillation of structures of representations specified to three types: intra-feature, local inter-feature, global inter-feature structures. To transfer them, we introduce feature structure distillation methods based on the Centered Kernel Alignment, which assigns a consistent value to similar features structures and reveals more informative relations. In particular, a memory-augmented transfer method with clustering is implemented for the global structures. The methods are empirically analyzed on the nine tasks for language understanding of the GLUE dataset with Bidirectional Encoder Representations from Transformers (BERT), which is a representative neural language model. In the results, the proposed methods effectively transfer the three types of structures and improve performance compared to state-of-the-art distillation methods. Indeed, the code for the methods is available in https://github.com/maroo-sky/FSD.