Ju Yeon Kang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Ju Yeon Kang has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Tae Hee Han
Department of Semiconductor Systems Engineering, Sungkyunkwan University, Suwon, South Korea
3 shared publications Chang Ho Ryu
Department of Artificial Intelligence, Sungkyunkwan University, Suwon, South Korea
2 shared publications- Chang Ho Ryu
Department of Artificial Intelligence, Sungkyunkwan University, Suwon 16419, Korea
1 shared publication - Hyun Jo Lim
Department of Artificial Intelligence, Sungkyunkwan University, Suwon 16419, Republic of Korea
1 shared publication - Sangmin Hyun
Department of Artificial Intelligence, Sungkyunkwan University, Suwon 16419, Korea
1 shared publication Suk Bong Kang
Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, South Korea
1 shared publication
Latest Publications
- Open Access
- 13 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 128306-128315
SFAO: Sign-Flipping-Aware Optimization for Early-Stopping of Binarized Neural Networks
One of the vital challenges for the binary neural networks (BNNs) is improving their inference performance by expanding their data representation capabilities for figuring out delicate patterns and nuances in the data. Addressing the explosive computational demands on neural network training is essential to guarantee sustainable development and scalable deployment. However, mitigating the increase in the computational cost during the training phase is critical for ensuring sustainability and scalability during deployment. In this study, an advanced sign-flipping-aware optimizer (SFAO) that focuses on BNNs was introduced to diminish the computational burden. SFAO balanced the model performance and computational cost through sign-flipping-aware updating rules throughout the training of BNNs. SFAO optimizer, tailored for BNNs with binary weight-specific updating rules, considerably reduced the computing resources needed for training on the CIFAR-10 dataset. Specifically, it surpassed the conventional full-precision updating rule by reducing the total instruction count by 21.89%. In contrast, SFAO showed a marginal 0.44% decline in the image classification accuracy relative to the updating rules for the full-precision parameters. Furthermore, the implementation of early stopping using the sign flip rate led to a notable reduction of 9.37% in the average computation time per network for the ImageNet dataset.
- 13 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 128306-128315
Open AccessSFAO: Sign-Flipping-Aware Optimization for Early-Stopping of Binarized Neural Networks
One of the vital challenges for the binary neural networks (BNNs) is improving their inference performance by expanding their data representation capabilities for figuring out delicate patterns and nuances in the data. Addressing the explosive computational demands on neural network training is essential to guarantee sustainable development and scalable deployment. However, mitigating the increase in the computational cost during the training phase is critical for ensuring sustainability and scalability during deployment. In this study, an advanced sign-flipping-aware optimizer (SFAO) that focuses on BNNs was introduced to diminish the computational burden. SFAO balanced the model performance and computational cost through sign-flipping-aware updating rules throughout the training of BNNs. SFAO optimizer, tailored for BNNs with binary weight-specific updating rules, considerably reduced the computing resources needed for training on the CIFAR-10 dataset. Specifically, it surpassed the conventional full-precision updating rule by reducing the total instruction count by 21.89%. In contrast, SFAO showed a marginal 0.44% decline in the image classification accuracy relative to the updating rules for the full-precision parameters. Furthermore, the implementation of early stopping using the sign flip rate led to a notable reduction of 9.37% in the average computation time per network for the ImageNet dataset.
- Open Access
- 23 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 8057-8064
Binarized Neural Network With Parameterized Weight Clipping and Quantization Gap Minimization for Online Knowledge Distillation
As the applications for artificial intelligence are growing rapidly, numerous network compression algorithms have been developed to restrict computing resources such as smartphones, edge, and IoT devices. Knowledge distillation (KD) leverages soft labels derived from a teacher model to a less parameterized model achieving high accuracy with reduced computational burden. Moreover, online KD provides parallel computing through collaborative learning between teacher and student networks, thus enhancing the training speed. A binarized neural network (BNN) offers an intriguing opportunity to facilitate aggressive compression at the expense of drastically degraded accuracy. In this study, two performance improvements are proposed for online KD when a BNN is applied as a student network: 1) parameterized weight clipping (PWC) to reduce dead weights in the student network and 2) quantization gap-aware adaptive temperature scheduling between the teacher and student networks. In contrast to constant weight clipping (CWC), PWC demonstrates a 3.78% top-1 test accuracy enhancement with trainable weight clipping by decreasing the gradient mismatch with CIFAR-10 dataset. Furthermore, the quantization gap-aware temperature scheduling increases the top-1 test accuracy by 0.08% over online KD at a constant temperature. By aggregating both methodologies, the top-1 test accuracy for CIFAR-10 dataset was 94.60%, and that for Tiny-ImageNet dataset was comparable to that of the 32-bit full-precision neural network.
- 23 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 8057-8064
Open AccessBinarized Neural Network With Parameterized Weight Clipping and Quantization Gap Minimization for Online Knowledge Distillation
As the applications for artificial intelligence are growing rapidly, numerous network compression algorithms have been developed to restrict computing resources such as smartphones, edge, and IoT devices. Knowledge distillation (KD) leverages soft labels derived from a teacher model to a less parameterized model achieving high accuracy with reduced computational burden. Moreover, online KD provides parallel computing through collaborative learning between teacher and student networks, thus enhancing the training speed. A binarized neural network (BNN) offers an intriguing opportunity to facilitate aggressive compression at the expense of drastically degraded accuracy. In this study, two performance improvements are proposed for online KD when a BNN is applied as a student network: 1) parameterized weight clipping (PWC) to reduce dead weights in the student network and 2) quantization gap-aware adaptive temperature scheduling between the teacher and student networks. In contrast to constant weight clipping (CWC), PWC demonstrates a 3.78% top-1 test accuracy enhancement with trainable weight clipping by decreasing the gradient mismatch with CIFAR-10 dataset. Furthermore, the quantization gap-aware temperature scheduling increases the top-1 test accuracy by 0.08% over online KD at a constant temperature. By aggregating both methodologies, the top-1 test accuracy for CIFAR-10 dataset was 94.60%, and that for Tiny-ImageNet dataset was comparable to that of the 32-bit full-precision neural network.
- Open Access
- 26 August 2022
- Published by MDPI AG in Electronics
- Vol. 11 (17) , 2678
CSGN: Combined Channel- and Spatial-Wise Dynamic Gating Architecture for Convolutional Neural Networks
The explosive computation and memory requirements of convolutional neural networks (CNNs) hinder their deployment in resource-constrained devices. Because conventional CNNs perform identical parallelized computations even on redundant pixels, the saliency of various features in an image should be reflected for higher energy efficiency and market penetration. This paper proposes a novel channel and spatial gating network (CSGN) for adaptively selecting vital channels and generating spatial-wise execution masks. A CSGN can be characterized as a dynamic channel and a spatial-aware gating module by maximally utilizing opportunistic sparsity. Extensive experiments were conducted on the CIFAR-10 and ImageNet datasets based on ResNet. The results revealed that, with the proposed architecture, the amount of multiply-accumulate (MAC) operations was reduced by 1.97–11.78× and 1.37–13.12× on CIFAR-10 and ImageNet, respectively, with negligible accuracy degradation in the inference stage compared with the baseline architectures.
- 26 August 2022
- Published by MDPI AG in Electronics
- Vol. 11 (17) , 2678
Open AccessCSGN: Combined Channel- and Spatial-Wise Dynamic Gating Architecture for Convolutional Neural Networks
The explosive computation and memory requirements of convolutional neural networks (CNNs) hinder their deployment in resource-constrained devices. Because conventional CNNs perform identical parallelized computations even on redundant pixels, the saliency of various features in an image should be reflected for higher energy efficiency and market penetration. This paper proposes a novel channel and spatial gating network (CSGN) for adaptively selecting vital channels and generating spatial-wise execution masks. A CSGN can be characterized as a dynamic channel and a spatial-aware gating module by maximally utilizing opportunistic sparsity. Extensive experiments were conducted on the CIFAR-10 and ImageNet datasets based on ResNet. The results revealed that, with the proposed architecture, the amount of multiply-accumulate (MAC) operations was reduced by 1.97–11.78× and 1.37–13.12× on CIFAR-10 and ImageNet, respectively, with negligible accuracy degradation in the inference stage compared with the baseline architectures.