Byung-Soo Kim
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Byung-Soo Kim has a total of 8 co-authors. Here are the top co-authors by number of shared publications:
- Jemin Lee
Department of Computer Science and Artificial Intelligence, Jeonbuk National University, Jeonju-si, Republic of Korea
2 shared publications - Seokhun Jeon
Korea Electronics Technology Institute, Seongnam-si, Republic of Korea
2 shared publications - Sihyeong Park
SoC Platform Research Center, Korea Electronics Technology Institute, Seongnam-si, Republic of Korea
2 shared publications - Chaelyn Lee
Korea Electronics Technology Institute, Seongnam-si, Republic of Korea
1 shared publication - Jae Hyung Ko
Korea Electronics Technology Institute, Seongnam, Gyeonggi-do, Republic of Korea
1 shared publication - Kyu Hyun Choi
Korea Electronics Technology Institute, Seongnam, Gyeonggi-do, Republic of Korea
1 shared publication
Latest Publications
- Top Cited PapersOpen Access
- 29 October 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
- Vol. 18 (1) , 1-111
A Survey on Inference Engines for Large Language Models: Perspectives on Optimization and Efficiency
Large Language Models (LLMs) are widely applied in chatbots, code generators, and search engines. Workload such as chain-of-thought, complex reasoning, and agent services significantly increase inference cost by invoking the model repeatedly. Optimization methods such as parallelism, compression, and caching have been adopted to reduce costs, but the diverse service requirements make it hard to select the right method. Recently, specialized LLM inference engines have emerged as a key component for integrating the optimization methods into service-oriented infrastructures. However, a systematic study on inference engines is still lacking. This article provides a comprehensive evaluation of 25 open source and commercial inference engines. We analyze each inference engine in terms of ease-of-use, ease-of-deployment, general-purpose support, scalability, and suitability for throughput- and latency-aware computation. Furthermore, we explore the design goals of each inference engine by investigating the optimization techniques it supports. In addition, we assess the ecosystem maturity of open source inference engines and handle the performance and cost policy of commercial solutions. We outline future research directions that include support for complex LLM-based services, support of various hardware, and enhanced security, offering practical guidance to researchers and developers in selecting and designing optimized LLM inference engines. We also provide a public repository to continually track developments in this fast-evolving field: https://github.com/sihyeong/Awesome-LLM-Inference-Engine .
- 29 October 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
- Vol. 18 (1) , 1-111
Top Cited PapersOpen AccessA Survey on Inference Engines for Large Language Models: Perspectives on Optimization and Efficiency
Large Language Models (LLMs) are widely applied in chatbots, code generators, and search engines. Workload such as chain-of-thought, complex reasoning, and agent services significantly increase inference cost by invoking the model repeatedly. Optimization methods such as parallelism, compression, and caching have been adopted to reduce costs, but the diverse service requirements make it hard to select the right method. Recently, specialized LLM inference engines have emerged as a key component for integrating the optimization methods into service-oriented infrastructures. However, a systematic study on inference engines is still lacking. This article provides a comprehensive evaluation of 25 open source and commercial inference engines. We analyze each inference engine in terms of ease-of-use, ease-of-deployment, general-purpose support, scalability, and suitability for throughput- and latency-aware computation. Furthermore, we explore the design goals of each inference engine by investigating the optimization techniques it supports. In addition, we assess the ecosystem maturity of open source inference engines and handle the performance and cost policy of commercial solutions. We outline future research directions that include support for complex LLM-based services, support of various hardware, and enhanced security, offering practical guidance to researchers and developers in selecting and designing optimized LLM inference engines. We also provide a public repository to continually track developments in this fast-evolving field: https://github.com/sihyeong/Awesome-LLM-Inference-Engine .
- Open Access
- 17 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 196040-196049
A Compute-in-Memory Decoding Architecture for Posit-to-Floating-Point Conversion
Modern computing systems are increasingly adopting low-precision floating-point formats to reduce memory overhead, particularly in large-scale applications such as large language models and scientific computing. Low-precision formats, however, are vulnerable to reduced numerical accuracy and provide only a limited dynamic range. The posit format, which incorporates a variable-length regime structure, has gained attention for providing higher precision and a broader dynamic range compared to conventional low-precision floating-point representations. Despite such advantages, conventional computing architectures are still based on IEEE 754 floating-point arithmetic, where achieving parallel and efficient posit-to-floating-point decoding remains a significant challenge. This paper proposes a Compute-in-Memory (CIM) architecture for posit-to-floating-point conversion that leverages the inherent parallelism of memory arrays to perform concurrent conversion of multiple posit values in a fixed number of cycles with negligible hardware overhead. The architecture implements key modules for posit decoding through bitwise in-memory logic operations, extending the state-of-the-art PIM microarchitecture to enable a wider range of micro-operations. Experimental results based on RRAM crossbar simulation demonstrate an average improvements of 3.3× and 5.04× in throughput and energy efficiency, respectively, compared to a baseline CMOS decoding architecture.
- 17 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 196040-196049
Open AccessA Compute-in-Memory Decoding Architecture for Posit-to-Floating-Point Conversion
Modern computing systems are increasingly adopting low-precision floating-point formats to reduce memory overhead, particularly in large-scale applications such as large language models and scientific computing. Low-precision formats, however, are vulnerable to reduced numerical accuracy and provide only a limited dynamic range. The posit format, which incorporates a variable-length regime structure, has gained attention for providing higher precision and a broader dynamic range compared to conventional low-precision floating-point representations. Despite such advantages, conventional computing architectures are still based on IEEE 754 floating-point arithmetic, where achieving parallel and efficient posit-to-floating-point decoding remains a significant challenge. This paper proposes a Compute-in-Memory (CIM) architecture for posit-to-floating-point conversion that leverages the inherent parallelism of memory arrays to perform concurrent conversion of multiple posit values in a fixed number of cycles with negligible hardware overhead. The architecture implements key modules for posit decoding through bitwise in-memory logic operations, extending the state-of-the-art PIM microarchitecture to enable a wider range of micro-operations. Experimental results based on RRAM crossbar simulation demonstrate an average improvements of 3.3× and 5.04× in throughput and energy efficiency, respectively, compared to a baseline CMOS decoding architecture.
- 30 June 2025
- Published by Korean Institute of Information Technology in The Journal of Korean Institute of Information Technology
- Vol. 23 (6) , 75-89
A Survey of Proprietary Accelerators for Large Language Models
대형 언어 모델(LLMsLarge Language Models)의 규모가 빠르게 확대되고, 동시에 모델 학습 및 추론 비용이 기하급수적으로 증가함에 따라 이를 효율적으로 처리하기 위한 전용 하드웨어 가속기의 중요성이 커지고 있다. 본 논문에서는 LLM 가속기의 필요성을 논의하고, NVIDIA, AMD, Google 등 주요 기업에서 개발한 상용 Application-Specific Integrated Circuit 기반 LLM 가속기의 하드웨어 및 소프트웨어 특성을 메모리 아키텍처, 성능, 인터커넥션, 소프트웨어 스택 등 다양한 관점에서 종합적으로 분석한다. 또한 LLM 가속기가 직면한 메모리 용량, 전력 효율, 확장성, 소프트웨어 호환성 등의 주요 기술적 도전 과제를 설명하고, 하드웨어 개발 방안을 검토한다. 이러한 분석을 바탕으로 혼합 정밀도 지원, 대규모 분산 학습을 위한 고속 인터커넥션, 컴파일러 등의 핵심 고려 사항을 제시하고, 나아가 차세대 LLM 가속기 개발에 필요한 연구 방향을 제안한다.
- 30 June 2025
- Published by Korean Institute of Information Technology in The Journal of Korean Institute of Information Technology
- Vol. 23 (6) , 75-89
A Survey of Proprietary Accelerators for Large Language Models
대형 언어 모델(LLMsLarge Language Models)의 규모가 빠르게 확대되고, 동시에 모델 학습 및 추론 비용이 기하급수적으로 증가함에 따라 이를 효율적으로 처리하기 위한 전용 하드웨어 가속기의 중요성이 커지고 있다. 본 논문에서는 LLM 가속기의 필요성을 논의하고, NVIDIA, AMD, Google 등 주요 기업에서 개발한 상용 Application-Specific Integrated Circuit 기반 LLM 가속기의 하드웨어 및 소프트웨어 특성을 메모리 아키텍처, 성능, 인터커넥션, 소프트웨어 스택 등 다양한 관점에서 종합적으로 분석한다. 또한 LLM 가속기가 직면한 메모리 용량, 전력 효율, 확장성, 소프트웨어 호환성 등의 주요 기술적 도전 과제를 설명하고, 하드웨어 개발 방안을 검토한다. 이러한 분석을 바탕으로 혼합 정밀도 지원, 대규모 분산 학습을 위한 고속 인터커넥션, 컴파일러 등의 핵심 고려 사항을 제시하고, 나아가 차세대 LLM 가속기 개발에 필요한 연구 방향을 제안한다.