Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Jinsung Kim has a total of 46 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Forget and Explain: Transparent Verification of GNN Unlearning

    by Imran Ahsan, Hyunwook Yu, Jinsung Kim, Mucheol Kim

  • Open Access

    Tensor Core-Adapted Sparse Matrix Multiplication for Accelerating Sparse Deep Neural Networks

    by Yoonsang Han, Inseo Kim, Jinsung Kim, Gordon Euhyun Moon

    Sparse matrix–matrix multiplication (SpMM) is essential for deep learning models and scientific computing. Recently, Tensor Cores (TCs) on GPUs, originally designed for dense matrix multiplication with mixed precision, have gained prominence. However, utilizing TCs for SpMM is challenging due to irregular memory access patterns and a varying number of non-zero elements in a sparse matrix. To improve data locality, previous studies have proposed reordering sparse matrices before multiplication, but this adds computational overhead. In this paper, we propose Tensor Core-Adapted SpMM (TCA-SpMM), which leverages TCs without requiring matrix reordering and uses the compressed sparse row (CSR) format. To optimize TC usage, the SpMM algorithm’s dot product operation is transformed into a blocked matrix–matrix multiplication. Addressing load imbalance and minimizing data movement are critical to optimizing the SpMM kernel. Our TCA-SpMM dynamically allocates thread blocks to process multiple rows simultaneously and efficiently uses shared memory to reduce data movement. Performance results on sparse matrices from the Deep Learning Matrix Collection public dataset demonstrate that TCA-SpMM achieves up to 29.58× speedup over state-of-the-art SpMM implementations optimized with TCs.


  • Open Access

    Exploring Attention Sparsity to Accelerate Transformer Training on GPUs

    by Bokyeong Yoon, Ah-Hyun Lee, Jinsung Kim, Gordon Euhyun Moon

    The computational complexity required for training a Transformer model quadratically increases as the length of the input sequence increases. Therefore, to accelerate the training of a large-scale Transformer with long sequences, it is crucial to reduce the number of operations for the multi-head attention computations, which dominate the overall Transformer training process. Previous approaches have sought to sparsify the multi-head attention before training by statically selecting the critical elements in the attention score matrix. However, since the critical elements in the attention score matrix can vary across different model tasks and datasets, dynamically considering the critical elements is essential for achieving better model quality. In this paper, we propose a new sparsity-aware Transformer that captures task- and input-dependent sparsity pattern in the attention score matrix during a small number of steps of the standard training of the Transformer. Then the identified sparsity pattern is utilized in the sparse training, transferred from the standard training, based on the degree of skewness and distance values of the attention score matrices. Experimental results demonstrate that our approach significantly reduces the number of operations in the multi-head attention operations, achieving up to 2.84×training speedup, 6.87×memory reduction and better accuracy compared to state-of-the-art sparse Transformer models.

  • Open Access

    GPU-accelerated non-dominated sorting genetic algorithm III for maximizing protein production

    by Donghyeon Kim, Jinsung Kim

    Maximizing protein expression levels poses a major challenge in bioengineering. To increase protein expression levels, numerous factors, including codon bias, codon context bias, hidden stop codons, homologous recombination, suitable guanine-cytosine ratio, and hairpin loop structure, are crucial and quantified by six objective functions: CAI, CPB, HSC, HD, GC3, and SL. Optimizing these six objectives simultaneously constitutes a multi-objective optimization problem, aiming to identify the favorable Pareto solutions rather than a singular optimal solution. However, achieving satisfactory solutions requires numerous cycles and solutions, thus leading to a large number of functional evaluations. While there are frameworks for multi-objective optimization problems, they often lack efficient support for objective function computation in protein encoding. In this paper, we proposed a method to design a set of coding sequences (CDSs) based on non-dominated sorting genetic algorithm III (NSGA-III), accelerated using NVIDIA graphical processing units (GPUs). Experimental results indicated that our method is 15,454 times faster than the Pymoo framework and is evaluated using 100 solutions and 100 cycles. Since our GPU implementation facilitated the use of larger solutions and more cycles, we were able to design a superior set of CDSs by increasing solutions to 400 and cycles to 12,800. In addition, our NSGA-III-based method consistently surpassed the NSGA-II approach when the number of cycles exceeded 3200 by utilizing 100 solutions. Finally, we observed that a gradual reduction of the mutation probability as the number of cycles increased yielded better quality results than maintaining a fixed mutation probability.


  • Open Access

    TAMM: Tensor algebra for many-body methods

    by Erdal Mutlu, Ajay Panyala, Nitin Gawande, Abhishek Bagusetty, Jeffrey Glabe, Jinsung Kim, Karol Kowalski, Nicholas P. Bauman, Bo Peng, Himadri Pathak, et al.

    Tensor algebra operations such as contractions in computational chemistry consume a significant fraction of the computing time on large-scale computing platforms. The widespread use of tensor contractions between large multi-dimensional tensors in describing electronic structure theory has motivated the development of multiple tensor algebra frameworks targeting heterogeneous computing platforms. In this paper, we present Tensor Algebra for Many-body Methods (TAMM), a framework for productive and performance-portable development of scalable computational chemistry methods. TAMM decouples the specification of the computation from the execution of these operations on available high-performance computing systems. With this design choice, the scientific application developers (domain scientists) can focus on the algorithmic requirements using the tensor algebra interface provided by TAMM, whereas high-performance computing developers can direct their attention to various optimizations on the underlying constructs, such as efficient data distribution, optimized scheduling algorithms, and efficient use of intra-node resources (e.g., graphics processing units). The modular structure of TAMM allows it to support different hardware architectures and incorporate new algorithmic advances. We describe the TAMM framework and our approach to the sustainable development of scalable ground- and excited-state electronic structure methods. We present case studies highlighting the ease of use, including the performance and productivity gains compared to other frameworks.


  • Open Access

    Optimization of designing multiple genes encoding the same protein based on NSGA-II for efficient execution on GPUs

    by Donghyeon Kim, Jinsung Kim

    In synthetic biology, it is a challenge to increase the production of target proteins by maximizing their expression levels. In order to augment expression levels, we need to focus on both homologous recombination and codon adaptation, which are estimated by three objective functions, namely HD (Hamming distance), LRCS (length of repeated or common substring) and CAI (codon adaptation index). Optimizing these objective functions simultaneously becomes a multi-objective optimization problem. The aim is to find satisfying solutions that have high codon adaptation and a low incidence of homologous recombination. However, obtaining satisfactory solutions requires calculating the objective functions multiple times with many cycles and solutions. In this paper, we propose an approach to accelerate the method of designing a set of CDSs (CoDing sequences) based on NSGA-II (non-dominated sorting genetic algorithm II) on NVIDIA GPUs. The implementation accelerated by GPUs improves overall performance by 187.5$ \times $ using $ 100 $ cycles and $ 128 $ solutions. Our implementation allows us to use larger solutions and more cycles, leading to outstanding solution quality. The improved implementation provides much better solutions in a similar amount of time compared to other available methods by 1.22$ \times $ improvements in hypervolume. Furthermore, our approach on GPUs also suggests how to efficiently utilize the latest computational resources in bioinformatics. Finally, we discuss the impacts of the number of cycles and the number of solutions on designing a set of CDSs.


  • A Code Generator for High-Performance Tensor Contractions on GPUs

    by Jinsung Kim, Aravind Sukumaran-Rajam, Vineeth Thumma, Sriram Krishnamoorthy, Ajay Panyala, Louis-Noel Pouchet, Atanas Rountev, P. Sadayappan

    Tensor contractions are higher dimensional generalizations of matrix-matrix multiplication. They form the compute-intensive core of many applications in computational science and data science. In this paper, we describe a high-performance GPU code generator for arbitrary tensor contractions. It exploits domain-specific properties about data reuse in tensor contractions to devise an effective code generation schema, coupled with an effective model-driven search, to determine parameters for mapping of computation to threads and staging of data through the GPU memory hierarchy. Experimental evaluation using a set of tensor contraction benchmarks demonstrates performance improvement and/or significantly reduced code generation time over other state-of-the-art tensor contraction libraries and code generators.