Cheng Li
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Interests
Cheng Li has a total of 32 co-authors. Here are the top co-authors by number of shared publications:
Wen-Mei Hwu
University of Illinois Urbana-Champaign
5 shared publicationsAbdul Dakkak
University of Illinois Urbana-Champaign
4 shared publicationsJinjun Xiong
IBM Research
3 shared publicationsArjun Khurana
University of Michigan
2 shared publicationsAustin Rovinski
University of Michigan
2 shared publicationsJason Mars
University of Michigan
2 shared publications
Latest Publications
- 1 July 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 372-382
TrIMS: Transparent and Isolated Model Sharing for Low Latency Deep Learning Inference in Function-as-a-Service
Deep neural networks (DNNs) have become core computation components within low latency Function as a Service (FaaS) prediction pipelines. Cloud computing, as the defacto backbone of modern computing infrastructure, has to be able to handle user-defined FaaS pipelines containing diverse DNN inference workloads while maintaining isolation and latency guarantees with minimal resource waste. The current solution for guaranteeing isolation and latency within FaaS is inefficient. A major cause of the inefficiency is the need to move large amount of data within and across servers. We propose TrIMS as a novel solution to address this issue. TrIMSis a generic memory sharing technique that enables constant data to be shared across processes or containers while still maintaining isolation between users. TrIMS consists of a persistent model store across the GPU, CPU, local storage, and cloud storage hierarchy, an efficient resource management layer that provides isolation, and a succinct set of abstracts, applicationAPIs, and container technologies for easy and transparent integration with FaaS, Deep Learning (DL) frameworks, and user code. We demonstrate our solution by interfacing TrIMS with the Apache MXNet framework and demonstrate up to 24x speedup in latency for image classification models, up to 210x speedup for large models, and up to8×system throughput improvement.
- 1 July 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 372-382
TrIMS: Transparent and Isolated Model Sharing for Low Latency Deep Learning Inference in Function-as-a-Service
Deep neural networks (DNNs) have become core computation components within low latency Function as a Service (FaaS) prediction pipelines. Cloud computing, as the defacto backbone of modern computing infrastructure, has to be able to handle user-defined FaaS pipelines containing diverse DNN inference workloads while maintaining isolation and latency guarantees with minimal resource waste. The current solution for guaranteeing isolation and latency within FaaS is inefficient. A major cause of the inefficiency is the need to move large amount of data within and across servers. We propose TrIMS as a novel solution to address this issue. TrIMSis a generic memory sharing technique that enables constant data to be shared across processes or containers while still maintaining isolation between users. TrIMS consists of a persistent model store across the GPU, CPU, local storage, and cloud storage hierarchy, an efficient resource management layer that provides isolation, and a succinct set of abstracts, applicationAPIs, and container technologies for easy and transparent integration with FaaS, Deep Learning (DL) frameworks, and user code. We demonstrate our solution by interfacing TrIMS with the Apache MXNet framework and demonstrate up to 24x speedup in latency for image classification models, up to 210x speedup for large models, and up to8×system throughput improvement.
- Open Access
- 26 June 2019
- Published by Association for Computing Machinery (ACM)
- p. 46-57
Accelerating reduction and scan using tensor core units
- 26 June 2019
- Published by Association for Computing Machinery (ACM)
- p. 46-57
Open AccessAccelerating reduction and scan using tensor core units
- Open Access
- 4 April 2019
- Published by Association for Computing Machinery (ACM)
- p. 209-218
Evaluating Characteristics of CUDA Communication Primitives on High-Bandwidth Interconnects
- 4 April 2019
- Published by Association for Computing Machinery (ACM)
- p. 209-218
Open AccessEvaluating Characteristics of CUDA Communication Primitives on High-Bandwidth Interconnects
- Open Access
- 13 August 2018
- Published by Association for Computing Machinery (ACM)
- p. 1-10
Matrix Factorization on GPUs with Memory Optimization and Approximate Computing
- 13 August 2018
- Published by Association for Computing Machinery (ACM)
- p. 1-10
Open AccessMatrix Factorization on GPUs with Memory Optimization and Approximate Computing
- 1 May 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 315-322
RAI: A Scalable Project Submission System for Parallel Programming Courses
A major component of many advanced programming courses is an open-ended "end-of-term project" assignment. Delivering and evaluating open-ended parallel programming projects for hundreds or thousands of students brings a need for broad system reconfigurability coupled with challenges of testing and development uniformity, access to esoteric hardware and programming environments, scalability, and security. We present RAI, a secure and extensible system for delivering open-ended programming assignments configured with access to different hardware and software requirements. We describe how the system was used to deliver a programming-competition-style final project in an introductory GPU programming course at the University of Illinois Urbana-Champaign.
- 1 May 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 315-322
RAI: A Scalable Project Submission System for Parallel Programming Courses
A major component of many advanced programming courses is an open-ended "end-of-term project" assignment. Delivering and evaluating open-ended parallel programming projects for hundreds or thousands of students brings a need for broad system reconfigurability coupled with challenges of testing and development uniformity, access to esoteric hardware and programming environments, scalability, and security. We present RAI, a secure and extensible system for delivering open-ended programming assignments configured with access to different hardware and software requirements. We describe how the system was used to deliver a programming-competition-style final project in an introductory GPU programming course at the University of Illinois Urbana-Champaign.
- 1 October 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-12
KLAP: Kernel launch aggregation and promotion for optimizing dynamic parallelism
Dynamic parallelism on GPUs simplifies the programming of many classes of applications that generate paral-lelizable work not known prior to execution. However, modern GPUs architectures do not support dynamic parallelism efficiently due to the high kernel launch overhead, limited number of simultaneous kernels, and limited depth of dynamic calls a device can support. In this paper, we propose Kernel Launch Aggregation and Promotion (KLAP), a set of compiler techniques that improve the performance of kernels which use dynamic parallelism. Kernel launch aggregation fuses kernels launched by threads in the same warp, block, or kernel into a single aggregated kernel, thereby reducing the total number of kernels spawned and increasing the amount of work per kernel to improve occupancy. Kernel launch promotion enables early launch of child kernels to extract more parallelism between parents and children, and to aggregate kernel launches across generations mitigating the problem of limited depth. We implement our techniques in a real compiler and show that kernel launch aggregation obtains a geometric mean speedup of 6.58x over regular dynamic parallelism. We also show that kernel launch promotion enables cases that were not originally possible, improving throughput by a geometric mean of 30.44 x.
- 1 October 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-12
KLAP: Kernel launch aggregation and promotion for optimizing dynamic parallelism
Dynamic parallelism on GPUs simplifies the programming of many classes of applications that generate paral-lelizable work not known prior to execution. However, modern GPUs architectures do not support dynamic parallelism efficiently due to the high kernel launch overhead, limited number of simultaneous kernels, and limited depth of dynamic calls a device can support. In this paper, we propose Kernel Launch Aggregation and Promotion (KLAP), a set of compiler techniques that improve the performance of kernels which use dynamic parallelism. Kernel launch aggregation fuses kernels launched by threads in the same warp, block, or kernel into a single aggregated kernel, thereby reducing the total number of kernels spawned and increasing the amount of work per kernel to improve occupancy. Kernel launch promotion enables early launch of child kernels to extract more parallelism between parents and children, and to aggregate kernel launches across generations mitigating the problem of limited depth. We implement our techniques in a real compiler and show that kernel launch aggregation obtains a geometric mean speedup of 6.58x over regular dynamic parallelism. We also show that kernel launch promotion enables cases that were not originally possible, improving throughput by a geometric mean of 30.44 x.
- 25 May 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Micro
- Vol. 36 (3) , 42-53
Sirius Implications for Future Warehouse-Scale Computers
Demand is expected to grow significantly for cloud services that deliver sophisticated artificial intelligence on the critical path of user queries, as is the case with intelligent personal assistants such as Apple's Siri. If the prediction of the trend is correct, these types of applications will likely consume most of the world's computing cycles. The Sirius project was motivated to investigate what this future might look like and how cloud architectures should evolve to achieve it.
- 25 May 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Micro
- Vol. 36 (3) , 42-53
Sirius Implications for Future Warehouse-Scale Computers
Demand is expected to grow significantly for cloud services that deliver sophisticated artificial intelligence on the critical path of user queries, as is the case with intelligent personal assistants such as Apple's Siri. If the prediction of the trend is correct, these types of applications will likely consume most of the world's computing cycles. The Sirius project was motivated to investigate what this future might look like and how cloud architectures should evolve to achieve it.
- Open Access
- 6 April 2016
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computer Systems
- Vol. 34 (1) , 1-32
Designing Future Warehouse-Scale Computers for Sirius, an End-to-End Voice and Vision Personal Assistant
As user demand scales for intelligent personal assistants (IPAs) such as Apple’s Siri, Google’s Google Now, and Microsoft’s Cortana, we are approaching the computational limits of current datacenter (DC) architectures. It is an open question how future server architectures should evolve to enable this emerging class of applications, and the lack of an open-source IPA workload is an obstacle in addressing this question. In this article, we present the design of Sirius, an open end-to-end IPA Web-service application that accepts queries in the form of voice and images, and responds with natural language. We then use this workload to investigate the implications of four points in the design space of future accelerator-based server architectures spanning traditional CPUs, GPUs, manycore throughput co-processors, and FPGAs. To investigate future server designs for Sirius, we decompose Sirius into a suite of eight benchmarks (Sirius Suite) comprising the computationally intensive bottlenecks of Sirius. We port Sirius Suite to a spectrum of accelerator platforms and use the performance and power trade-offs across these platforms to perform a total cost of ownership (TCO) analysis of various server design points. In our study, we find that accelerators are critical for the future scalability of IPA services. Our results show that GPU- and FPGA-accelerated servers improve the query latency on average by 8.5× and 15×, respectively. For a given throughput, GPU- and FPGA-accelerated servers can reduce the TCO of DCs by 2.3× and 1.3×, respectively.
- 6 April 2016
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computer Systems
- Vol. 34 (1) , 1-32
Open AccessDesigning Future Warehouse-Scale Computers for Sirius, an End-to-End Voice and Vision Personal Assistant
As user demand scales for intelligent personal assistants (IPAs) such as Apple’s Siri, Google’s Google Now, and Microsoft’s Cortana, we are approaching the computational limits of current datacenter (DC) architectures. It is an open question how future server architectures should evolve to enable this emerging class of applications, and the lack of an open-source IPA workload is an obstacle in addressing this question. In this article, we present the design of Sirius, an open end-to-end IPA Web-service application that accepts queries in the form of voice and images, and responds with natural language. We then use this workload to investigate the implications of four points in the design space of future accelerator-based server architectures spanning traditional CPUs, GPUs, manycore throughput co-processors, and FPGAs. To investigate future server designs for Sirius, we decompose Sirius into a suite of eight benchmarks (Sirius Suite) comprising the computationally intensive bottlenecks of Sirius. We port Sirius Suite to a spectrum of accelerator platforms and use the performance and power trade-offs across these platforms to perform a total cost of ownership (TCO) analysis of various server design points. In our study, we find that accelerators are critical for the future scalability of IPA services. Our results show that GPU- and FPGA-accelerated servers improve the query latency on average by 8.5× and 15×, respectively. For a given throughput, GPU- and FPGA-accelerated servers can reduce the TCO of DCs by 2.3× and 1.3×, respectively.
- Top Cited PapersOpen Access
- 29 May 2013
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Stochastic circuits for real-time image-processing applications
- 29 May 2013
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Top Cited PapersOpen AccessStochastic circuits for real-time image-processing applications