Cong Hao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Cong Hao has a total of 93 co-authors. Here are the top co-authors by number of shared publications:
- Deming Chen
University of Illinois Urbana-Champaign, Urbana, IL, USA
5 shared publications - Hanqiu Chen
Georgia Institute of Technology
5 shared publications - Hang Yang
Georgia Institute of Technology Atlanta USA
4 shared publications - Yuan Xie
The Hong Kong University of Science and Technology, Hong Kong, Hong Kong
4 shared publications - Nan Wu
Department of Orthopedic Surgery, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Peking Union Medical College and Chinese Academy of Medical Sciences, Beijing, China
3 shared publications - Cunxi Yu
University of Maryland, College Park, Salt Lake City, MD, USA
2 shared publications
Latest Publications
- Open Access
- 26 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Faster-MoA: Low-Latency Tree-Structured MoA Serving with Early Exit and Agent-Aware Prefill-Decode Overlap
- 26 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Open AccessFaster-MoA: Low-Latency Tree-Structured MoA Serving with Early Exit and Agent-Aware Prefill-Decode Overlap
- Open Access
- 26 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Escaping Flatland: A Placement Flow for Enabling 3D FPGAs
- 26 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Open AccessEscaping Flatland: A Placement Flow for Enabling 3D FPGAs
- Open Access
- 17 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 1334-1346
OmniSim: Simulating Hardware with C Speed and RTL Accuracy for High-Level Synthesis Designs
- 17 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 1334-1346
Open AccessOmniSim: Simulating Hardware with C Speed and RTL Accuracy for High-Level Synthesis Designs
- 8 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
- Vol. 45 (3) , 1352-1365
Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimization
Transformer models have achieved state-of-the-art performance across a wide range of machine learning tasks. There is growing interest in training transformers on resource-constrained edge devices due to considerations, such as privacy, domain adaptation, and on-device scientific machine learning. However, the significant computational and memory demands required for transformer training often exceed the capabilities of an edge device. Leveraging low-rank tensor compression, this paper presents the first on-FPGA accelerator for transformer training. On the algorithm side, we present a bi-directional contraction flow for tensorized transformer training, significantly reducing the computational FLOPS and intralayer memory costs compared to existing tensor operations. On the hardware side, we store all highly compressed model parameters and gradient information on chip, creating an on-chip-memory-only framework for each stage in training. This reduces off-chip communication and minimizes latency and energy costs. Additionally, we implement custom computing kernels for each training stage and employ intralayer parallelism and pipe-lining to further enhance run-time and memory efficiency. Through experiments on transformer models within 36.7 to 93.5 MB using forward propagation (FP)-32 data formats on the ATIS dataset, our tensorized FPGA accelerator could conduct single-batch end-to-end training on the AMD Alevo U50 FPGA, with a memory budget of less than 6-MB BRAM and 22.5-MB URAM. Compared to uncompressed training on the NVIDIA RTX 3090 GPU, our on-FPGA training achieves a memory reduction of$30 \times $ to$51 \times $ . Our FPGA accelerator also achieves up to$4.0 \times $ less energy cost per epoch compared with tensor transformer training on an NVIDIA RTX 3090 GPU. As an initial result, this article highlights the significant potential of large-scale tensor training on edge devices.- 8 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
- Vol. 45 (3) , 1352-1365
Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimization
Transformer models have achieved state-of-the-art performance across a wide range of machine learning tasks. There is growing interest in training transformers on resource-constrained edge devices due to considerations, such as privacy, domain adaptation, and on-device scientific machine learning. However, the significant computational and memory demands required for transformer training often exceed the capabilities of an edge device. Leveraging low-rank tensor compression, this paper presents the first on-FPGA accelerator for transformer training. On the algorithm side, we present a bi-directional contraction flow for tensorized transformer training, significantly reducing the computational FLOPS and intralayer memory costs compared to existing tensor operations. On the hardware side, we store all highly compressed model parameters and gradient information on chip, creating an on-chip-memory-only framework for each stage in training. This reduces off-chip communication and minimizes latency and energy costs. Additionally, we implement custom computing kernels for each training stage and employ intralayer parallelism and pipe-lining to further enhance run-time and memory efficiency. Through experiments on transformer models within 36.7 to 93.5 MB using forward propagation (FP)-32 data formats on the ATIS dataset, our tensorized FPGA accelerator could conduct single-batch end-to-end training on the AMD Alevo U50 FPGA, with a memory budget of less than 6-MB BRAM and 22.5-MB URAM. Compared to uncompressed training on the NVIDIA RTX 3090 GPU, our on-FPGA training achieves a memory reduction of$30 \times $ to$51 \times $ . Our FPGA accelerator also achieves up to$4.0 \times $ less energy cost per epoch compared with tensor transformer training on an NVIDIA RTX 3090 GPU. As an initial result, this article highlights the significant potential of large-scale tensor training on edge devices.- Open Access
- 16 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 242-250
Invited: Modeling and Design Methodology for Backside Integration of Voltage Converters
- 16 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 242-250
Open AccessInvited: Modeling and Design Methodology for Backside Integration of Voltage Converters
- Open Access
- 20 January 2025
- Published by Association for Computing Machinery (ACM)
- p. 1041-1047
Exploring and Exploiting Runtime Reconfigurable Floating Point Precision in Scientific Computing: a Case Study for Solving PDEs
Scientific computing applications, such as computational fluid dynamics and climate modeling, typically rely on 64-bit double-precision floating-point operations, which are extremely costly in terms of computation, memory, and energy. While the machine learning community has successfully utilized low-precision computations, scientific computing remains cautious due to concerns about numerical stability. To tackle this long-standing challenge, we propose a novel approach to dynamically adjust the floating-point data precision at runtime, maintaining computational fidelity using lower bit widths. We first conduct a thorough analysis of data range distributions during scientific simulations to identify opportunities and challenges for dynamic precision adjustment. We then propose a runtime reconfigurable, flexible floating-point multiplier (R2F2), which automatically and dynamically adjusts multiplication precision based on the current operands, ensuring accurate results with lower bit widths. Our evaluation shows that 16-bit R2F2 significantly reduces error rates by 70.2% compared to standard half-precision, with resource overhead ranging from a 5% reduction to a 7% increase and no latency overhead. In two representative scientific computing applications, R2F2, using 16 or fewer bits, can achieve the same simulation results as 32-bit precision, while standard half precision will fail. This study pioneers runtime reconfigurable arithmetic, demonstrating great potential to enhance scientific computing efficiency. Code available at https://github.com/sharc-lab/R2F2.
- 20 January 2025
- Published by Association for Computing Machinery (ACM)
- p. 1041-1047
Open AccessExploring and Exploiting Runtime Reconfigurable Floating Point Precision in Scientific Computing: a Case Study for Solving PDEs
Scientific computing applications, such as computational fluid dynamics and climate modeling, typically rely on 64-bit double-precision floating-point operations, which are extremely costly in terms of computation, memory, and energy. While the machine learning community has successfully utilized low-precision computations, scientific computing remains cautious due to concerns about numerical stability. To tackle this long-standing challenge, we propose a novel approach to dynamically adjust the floating-point data precision at runtime, maintaining computational fidelity using lower bit widths. We first conduct a thorough analysis of data range distributions during scientific simulations to identify opportunities and challenges for dynamic precision adjustment. We then propose a runtime reconfigurable, flexible floating-point multiplier (R2F2), which automatically and dynamically adjusts multiplication precision based on the current operands, ensuring accurate results with lower bit widths. Our evaluation shows that 16-bit R2F2 significantly reduces error rates by 70.2% compared to standard half-precision, with resource overhead ranging from a 5% reduction to a 7% increase and no latency overhead. In two representative scientific computing applications, R2F2, using 16 or fewer bits, can achieve the same simulation results as 32-bit precision, while standard half precision will fail. This study pioneers runtime reconfigurable arithmetic, demonstrating great potential to enhance scientific computing efficiency. Code available at https://github.com/sharc-lab/R2F2.
- Open Access
- 27 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-9
Residual-INR: Communication Efficient On-Device Learning Using Implicit Neural Representation
- 27 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-9
Open AccessResidual-INR: Communication Efficient On-Device Learning Using Implicit Neural Representation
- Open Access
- 9 September 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-9
HLSFactory: A Framework Empowering High-Level Synthesis Datasets for Machine Learning and Beyond
Machine learning (ML) techniques have been applied to high-level synthesis (HLS) flows for quality-of-result (QoR) prediction and design space exploration (DSE). Nevertheless, the scarcity of accessible high-quality HLS datasets and the complexity of building such datasets present great challenges to FPGA and ML researchers. Existing datasets either cover only a subset of previously published benchmarks, provide no way to enumerate optimization design spaces, are limited to a specific vendor, or have no reproducible and extensible software for dataset construction. Many works also lack user-friendly ways to add more designs to existing datasets, limiting wider adoption and sustainability of such datasets. In response to these challenges, we introduce HLSFactory, a comprehensive framework designed to facilitate the curation and generation of high-quality HLS design datasets. HLSFactory has three main stages: 1) a design space expansion stage to elaborate single HLS designs into large design spaces using various optimization directives across multiple vendor tools, 2) a design synthesis stage to execute HLS and FPGA tool flows concurrently across designs, and 3) a data aggregation stage for extracting standardized data into packaged datasets for ML usage. This tripartite architecture not only ensures broad coverage of data points via design space expansion but also supports interoperability with tools from multiple vendors. Users can contribute to each stage easily by submitting their own HLS designs or synthesis results via provided user APIs. The framework is also flexible, allowing extensions at every step via user APIs with custom frontends, synthesis tools, and scripts. To demonstrate the framework functionality, we include an initial set of built-in base designs from PolyBench, MachSuite, Rosetta, CHStone, Kastner et al.'s Parallel Programming for FPGAs, and curated kernels from existing open-source HLS designs. We report the statistical analyses and design space visualizations to demonstrate the completed end-to-end compilation flow, and to highlight the effectiveness of our design space expansion beyond the initial base dataset, which greatly contributes to dataset diversity and coverage. In addition to its evident application in ML, we showcase the versatility and multi-functionality of our framework through seven case studies: I) Building an ML model for post-implementation QoR prediction II) Using design space sampling in stage 1 to expand the design space covered from a small base set of HLS designs; III) Demonstrating the speedup from the fine-grained design parallelism backend; IV) Extending HLSFactory to target Intel's HLS flow across all stages; V) Adding and running new auxiliary designs using HLSFactory; VI) Integration of previously published HLS data in stage 3; VII) Using HLSFactory to perform HLS tool version regression benchmarking. Code available at https://github.com/sharc-lab/HLSFactory.
- 9 September 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-9
Open AccessHLSFactory: A Framework Empowering High-Level Synthesis Datasets for Machine Learning and Beyond
Machine learning (ML) techniques have been applied to high-level synthesis (HLS) flows for quality-of-result (QoR) prediction and design space exploration (DSE). Nevertheless, the scarcity of accessible high-quality HLS datasets and the complexity of building such datasets present great challenges to FPGA and ML researchers. Existing datasets either cover only a subset of previously published benchmarks, provide no way to enumerate optimization design spaces, are limited to a specific vendor, or have no reproducible and extensible software for dataset construction. Many works also lack user-friendly ways to add more designs to existing datasets, limiting wider adoption and sustainability of such datasets. In response to these challenges, we introduce HLSFactory, a comprehensive framework designed to facilitate the curation and generation of high-quality HLS design datasets. HLSFactory has three main stages: 1) a design space expansion stage to elaborate single HLS designs into large design spaces using various optimization directives across multiple vendor tools, 2) a design synthesis stage to execute HLS and FPGA tool flows concurrently across designs, and 3) a data aggregation stage for extracting standardized data into packaged datasets for ML usage. This tripartite architecture not only ensures broad coverage of data points via design space expansion but also supports interoperability with tools from multiple vendors. Users can contribute to each stage easily by submitting their own HLS designs or synthesis results via provided user APIs. The framework is also flexible, allowing extensions at every step via user APIs with custom frontends, synthesis tools, and scripts. To demonstrate the framework functionality, we include an initial set of built-in base designs from PolyBench, MachSuite, Rosetta, CHStone, Kastner et al.'s Parallel Programming for FPGAs, and curated kernels from existing open-source HLS designs. We report the statistical analyses and design space visualizations to demonstrate the completed end-to-end compilation flow, and to highlight the effectiveness of our design space expansion beyond the initial base dataset, which greatly contributes to dataset diversity and coverage. In addition to its evident application in ML, we showcase the versatility and multi-functionality of our framework through seven case studies: I) Building an ML model for post-implementation QoR prediction II) Using design space sampling in stage 1 to expand the design space covered from a small base set of HLS designs; III) Demonstrating the speedup from the fine-grained design parallelism backend; IV) Extending HLSFactory to target Intel's HLS flow across all stages; V) Adding and running new auxiliary designs using HLSFactory; VI) Integration of previously published HLS data in stage 3; VII) Using HLSFactory to perform HLS tool version regression benchmarking. Code available at https://github.com/sharc-lab/HLSFactory.
- Open Access
- 23 June 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-6
ICGMM: CXL-enabled Memory Expansion with Intelligent Caching Using Gaussian Mixture Model
Compute Express Link (CXL) emerges as a solution for wide gap between computational speed and data communication rates among host and multiple devices. It fosters a unified and coherent memory space between host and CXL storage devices such as such as Solid-state drive (SSD) for memory expansion, with a corresponding DRAM implemented as the device cache. However, this introduces challenges such as substantial cache miss penalties, sub-optimal caching due to data access granularity mismatch between the DRAM "cache" and SSD "memory", and inefficient hardware cache management. To address these issues, we propose a novel solution, named ICGMM, which optimizes caching and eviction directly on hardware, employing a Gaussian Mixture Model (GMM)-based approach. We prototype our solution on an FPGA board, which demonstrates a noteworthy improvement compared to the classic Least Recently Used (LRU) cache strategy. We observe a decrease in the cache miss rate ranging from 0.32% to 6.14%, leading to a substantial 16.23% to 39.14% reduction in the average SSD access latency. Furthermore, when compared to the state-of-the-art Long Short-Term Memory (LSTM)-based cache policies, our GMM algorithm on FPGA showcases an impressive latency reduction of over 10,000 times. Remarkably, this is achieved while demanding much fewer hardware resources.
- 23 June 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Open AccessICGMM: CXL-enabled Memory Expansion with Intelligent Caching Using Gaussian Mixture Model
Compute Express Link (CXL) emerges as a solution for wide gap between computational speed and data communication rates among host and multiple devices. It fosters a unified and coherent memory space between host and CXL storage devices such as such as Solid-state drive (SSD) for memory expansion, with a corresponding DRAM implemented as the device cache. However, this introduces challenges such as substantial cache miss penalties, sub-optimal caching due to data access granularity mismatch between the DRAM "cache" and SSD "memory", and inefficient hardware cache management. To address these issues, we propose a novel solution, named ICGMM, which optimizes caching and eviction directly on hardware, employing a Gaussian Mixture Model (GMM)-based approach. We prototype our solution on an FPGA board, which demonstrates a noteworthy improvement compared to the classic Least Recently Used (LRU) cache strategy. We observe a decrease in the cache miss rate ranging from 0.32% to 6.14%, leading to a substantial 16.23% to 39.14% reduction in the average SSD access latency. Furthermore, when compared to the state-of-the-art Long Short-Term Memory (LSTM)-based cache policies, our GMM algorithm on FPGA showcases an impressive latency reduction of over 10,000 times. Remarkably, this is achieved while demanding much fewer hardware resources.