SHShan Huang

Shan Huang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Shan Huang has a total of 17 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • DiffAttn: Sparse Attention Architecture for Video Generation With Spatial Information-Guided Differential Estimation

    by Li Ding, Jun Liu, Shuaiheng Li, Tianlang Zhao, Shan Huang, Guohao Dai, Member

    Recent progress in video generation models (VGMs) has enabled long video generation with strong visual fidelity. VGM architectures have evolved from decomposed 2D attention (e.g., separate spatial/temporal attention) to 3D full attention that jointly processes all frames. However, inference latency remains a major bottleneck due to algorithmic and architectural inefficiencies. Differential methods have become a potential solution by leveraging sparsity to reduce computation. However, existing studies face two challenges: Challenge-1: The existing differential methods fail to identify the weakly connected parts in VGM that can be approximately calculated, leading to many potential redundant computations or severe impacts on generation quality (> 5% loss). Challenge-2: The traditional architecture fails to effectively utilize the unique redundancy distribution (i.e., column-concentrated pattern) in compressed VGMs. To address the above challenges, we propose DiffAttn, a sparse attention architecture for video generation with differential approximation. Building on our prior conference work (ViDA), DiffAttn generalizes differential computing to 3D full attention and strengthens architectural support to exploit the resulting column-concentrated sparsity. It combines three tightly coupled techniques. (1) Spatial information-guided differential estimation method utilizes spatial information to estimate the weakly connected regions in attention computation, which reduces the attention computation by 66.84%. (2) Column-concentrated computing architecture fully leverages the unique column-concentrated sparsity characteristics in the compressed VGMs, which achieves 1.52× speedup over a fully dense architecture. (3) Sparsity-aware inter-layer dataflow dynamically adjusts the workload ratio between the two cores by analyzing layer-wise sparsity and column distribution at runtime, improving the utilization by 36.33%. Experiments on advanced VGMs (e.g., CogVideoX1.5, OpenSora2.0) show that DiffAttn achieves up to 2.62× and 1.89× end-to-end speedup and 3.14×/ and 2.87× energy efficiency improvement over NVIDIA A800 GPU and advanced VGM accelerator, respectively.

  • MARCA-v2: Mamba Accelerator With Complementary State-Space Model Sparsity and Reconfigurable Architecture

    by Jinhao Li, Shan Huang, Jiaming Xu, Jun Liu, Ningyi Xu, Guohao Dai

    Large language models (LLMs) with a state-space model (SSM), especially Mamba, have demonstrated remarkable capabilities in various domains. Compared to Transformers, Mamba reduces the quadratic computational complexity and achieves a higher algorithm performance. Current research on Mamba focuses primarily on integrating it with various application scenarios. However, there is limited research on optimizing for Mamba processing. Therefore, we profile the processing carefully and identify three main challenges in Mamba computations: 1) Large memory access overhead of element-wise operations in the SSM. Based on the MARCA architecture, the time proportion of the SSM is still the bottleneck when the sequence length reaches 2048, accounting for 62.52% of the total runtime. In the SSM, the memory access overhead of element-wise operations accounts for 97.17%, leading to consuming 96.56% of the time. 2) Inefficient sparse element-wise (Sp-EW) execution on MARCA architecture. SOTA architectures like MARCA propose a reconfigurable reduction tree to accelerate dense element-wise operations but lack effective sparse support for sparse execution. When 30% of the element-wise operations are skipped, these skipped operations are still mapped to the PE array and trigger redundant execution cycles, incurring $1.43\times $ performance gap with the ideal. 3) Large-area overhead for the nonlinear function unit. Exponential function and SiLU are two main nonlinear functions in the SSM. Previous methods designed a specific unit for acceleration, leading to 38% and 18% area overheads of the PE. In response to these challenges, we propose a new Mamba accelerator with complementary SSM sparsity and reconfigurable architecture, MARCA-v2, based on MARCA, to support fast and energy-efficient Mamba computations. Three novel techniques are as follows: 1) Complementary SSM sparsity with column-wise granularity. We first profile the numerical distributions of activations in the SSM and propose a column-wise complementary static sparsity for SSM computation. To further enable lightweight and hardware-friendly sparse computation, we propose a $\delta $ -bitmap encoding scheme for compressed storage and introduce two abstractions for Sp-EW operations. 2) Lightweight Sp-EW architecture. Based on the hardware-friendly sparsity algorithm and MARCA architecture, we design and integrate a lightweight metadata processing unit (Meta-PU) into the existing pipeline, which decodes the sparsity metadata and dynamically generates control signals to guide PE arrays. The overall architecture can efficiently support both dense and sparse operations, maximizing speed and energy efficiency. 3) Reusable nonlinear function unit based on reconfigurable PE arrays. We decompose the exponential function and SiLU into several element-wise operations. Thus, the reconfigurable PEs are fully reused to execute nonlinear functions with negligible accuracy loss. We conduct extensive experiments on Mamba model families with different sizes. Experimental results show that in the prefill stage, MARCA-v2 achieve 1.77– $10.87\times $ , 1.03– $1.08\times $ , and 4.78– $9.10\times $ speedup, and 8.29– $33.47\times $ /1.03- $1.08\times $ /4.78– $9.10\times $ energy efficiency improvement compared with Mamba–GPU, MARCA, and Spada, respec-tively. In the decode stage, MARCA-v2 achieve 0.88– $7.65\times $ / 1.00– $1.01\times $ /1.19– $1.64\times $ speedup and 3.11– $27.01\times $ /1.00– $1.01\times $ / 1.19– $1.64\times $ energy efficiency improvement compared with Mamba–GPU, MARCA, and Spada, respectively.


  • Open Access

    ViDA: Video Diffusion Transformer Acceleration with Differential Approximation and Adaptive Dataflow

    by Li Ding, Jun Liu, Shan Huang, Guohao Dai

    Recent advancements in Video Diffusion Transformer (VDiT) models have greatly promoted the development of video generation, as exemplified by Sora of OpenAI. However, there are still two challenges for VDiT: 1) There is still existing large inter-frame redundant computation. Previous works on reducing computation based on inter-frame similarity simply consider the Act-W operators. The remaining Act-Act operators still dominate the execution of VDiT (about 57%). 2) Operational intensity varies greatly, leading to under-utilization. There is a massive gap between the operational intensity of Act-W and Act-Act operators in VDiT with multiple frames. Previous works with the static hardware architecture and dataflow lead to under-utilization (ViDA, a Video Diffusion Transformer Accelerator with Diferential Approximation and Adaptive Dataflow. 1) At the algorithm level, we propose the differential approximation method that exploits similarity for both Act-Act and Act-W operators to reduce redundant computation. 2) At the hardware level, we propose the column-concentrated PE by exploiting the column sparsity pattern in differential computing. 3) At the dataflow level, we propose an intensity adaptive dataflow architecture to balance the execution of operators with significant operational intensity differences. Experiments show that ViDA achieves average 16.44×/2.18× speedup and 18.39×/2.35× area efficiency compared with NVIDIA A100 GPU and SOTA vision accelerator.


  • Open Access

    LLSM: LLM-enhanced Logic Synthesis Model with EDA-guided CoT Prompting, Hybrid Embedding and AIG-tailored Acceleration

    by Shan Huang, Jinhao Li, Zhen Yu, Jiancai Ye, Jiaming Xu, Ningyi Xu, Guohao Dai

    Machine learning-based methods have shown promising results in the field of Electronic Design Automation (EDA) like logic synthesis result prediction, enabling a shift-left in the overall EDA flow. Designers should fully optimize their Register Transfer Level (RTL) designs early because remedying low-quality RTL in downstream synthesis stages is extremely challenging. However, previous works mainly start modeling from the netlist level or layout level and apply Graph Neural Networks (GNNs) to make predictions. RTL captures the logic and scale information of circuits with uniform representation, making it suitable for a unified embedding approach. Since Large Language Models (LLMs) possess the ability to understand text modality, making them a potential method for understanding RTL and performing various EDA tasks. Therefore, we propose LLSM, the first LLM-enhanced logic synthesis model to extract information directly from RTL code. We also propose three novel approaches for LLSM in this paper. (1) EDA-guided Chain-of-Thought (CoT) prompting. We apply LLMs guided by circuit knowledge to summarize, analyze RTL code, and generate text with circuit information. (2) Text-circuit hybrid embedding. We train a small Language Model (LM) to encode the generated circuit information from LLM and fuse the embeddings of text and circuit modalities with weighted summation. (3) AIG-tailored acceleration library. We utilize an ELL2 format with zero padding tailored for And-Inverter-Graph (AIG) circuit representation and fuse the computation and format conversion. We also design a cacheable state strategy to avoid redundant computation for LM. We are the first to work with both LLM and GNN for the prediction of logic synthesis results and conduct extensive experiments on the OpenABC-D dataset. LLSM achieve up to 21.53% and 19.27% loss reduction in delay and area prediction respectively. It also achieves 1.34× and 6296.77× speedup on average compared to PyG implementation and Synopsis Design Compiler, respectively.


  • Enabling Efficient Sparse Multiplications on GPUs With Heuristic Adaptability

    by Jiaming Xu, Shan Huang, Jinhao Li, Guyue Huang, Yuan Xie, Yu Wang, Guohao Dai

    Sparse Matrix-Vector/Matrix Multiplication, namely SpMMul, has become a fundamental operation during model inference in various domains. Previous studies have explored numerous optimizations to accelerate it. However, to enable efficient end-to-end inference, the following challenges remain unsolved: (1) Incomplete design space and time-consuming preprocessing. Previous methods optimize SpMMul in limited loops and neglect the potential space exploration for further optimization, resulting in >30% waste of computing power. Additionally, the preprocessing overhead in SparseTIR and DTC-SpMM is 1000× larger than sparse computing. (2) Incompatibility between static dataflow and dynamic input. A static dataflow can not always be efficient to all input, leading to >80% performance loss. (3) Simplistic algorithm performance analysis. Previous studies primarily analyze performance from algorithmic advantages, without considering other aspects like hardware and data features. To tackle the above challenges, we present DA-SpMMul, a Data-Aware heuristic GPU implementation for SpMMul in multi-platforms. DA-SpMMul creatively proposes: (1) Complete design space based on theoretical computations and nontrivial implementations without preprocessing. We propose three orthogonal design principles based on theoretical computations and provide nontrivial implementations on standard formats, eliminating the complex preprocessing. (2) Feature-enabled adaptive algorithm selection mechanism. We design a heuristic model to enable algorithm selection considering various features. (3) Comprehensive algorithm performance analysis. We extract the features from multiple perspectives and present a comprehensive performance analysis of all algorithms. DA-SpMMul supports PyTorch on both NVIDIA and AMD and achieves an average speedup of 3.33× and 3.02× over NVIDIA cuSPARSE, and 12.05× and 8.32× over AMD rocSPARSE for SpMV and SpMM, and up to 1.48× speedup against the state-of-the-art open-source algorithm. Integrated with GNN framework, PyG, DA-SpMMul achieves up to 1.22× speedup on GCN inference.


  • Open Access

    MARCA: Mamba Accelerator with Reconfigurable Architecture

    by Jinhao Li, Shan Huang, Jiaming Xu, Jun Liu, Li Ding, Ningyi Xu, Guohao Dai