CWChong Wu

Chong Wu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Chong Wu has a total of 49 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    DuSA: Fast and Accurate Dual-Stage Sparse Attention Mechanism Accelerating Both Training and Inference

    by Chong Wu, Jiawang Cao, Renjie Xu, Zhuoheng Ran, Maolin Che, Wenbo Zhu, Hong Yan

    This paper proposes the Dual-Stage Sparse Attention (DuSA) mechanism for attention acceleration of transformers. In the first stage, DuSA performs intrablock sparse attention to aggregate local inductive biases. In the second stage, DuSA performs interblock sparse attention to obtain long-range dependencies. Both stages have low computational complexity and can be further accelerated by memory acceleration attention mechanisms directly, which makes DuSA faster than some extremely fast attention mechanisms. The dual-stage sparse attention design provides a lower error in approximating vanilla scaled-dot product attention than the basic single-stage sparse attention mechanisms and further advances the basic sparse attention mechanisms to match or even outperform vanilla scaled-dot product attention. Even in some plug and play situations, DuSA can still maintain low performance loss. DuSA can be used in both training and inference acceleration. DuSA achieves leading performance in different benchmarks: long range arena, image classification, semantic segmentation, object detection, text to video generation, and long context understanding, and accelerates models of different sizes.

  • The CUR Decomposition of Self-Attention Matrices in Vision Transformers

    by Chong Wu, Maolin Che, Hong Yan

    Transformers have achieved great success in natural language processing and computer vision. The core and basic technique of transformers is the self-attention mechanism. The vanilla self-attention mechanism has quadratic complexity, which limits its applications to vision tasks. Most of the existing linear self-attention mechanisms will sacrifice performance to some extent to reduce complexity. In this paper, we propose a novel linear approximation of the vanilla self-attention mechanism named CURSA to achieve both high performance and low complexity at the same time. CURSA is based on the CUR decomposition to decompose the multiplication of large matrices into the multiplication of several small matrices to achieve almost linear complexity. Experiment results of CURSA in image classification tasks, semantic segmentation tasks, object detection tasks, and long-range arena show that it outperforms state-of-the-art self-attention mechanisms with better data efficiency, faster speed, and higher accuracy.


  • RO-SVD+: A Hardware-Based Framework Against AI Attacks for Cost-Effective GenAI Copyright Protection in Consumer Electronics

    by Zhuoheng Ran, Xinmin Meng, Chong Wu, Zewen Ye, Ray C. C. Cheung, Hong Yan

    The widespread integration of generative artificial intelligence (GenAI) in consumer devices has heightened the need for a cost-effective and energy-efficient mechanism to safeguard the intellectual property (IP) of artificial intelligence-generated content (AIGC). This paper proposes RO-SVD+, a hardware-based framework that exploits low-rank matrix decomposition and on-chip process variations to enable device-level traceability and high-quality randomness generation while exhibiting strong resistance to AI-driven attacks. Using AI-generated images as a case study, evaluations of a 28-nm FPGA and an ASIC design flow show up to 185× and 153× improvements in per-pixel cost-effectiveness as image resolutions scale from 32×32 to 1024×1024 under consistent power consumption. Compared with our prior work, RO-SVD+ achieves a 2× speedup in entropy extraction and a 4× improvement in cost-effectiveness for 1024×1024 matrices. Compared to other equivalent efforts, RO-SVD+ delivers 8× to 53× higher throughput with reduced FPGA design complexity and ASIC area, making it a promising hardware-based solution for copyright protection in modern consumer electronics where high-resolution and dense GenAI workloads are widely deployed.

  • Open Access

    UniFormer: Unified and Efficient Transformer for Reasoning Across General and Custom Computing

    by Zhuoheng Ran, Chong Wu, Renjie Xu, Maolin Che, Hong Yan

    The success of neural networks such as convolutional neural networks (CNNs) has been largely attributed to their effective and widespread deployment on customised computing platforms, including field-programmable gate arrays (FPGAs) and application-specific integrated circuits (ASICs). In the current era, Transformer-based architectures underpin the majority of state-of-the-art (SOTA) larger models that are also increasingly deployed on customised computing hardware for low-power and real-time applications. However, the fundamentally different parallel computation paradigms between general-purpose and customised computing often lead to compromises in model transfer and deployability, which typically come at the cost of complexity, efficiency or accuracy. Moreover, many cross-platform optimisation principles have also remained underexplored in existing studies. This paper introduces UniFormer, a unified and efficient Transformer architecture for both general-purpose and customised computing platforms. By enabling higher parallelism and compute-storage fusion, UniFormer achieves state-of-the-art (SOTA) accuracy and latency on GPUs while exhibiting strong adaptability on FPGAs. To the best of our knowledge, this paper is the first efficient Transformer work that jointly considers both general-purpose and customised computing architectures.

  • Open Access

    ELFATT: Efficient Linear Fast Attention for Vision Transformers

    by Chong Wu, Maolin Che, Renjie Xu, Zhuoheng Ran, Hong Yan


  • Open Access

    The CUR Decomposition of Self-Attention Matrices in Vision Transformers

    by Chong Wu, Maolin Che, Hong Yan

    Transformers have achieved great success in natural language processing and computer vision. The core and basic technique of transformers is the self-attention mechanism. The vanilla self-attention mechanism has quadratic complexity, which limits its applications to vision tasks. Most of the existing linear self-attention mechanisms will sacrifice performance to some extent to reduce complexity. In this paper, we propose a novel linear approximation of the vanilla self-attention mechanism named CURSA to achieve both high performance and low complexity at the same time. CURSA is based on the CUR decomposition to decompose the multiplication of large matrices into the multiplication of several small matrices to achieve almost linear complexity. Experiment results of CURSA in image classification tasks, semantic segmentation tasks, object detection tasks, and long-range arena show that it outperforms state-of-the-art self-attention mechanisms with better data efficiency, faster speed, and higher accuracy.

  • Open Access

    A survey of superpixel methods and their applications

    by Chong Wu, Hong Yan

    Superpixels can preserve the structure and reduce the redundancy of the original image. Because of these advantages, superpixel generation or superpixel segmentation is widely used as a pre-processing step in many image processing tasks. Although superpixels can be employed to reduce computational complexity, some challenges, such as the non-Euclidean feature learning problem introduced by superpixels, still exist. This survey provides a comprehensive overview of the state-of-the-art superpixel methods, major challenges, commonly used evaluation metrics, applications of superpixels, and potential future directions for the study of superpixels. We first give a review of the state-of-the-art superpixel methods. Next, we use different evaluation metrics to evaluate the performance of 25 up-to-date superpixel methods on different datasets in different noisy environments. After that, we introduce several up-to-date applications of superpixels. Finally, we give several possible future directions for addressing the challenges of superpixels.


  • Open Access

    Star Topology Convolution for Graph Representation Learning

    by Chong Wu, Zhenan Feng, Jiangbin Zheng, Houwang Zhang, Jiawang Cao, Hong Yan

    We present a novel graph convolutional method called star topology convolution (STC). This method makes graph convolution more similar to conventional convolutional neural networks (CNNs) in Euclidean feature spaces. STC learns subgraphs which have a star topology rather than learning a fixed graph like most spectral methods. Due to the properties of a star topology, STC is graph-scale free (without a fixed graph size constraint). It has fewer parameters in its convolutional filter and is inductive, so it is more flexible and can be applied to large and evolving graphs. The convolutional filter is learnable and localized, similar to CNNs in Euclidean feature spaces, and can share weights across graphs. To test the method, STC was compared with state-of-the-art graph convolutional methods in a supervised learning setting on nine node properties prediction benchmark datasets: Cora, Citeseer, Pubmed, PPI, Arxiv, MAG, ACM, DBLP, and IMDB. The experimental results showed that STC achieved state-of-the-art performance on all these datasets and maintained good robustness. In an essential protein identification task, STC outperformed state-of-the-art essential protein identification methods. An application of using pretrained STC as the embedding for feature extraction of some downstream classification tasks was introduced. The experimental results showed that STC can share weights across different graphs and be used as the embedding to improve the performance of downstream tasks.

  • Open Access

    Star topology convolution for graph representation learning

    by Chong Wu, Zhenan Feng, Jiangbin Zheng, Houwang Zhang, Jiawang Cao, Hong Yan

    We present a novel graph convolutional method called star topology convolution (STC). This method makes graph convolution more similar to conventional convolutional neural networks (CNNs) in Euclidean feature spaces. STC learns subgraphs which have a star topology rather than learning a fixed graph like most spectral methods. Due to the properties of a star topology, STC is graph-scale free (without a fixed graph size constraint). It has fewer parameters in its convolutional filter and is inductive, so it is more flexible and can be applied to large and evolving graphs. The convolutional filter is learnable and localized, similar to CNNs in Euclidean feature spaces, and can share weights across graphs. To test the method, STC was compared with the state-of-the-art graph convolutional methods in a supervised learning setting on nine node properties prediction benchmark datasets: Cora, Citeseer, Pubmed, PPI, Arxiv, MAG, ACM, DBLP, and IMDB. The experimental results showed that STC achieved the state-of-the-art performance on all these datasets and maintained good robustness. In an essential protein identification task, STC outperformed the state-of-the-art essential protein identification methods. An application of using pretrained STC as the embedding for feature extraction of some downstream classification tasks was introduced. The experimental results showed that STC can share weights across different graphs and be used as the embedding to improve the performance of downstream tasks.