DZDongcheng Zhao

Dongcheng Zhao

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Dongcheng Zhao has a total of 37 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Incorporating brain-inspired mechanisms for multimodal learning in artificial intelligence

    by Xiang He, Dongcheng Zhao, Yang Li, Qingqun Kong, Xin Yang, Yi Zeng

    Multimodal learning enhances the perceptual ability of intelligent systems by integrating information across sensory modalities. However, most artificial intelligence approaches still rely on static fusion schemes and do not account for the dynamic nature of multisensory integration observed in the brain. In biological systems, the principle of inverse effectiveness states that weaker unimodal cues contribute more strongly to multisensory integration, whereas strong cues reduce the relative benefit of fusion. Motivated by this mechanism, we investigate the relationship between multimodal outputs and modality-specific information and introduce an inverse effectiveness–driven multimodal fusion (IEMF) strategy. Integrating IEMF into neural networks yields more adaptive fusion behavior, improved accuracy, and substantial gains in computational efficiency. We validate the generality of IEMF across audiovisual perception, vision-language understanding, and trimodal sentiment analysis, demonstrating consistent improvements over state-of-the-art baselines in classification, continual learning, and question answering. The approach also transfers across architectures, including both artificial neural networks and spiking neural networks.


  • Biologically inspired spiking diffusion model with adaptive lateral selection mechanism

    by Linghao Feng, Dongcheng Zhao, Sicheng Shen, Yi Zeng

  • FireFly-T: High-Throughput Sparsity Exploitation for Spiking Transformer Acceleration With Dual-Engine Overlay Architecture

    by Tenglong Li, Jindong Li, Guobin Shen, Dongcheng Zhao, Qian Zhang, Yi Zeng

    Spiking transformers are emerging as a promising architecture that combines the energy efficiency of Spiking Neural Networks (SNNs) with the powerful attention mechanisms of transformers. However, existing hardware accelerators lack support for spiking attention, exhibit limited throughput when exploiting fine-grained sparsity, and struggle with scalable parallelism in sparse computation. To address these challenges, we propose FireFly-T, a dual-engine overlay architecture that integrates a sparse engine for activation sparsity and a binary engine for spiking attention. In the sparse engine, we present a high-throughput sparse decoder that exploits fine-grained sparsity by concurrently extracting multiple non-zero spikes. To complement this, we introduce a scalable load balancing mechanism with weight dispatch and out-of-order execution, eliminating bank conflicts to support scalable multidimensional parallelism. In the binary engine, we leverage the byte-level write capability of SRAMs to efficiently manipulate the 3D dataflows required for spiking attention with minimal resource overhead. We also optimize the core AND-PopCount operation in spiking attention through a LUT6-based implementation, improving timing closure and reducing LUT utilization on Xilinx FPGAs. As an overlay architecture, FireFly-T further incorporates an orchestrator that dynamically manipulates input dataflows with flexible adaptation to diverse network topologies, while ensuring efficient resource utilization and maintaining high throughput. Experimental results demonstrate that our accelerator achieves $1.39\times$ and $2.40\times$ higher energy efficiency, as well as $4.21\times$ and $7.10\times$ greater DSP efficiency, compared to FireFly v2 and the transformer-enabled SpikeTA, respectively. These results highlight its potential as an efficient hardware platform for spiking transformers.


  • Open Access

    Hummingbird+: Advancing FPGA-based LLM Deployment from Research Prototype to Edge Product

    by Jindong Li, Tenglong Li, Guobin Shen, Dongcheng Zhao, Qian Zhang, Yi Zeng

    Field-Programmable Gate Arrays (FPGAs) have been shown to be viable for Large Language Model (LLM) deployment, but they remain less competitive than embedded GPUs and NPUs for final edge products. This is largely because existing FPGA-based LLM accelerator prototypes rely on large, expensive FPGA devices to provide sufficient hardware resources for satisfactory performance, whereas edge products are highly cost-sensitive. In this work, we move beyond pure architectural prototyping to evaluate the feasibility of using low-cost FPGAs as the final implementation medium for LLM deployment. We propose Hummingbird+, which encompasses: (1) a compact embedded FPGA-based LLM accelerator designed to deliver comparable inference performance compared to embedded GPUs and NPUs, and (2) a custom Printed Circuit Board (PCB) built around a Zynq UltraScale XCZU2CG/3EG SoC, equipped with 24GB of memory and an expected Bill of Materials (BOMs) under \150 in mass production. Through extensive FPGA-centric optimizations, we significantly reduce the accelerator's resource consumption, enabling deployment on entry-level FPGAs with exceptional cost efficiency. On this platform, we successfully deploy the GPTQ 4-bit Qwen3-30B-A3B LLM, achieving a decoding speed of over 18 token/s and a prefill speed of over 50 token/s without further model compression. To our knowledge, this is the first demonstration of an FPGA-based edge product serving as a practical and cost-effective final implementation medium for LLM deployment.


  • Time Cell Inspired Temporal Codebook in Spiking Neural Networks for Enhanced Image Generation

    by Linghao Feng, Dongcheng Zhao, Sicheng Shen, Yiting Dong, Guobin Shen, Yi Zeng

    Spiking Neural Networks (SNNs) have garnered significant attention due to their biological plausibility and energy efficiency. However, their generative modeling capabilities remain relatively underexplored, particularly in the context of handling temporal information. In this work, we propose a novel SNN-based Vector Quantized-Variational Autoencoder (VQ-VAE) framework that incorporates a temporal codebook inspired by hippocampal time cells. The temporal codebook enables the model to effectively encode and utilize temporal dependencies, facilitating the generation of high-resolution, temporally consistent images and event-based data. Extensive experiments conducted on diverse datasets, including CelebA-HQ, LSUN Bedroom, LSUN Church, CIFAR-10, and N-MNIST, demonstrate the superior performance of the proposed framework. Our method achieves state-of-the-art results in image quality and temporal consistency, outperforming existing SNN-based generative models. Furthermore, through destructive experiments, we validate the critical role of the temporal codebook in enhancing temporal correlation and improving generative performance. In addition to its superior image generation capabilities, our model exhibits significantly lower energy consumption compared to ANN-based counterparts, leveraging the inherent sparsity of SNNs. These findings highlight the potential of integrating temporal dynamics into the latent space of generative models, paving the way for advancements in neuromorphic computing and biologically inspired machine learning.


  • Open Access

    Position: Re-orienting Spiking Transformer Research from Energy Efficiency to Brain-Inspired Neural Dynamics

    by Dongcheng Zhao, Jinyan Yu, Xiaorui Dong, Zhenyu Yang, Zhiyuan Li, Tengyu Zhang, Xinyu Chen, Tielin Zhang

    Spiking Transformers integrate spiking neural networks with attention-based architectures, offering a promising route toward energy-efficient and event-driven sequence modeling. While recent work has demonstrated their practical feasibility across multiple tasks, current research remains largely driven by energy-efficiencyoriented, transfer-based design principles, in which spiking mechanisms primarily function as implementation-level substitutes for ANN Transformers. In this position paper, we argue that continuing to center progress on energy efficiency fundamentally limits the long-term potential of Spiking Transformers. We identify a set of structural bottlenecks that suppress the expressive role of temporal dynamics, adaptation, and statedependent computation, and show how these bottlenecks increasingly drive spiking models toward ANN-like behavior as scale grows. We contend that future progress requires a deliberate shift in research emphasis, from optimizing efficiency within inherited Transformer templates toward re-centering neural dynamics and temporal structure as first-class modeling principles. Building on this position, we outline a temporally grounded research agenda centered on neuronal dynamics, synaptic plasticity, and circuit-level motifs, and argue that meaningful advances depend on modeling frameworks in which time and adaptation are intrinsic to computation rather than auxiliary implementation artifacts.

  • Open Access

    FireFly-P: FPGA-Accelerated Spiking Neural Network Plasticity for Robust Adaptive Control

    by Tenglong Li, Jindong Li, Guobin Shen, Dongcheng Zhao, Qian Zhang, Yi Zeng

    Spiking Neural Networks (SNNs) offer a biologically plausible learning mechanism through synaptic plasticity, enabling unsupervised adaptation without the computational overhead of backpropagation. To harness this capability for robotics, this paper presents FireFly-P, an FPGA-based hardware accelerator that implements a novel plasticity algorithm for real-time adaptive control. By leveraging on-chip plasticity, our architecture enhances the network's generalization, ensuring robust performance in dynamic and unstructured environments. The hardware design achieves an end-to-end latency of just 8~$μ$s for both inference and plasticity updates, enabling rapid adaptation to unseen scenarios. Implemented on a tiny Cmod A7-35T FPGA, FireFly-P consumes only 0.713~W and $\sim$10K~LUTs, making it ideal for power- and resource-constrained embedded robotic platforms. This work demonstrates that hardware-accelerated SNN plasticity is a viable path toward enabling adaptive, low-latency, and energy-efficient control systems.

  • Open Access

    Safety Instincts: LLMs Learn to Trust Their Internal Compass for Self-Defense

    by Guobin Shen, Dongcheng Zhao, Haibo Tong, Jindong Li, Feifei Zhao, Yi Zeng

    Ensuring Large Language Model (LLM) safety remains challenging due to the absence of universal standards and reliable content validators, making it difficult to obtain effective training signals. We discover that aligned models already possess robust internal safety beliefs: they consistently produce high-confidence refusals to harmful requests while exhibiting high entropy when generating potentially dangerous content. This entropy gap reveals an untapped signal--models intrinsically "know" when to refuse. We introduce Safety Instincts Reinforcement Learning (SIRL), which transforms this internal confidence into a self-generated reward signal, eliminating dependence on external validators or human annotations. SIRL teaches models to trust their safety instincts by reinforcing low-entropy refusal behaviors. Evaluated on Llama and Qwen models, SIRL maintains 89%+ Defense Success Rates (DSRs) against 20+ jailbreak methods, from static prompts to adaptive attacks. Using only 15,000 unlabeled prompts, SIRL surpasses resource-intensive supervised methods while preserving performance on mathematics, coding, and conversation benchmarks. Our work demonstrates that effective alignment can emerge from within, paving the way for more autonomous and robust AI safety mechanisms that scale without extensive human oversight.

  • Open Access

    Brain-Inspired Stepwise Patch Merging for Vision Transformers

    by Yonghao Yu, Dongcheng Zhao, Guobin Shen, Yiting Dong, Yi Zeng

    The hierarchical architecture has become a mainstream design paradigm for Vision Transformers (ViTs), with Patch Merging serving as the pivotal component that transforms a columnar architecture into a hierarchical one. Drawing inspiration from the brain's ability to integrate global and local information for comprehensive visual understanding, we propose Stepwise Patch Merging (SPM), which enhances the subsequent attention mechanism's ability to 'see' better. SPM consists of Multi-Scale Aggregation (MSA) and Guided Local Enhancement (GLE) striking a proper balance between long-range dependency modeling and local feature enhancement. Extensive experiments conducted on benchmark datasets, including ImageNet-1K, COCO, and ADE20K, demonstrate that SPM significantly improves the performance of various models, particularly in dense prediction tasks such as object detection and semantic segmentation. Meanwhile, experiments show that combining SPM with different backbones can further improve performance. The code has been released at https://github.com/Yonghao-Yu/StepwisePatchMerging.

  • Open Access

    FireFly-T: High-Throughput Sparsity Exploitation for Spiking Transformer Acceleration with Dual-Engine Overlay Architecture

    by Tenglong Li, Jindong Li, Guobin Shen, Dongcheng Zhao, Qian Zhang, Yi Zeng

    Spiking transformers are emerging as a promising architecture that combines the energy efficiency of Spiking Neural Networks (SNNs) with the powerful attention mechanisms of transformers. However, existing hardware accelerators lack support for spiking attention, exhibit limited throughput in exploiting fine-grained sparsity, and struggle with scalable parallelism in sparse computation. To address these, we propose FireFly-T, a dual-engine overlay architecture that integrates a sparse engine for activation sparsity and a binary engine for spiking attention. In the sparse engine, we propose a highthroughput sparse decoder that exploits fine-grained sparsity by concurrently extracting multiple non-zero spikes. To complement this, we introduce a scalable load balancing mechanism with weight dispatch and out-of-order execution, eliminating bank conflicts to support scalable multidimensional parallelism. In the binary engine, we leverage the byte-level write capability of SRAMs to efficiently manipulate the 3D dataflows required for spiking attention with minimal resource overhead. We also optimize the core AND-PopCount operation in spiking attention through a LUT6-based implementation, improving timing closure and reducing LUT utilization on Xilinx FPGAs. As an overlay architecture, FireFly-T further incorporates an orchestrator that dynamically manipulates input dataflows with flexible adaptation for diverse network topologies, while ensuring efficient resource utilization and maintaining high throughput. Experimental results demonstrate that our accelerator achieves $1.39\times$ and $2.40\times$ higher energy efficiency, as well as $4.21\times$ and $7.10\times$ greater DSP efficiency, compared to FireFly v2 and the transformer-enabled SpikeTA, respectively. These results highlight its potential as an efficient hardware platform for spiking transformer.