ZWZiheng Wang

Ziheng Wang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Ziheng Wang has a total of 48 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Perturbation-Based Error Tolerance for Large-Scale Networks

    by Ziheng Wang, Pedro Reviriego, Shanshan Liu, Farzad Niknia, Álvaro Alonso, Zhen Gao, Fabrizio Lombardi

  • Perturbation-based error detection and correction (PBEDC) in dependable large-scale machine learning systems

    by Ziheng Wang, Pedro Reviriego, Shanshan Liu, Farzad Niknia, Xiaochen Tang, Zhen Gao, Fabrizio Lombardi


  • A Configurable Floating-Point Fused Multiply-Add Design With Mixed Precision for AI Accelerators

    by Farzad Niknia, Ziheng Wang, Shanshan Liu, Pedro Reviriego, Zhen Gao, Paolo Montuschi, Fabrizio Lombardi

    Hardware accelerators for deep learning in artificial intelligence applications must often meet stringent constraints for accuracy and throughput. In addition to architecture/algorithm improvements, high performance computational techniques such as mixed precision are also required. In this paper, a floating-point (FP) fused multiply-add (FMA) unit supporting mixed/multiple precision is proposed. A wide range of conventional FP formats (such as half and single) as well as emerging formats (including E4M3, E5M2, DLFloat, BFLoat16 and TF32) are supported in the proposed design. In addition to all these formats, the proposed design is flexible in manipulating the exponent and mantissa lengths for 8, 16 and 32-bit FP numbers based on the needs of an application. The proposed FMA can be configured to support either multiple normal FMA operations, or alternatively mixed precision in ASIC. It is fully pipelined and in each cycle, the input bit streams are processed based on the provided configuration, so independent of the previous cycles. For normal FMA operations, the proposed design utilizes sharing of resources to parallelize multiple operations based on the available hardware and required precision. For mixed precision the FMA accumulates the lower precision dot products into higher precision to avoid overflow/underflow. It improves computational accuracy by adding all possible dot products at the same time while decreasing the number of rounding operations to prevent rounding errors. An innovative method to accumulate the dot products and the aligned addend is also proposed. By considering tradeoffs between reusing the available hardware and removing unnecessary complex units, a more efficient and flexible design is attained in terms of hardware metrics and supported different precision computation compared to other designs found in the technical literature. Extensive simulation results for comparative analysis are provided.


  • Energy-Efficient Stochastic Computing (SC) Neural Networks for Internet of Things Devices With Layer-Wise Adjustable Sequence Length (ASL)

    by Ziheng Wang, Pedro Reviriego, Farzad Niknia, Zhen Gao, Javier Conde, Shanshan Liu, Fabrizio Lombardi

    Stochastic computing (SC) has emerged as an efficient low-power alternative for deploying neural networks (NNs) in resource-limited scenarios, such as the Internet of Things (IoT). By encoding values as serial bitstreams, SC significantly reduces energy dissipation compared to conventional floating-point (FP) designs; however, further improvement of layer-wise mixed-precision implementation for SC remains unexplored. This article introduces Adjustable Sequence Length (ASL), a novel scheme that applies mixed-precision concepts specifically to SC NNs. By introducing an operator-norm-based theoretical model, this article shows that truncation noise can cumulatively propagate through the layers by the estimated amplification factors. An extended sensitivity analysis is presented, using random forest (RF) regression to evaluate multilayer truncation effects and validate the alignment of theoretical predictions with practical network behaviors. To accommodate different application scenarios, this article proposes two truncation strategies (coarse-grained and fine-grained), which apply diverse sequence length configurations at each layer. Evaluations on a pipelined SC MLP synthesized at 32nm demonstrate that ASL can reduce energy and latency overheads by up to over 60% with negligible accuracy loss. It confirms the feasibility of the ASL scheme for IoT applications and highlights the distinct advantages of mixed-precision truncation in SC designs.


  • Adaptive Separately Constrained Triplet Loss (A-SCTL) for High-Performance Triplet Networks

    by Ziheng Wang, Farzad Niknia, Shanshan Liu, Honglan Jiang, Siting Liu, Pedro Reviriego, Jun Zhou, Fabrizio Lombardi

    Triplet Networks (TNs) consist of three subchannels and are widely utilized in machine learning applications. The efficacy of TNs is highly dependent on the loss function employed during training. This paper proposes a novel loss function for TNs, referred to as the Adaptive Separately Constrained Triplet Loss (A-SCTL). The unique feature of A-SCTL is the separation of intra-class and inter-class constraints, strictly adhering to the objective of similarity-measuring networks. Its adaptive strategy leverages the dynamics between inter-class and intra-class terms to achieve a balanced convergence; without manually adjusting hyperparameters, it enhances flexibility and facilitates adaptation across various applications. Moreover, A-SCTL mitigates possible false solutions and offers insights into network behavior through the dependency of the two constraint terms. Performance metrics of the loss functions are evaluated in deep metric learning classification and face recognition tasks. Simulations illustrate the evolution of the two loss terms and the adaptive hyperparameter across training epochs; the results demonstrate that TNs utilizing A-SCTL outperform other existing loss functions in accuracy. Additionally, this paper details the hardware implementation of A-SCTL and evaluates its associated overhead. Results show that compared to other losses, the additional hardware overhead required for A-SCTL is negligible (0.008% energy per operation) when considering the entire TN system.

  • Fault Tolerance in Triplet Network Training: Analysis, Evaluation and Protection Methods

    by Ziheng Wang, Farzad Niknia, Shanshan Liu, Pedro Reviriego, Ahmed Louri, Fabrizio Lombardi

    This paper investigates the tolerance of Triplet Networks (TNs) with a focus on faults in the training process. For compatibility with the existing literature. So-called stuck-at faults of a functional nature are considered for the operation of the neurons and activation function. While TNs are shown to be generally robust against such faults in the anchor and positive subnetworks, the presented analysis reveals a significant vulnerability in the negative subnetwork, in which stuck-at faults can lead to false convergence and training failures. An in-depth treatment is provided to show the incorrect convergence of training in the presence of stuck-at faults, highlighting the behavior of the network with faulty neurons. Extensive simulations are presented to evaluate the impact of these faults and propose two innovative fault-tolerant methods: the regularization of the anchor outputs and the modified margin. Simulation shows that false convergence can be very efficiently avoided by utilizing the proposed techniques, and thus the overall accuracy loss of the TN is negligible. These findings contribute to the understanding of fault tolerance in emerging neural networks such as TNs and offer practical solutions for enhancing their robustness against faults.

  • Are Emerging Machine Learning Models Dependable at the Nanoscales?

    by Shanshan Liu, Ziheng Wang, Zhen Gao, Pedro Reviriego, Farzad Niknia, Xiaochen Tang, Jun Zhou, Fabrizio Lombardi

    Machine learning models by emerging neural networks (NNs) are being developed to solve advanced and complex tasks; for example, multi-branch NNs (such as Siamese and Triplet networks) have been proposed to perform similarity checking tasks, and large-scale models (such as Bidirectional Encoder Representations from Transformers (BERT) and Stable Diffusion) have been developed for nature language and image processing. These emerging NNs that often have a large model size, are potentially resilient to small errors during computation; however, when considering random errors originating from the underlying hardware implementation (such as soft errors on nanoscale circuits induced by radiation or voltage scaling), their complexity poses an obvious question: can these models still operate dependably when implemented by nano-scaled CMOS technologies? This article explores this concern by analyzing and evaluating the dependability of emerging NNs in the presence of hardware induced errors; the results are expected to provide insights for relevant error protection techniques when applying these models in safety-critical applications.


  • ASIC Design of Nanoscale Artificial Neural Networks for Inference/Training by Floating-Point Arithmetic

    by Farzad Niknia, Ziheng Wang, Shanshan Liu, Pedro Reviriego, Ahmed Louri, Fabrizio Lombardi

    Inference and on-chip training of Artificial Neural Networks (ANNs) are challenging computational processes for large datasets; hardware implementations are needed to accelerate this computation, while meeting metrics such as operating frequency, power dissipation and accuracy. In this paper, a high-performance ASIC-based design is proposed to implement both forward and backward propagations of multilayer perceptrons (MLPs) at the nanoscales. To attain a higher accuracy, floating-point arithmetic units for a multiply-andaccumulate (MAC) array are employed in the proposed design; moreover, a hybrid implementation scheme is utilized to achieve flexibility (for networks of different size) and comprehensively low hardware overhead. The proposed design is fully pipelined, and its performance is independent of network size, except for the number of cycles and latency. The efficiency of the proposed nanoscale MLP-based design for inference (as taking place over multiple steps) and training (due to the complex processing in backward propagation by eliminating many redundant calculations) is analyzed. Moreover, the impact of different floating-point precision formats on the final accuracy and hardware metrics under the same design constraints is studied. A comparative evaluation of the proposed MLP design for different datasets and floating-point precision formats is provided. Results show that compared to current schemes found in the technical literatures, the proposed design has the best operating frequency and accuracy with still good latency and energy dissipation.


  • Concurrent Classifier Error Detection (CCED) in Large Scale Machine Learning Systems

    by Pedro Reviriego, Ziheng Wang, Álvaro Alonso, Zhen Gao, Farzad Niknia, Shanshan Liu, Fabrizio Lombardi

    The complexity of machine learning (ML) systems increases each year. As these systems are widely utilized, ensuring their reliable operation is becoming a design requirement. Traditional error detection mechanisms introduce circuit or time redundancy that significantly impacts system performance. An alternative is the use of concurrent error detection (CED) schemes that operate in parallel with the system and exploit their properties to detect errors. CED is attractive for large ML systems because it can potentially reduce the cost of error detection. In this article, we introduce concurrent classifier error detection (CCED), a scheme to implement CED in ML systems using a concurrent ML classifier to detect errors. CCED identifies a set of check signals in the main ML system and feed them to the concurrent ML classifier that is trained to detect errors. The proposed CCED scheme has been implemented and evaluated on two widely used large-scale ML models: Contrastive language-image pretraining (CLIP) used for image classification and bidirectional encoder representations from transformers (BERT) used for natural language applications. The results show that more than 95% of the errors are detected when using a simple Random Forest classifier that is orders of magnitude simpler than CLIP or BERT.


  • Emerging Machine Learning Using Siamese and Triplet Neural Networks

    by Ziheng Wang, Farzad Niknia, Shanshan Liu, Pedro Reviriego, Fabrizio Lombardi