Ziheng Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Ziheng Wang has a total of 48 co-authors. Here are the top co-authors by number of shared publications:
- Fabrizio Lombardi
Northeastern University
15 shared publications - Pedro Reviriego
Universidad Politécnica de Madrid
15 shared publications - Shanshan Liu
University of Electronic Science and Technology of China
15 shared publications - Farzad Niknia
Northeastern University
14 shared publications - Zhen Gao
School of Electrical and Information Engineering, Tianjin University, Tianjin, China
7 shared publications - Xiaochen Tang
University of Electronic Science and Technology of China
4 shared publications
Latest Publications
- 23 May 2026
- Published by Springer Nature
- p. 229-254
Perturbation-Based Error Tolerance for Large-Scale Networks
- 23 May 2026
- Published by Springer Nature
- p. 229-254
Perturbation-Based Error Tolerance for Large-Scale Networks
- 13 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Artificial Intelligence
- Vol. 2 (3) , 248-261
A Configurable Floating-Point Fused Multiply-Add Design With Mixed Precision for AI Accelerators
Hardware accelerators for deep learning in artificial intelligence applications must often meet stringent constraints for accuracy and throughput. In addition to architecture/algorithm improvements, high performance computational techniques such as mixed precision are also required. In this paper, a floating-point (FP) fused multiply-add (FMA) unit supporting mixed/multiple precision is proposed. A wide range of conventional FP formats (such as half and single) as well as emerging formats (including E4M3, E5M2, DLFloat, BFLoat16 and TF32) are supported in the proposed design. In addition to all these formats, the proposed design is flexible in manipulating the exponent and mantissa lengths for 8, 16 and 32-bit FP numbers based on the needs of an application. The proposed FMA can be configured to support either multiple normal FMA operations, or alternatively mixed precision in ASIC. It is fully pipelined and in each cycle, the input bit streams are processed based on the provided configuration, so independent of the previous cycles. For normal FMA operations, the proposed design utilizes sharing of resources to parallelize multiple operations based on the available hardware and required precision. For mixed precision the FMA accumulates the lower precision dot products into higher precision to avoid overflow/underflow. It improves computational accuracy by adding all possible dot products at the same time while decreasing the number of rounding operations to prevent rounding errors. An innovative method to accumulate the dot products and the aligned addend is also proposed. By considering tradeoffs between reusing the available hardware and removing unnecessary complex units, a more efficient and flexible design is attained in terms of hardware metrics and supported different precision computation compared to other designs found in the technical literature. Extensive simulation results for comparative analysis are provided.
- 13 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Artificial Intelligence
- Vol. 2 (3) , 248-261
A Configurable Floating-Point Fused Multiply-Add Design With Mixed Precision for AI Accelerators
Hardware accelerators for deep learning in artificial intelligence applications must often meet stringent constraints for accuracy and throughput. In addition to architecture/algorithm improvements, high performance computational techniques such as mixed precision are also required. In this paper, a floating-point (FP) fused multiply-add (FMA) unit supporting mixed/multiple precision is proposed. A wide range of conventional FP formats (such as half and single) as well as emerging formats (including E4M3, E5M2, DLFloat, BFLoat16 and TF32) are supported in the proposed design. In addition to all these formats, the proposed design is flexible in manipulating the exponent and mantissa lengths for 8, 16 and 32-bit FP numbers based on the needs of an application. The proposed FMA can be configured to support either multiple normal FMA operations, or alternatively mixed precision in ASIC. It is fully pipelined and in each cycle, the input bit streams are processed based on the provided configuration, so independent of the previous cycles. For normal FMA operations, the proposed design utilizes sharing of resources to parallelize multiple operations based on the available hardware and required precision. For mixed precision the FMA accumulates the lower precision dot products into higher precision to avoid overflow/underflow. It improves computational accuracy by adding all possible dot products at the same time while decreasing the number of rounding operations to prevent rounding errors. An innovative method to accumulate the dot products and the aligned addend is also proposed. By considering tradeoffs between reusing the available hardware and removing unnecessary complex units, a more efficient and flexible design is attained in terms of hardware metrics and supported different precision computation compared to other designs found in the technical literature. Extensive simulation results for comparative analysis are provided.
- 24 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 12 (14) , 26955-26967
Energy-Efficient Stochastic Computing (SC) Neural Networks for Internet of Things Devices With Layer-Wise Adjustable Sequence Length (ASL)
Stochastic computing (SC) has emerged as an efficient low-power alternative for deploying neural networks (NNs) in resource-limited scenarios, such as the Internet of Things (IoT). By encoding values as serial bitstreams, SC significantly reduces energy dissipation compared to conventional floating-point (FP) designs; however, further improvement of layer-wise mixed-precision implementation for SC remains unexplored. This article introduces Adjustable Sequence Length (ASL), a novel scheme that applies mixed-precision concepts specifically to SC NNs. By introducing an operator-norm-based theoretical model, this article shows that truncation noise can cumulatively propagate through the layers by the estimated amplification factors. An extended sensitivity analysis is presented, using random forest (RF) regression to evaluate multilayer truncation effects and validate the alignment of theoretical predictions with practical network behaviors. To accommodate different application scenarios, this article proposes two truncation strategies (coarse-grained and fine-grained), which apply diverse sequence length configurations at each layer. Evaluations on a pipelined SC MLP synthesized at 32nm demonstrate that ASL can reduce energy and latency overheads by up to over 60% with negligible accuracy loss. It confirms the feasibility of the ASL scheme for IoT applications and highlights the distinct advantages of mixed-precision truncation in SC designs.
- 24 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 12 (14) , 26955-26967
Energy-Efficient Stochastic Computing (SC) Neural Networks for Internet of Things Devices With Layer-Wise Adjustable Sequence Length (ASL)
Stochastic computing (SC) has emerged as an efficient low-power alternative for deploying neural networks (NNs) in resource-limited scenarios, such as the Internet of Things (IoT). By encoding values as serial bitstreams, SC significantly reduces energy dissipation compared to conventional floating-point (FP) designs; however, further improvement of layer-wise mixed-precision implementation for SC remains unexplored. This article introduces Adjustable Sequence Length (ASL), a novel scheme that applies mixed-precision concepts specifically to SC NNs. By introducing an operator-norm-based theoretical model, this article shows that truncation noise can cumulatively propagate through the layers by the estimated amplification factors. An extended sensitivity analysis is presented, using random forest (RF) regression to evaluate multilayer truncation effects and validate the alignment of theoretical predictions with practical network behaviors. To accommodate different application scenarios, this article proposes two truncation strategies (coarse-grained and fine-grained), which apply diverse sequence length configurations at each layer. Evaluations on a pipelined SC MLP synthesized at 32nm demonstrate that ASL can reduce energy and latency overheads by up to over 60% with negligible accuracy loss. It confirms the feasibility of the ASL scheme for IoT applications and highlights the distinct advantages of mixed-precision truncation in SC designs.
- 18 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Nanotechnology
- Vol. 24 (1536125X) , 157-165
Adaptive Separately Constrained Triplet Loss (A-SCTL) for High-Performance Triplet Networks
Triplet Networks (TNs) consist of three subchannels and are widely utilized in machine learning applications. The efficacy of TNs is highly dependent on the loss function employed during training. This paper proposes a novel loss function for TNs, referred to as the Adaptive Separately Constrained Triplet Loss (A-SCTL). The unique feature of A-SCTL is the separation of intra-class and inter-class constraints, strictly adhering to the objective of similarity-measuring networks. Its adaptive strategy leverages the dynamics between inter-class and intra-class terms to achieve a balanced convergence; without manually adjusting hyperparameters, it enhances flexibility and facilitates adaptation across various applications. Moreover, A-SCTL mitigates possible false solutions and offers insights into network behavior through the dependency of the two constraint terms. Performance metrics of the loss functions are evaluated in deep metric learning classification and face recognition tasks. Simulations illustrate the evolution of the two loss terms and the adaptive hyperparameter across training epochs; the results demonstrate that TNs utilizing A-SCTL outperform other existing loss functions in accuracy. Additionally, this paper details the hardware implementation of A-SCTL and evaluates its associated overhead. Results show that compared to other losses, the additional hardware overhead required for A-SCTL is negligible (0.008% energy per operation) when considering the entire TN system.
- 18 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Nanotechnology
- Vol. 24 (1536125X) , 157-165
Adaptive Separately Constrained Triplet Loss (A-SCTL) for High-Performance Triplet Networks
Triplet Networks (TNs) consist of three subchannels and are widely utilized in machine learning applications. The efficacy of TNs is highly dependent on the loss function employed during training. This paper proposes a novel loss function for TNs, referred to as the Adaptive Separately Constrained Triplet Loss (A-SCTL). The unique feature of A-SCTL is the separation of intra-class and inter-class constraints, strictly adhering to the objective of similarity-measuring networks. Its adaptive strategy leverages the dynamics between inter-class and intra-class terms to achieve a balanced convergence; without manually adjusting hyperparameters, it enhances flexibility and facilitates adaptation across various applications. Moreover, A-SCTL mitigates possible false solutions and offers insights into network behavior through the dependency of the two constraint terms. Performance metrics of the loss functions are evaluated in deep metric learning classification and face recognition tasks. Simulations illustrate the evolution of the two loss terms and the adaptive hyperparameter across training epochs; the results demonstrate that TNs utilizing A-SCTL outperform other existing loss functions in accuracy. Additionally, this paper details the hardware implementation of A-SCTL and evaluates its associated overhead. Results show that compared to other losses, the additional hardware overhead required for A-SCTL is negligible (0.008% energy per operation) when considering the entire TN system.
- 22 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computing
- Vol. 13 (3) , 714-723
Fault Tolerance in Triplet Network Training: Analysis, Evaluation and Protection Methods
This paper investigates the tolerance of Triplet Networks (TNs) with a focus on faults in the training process. For compatibility with the existing literature. So-called stuck-at faults of a functional nature are considered for the operation of the neurons and activation function. While TNs are shown to be generally robust against such faults in the anchor and positive subnetworks, the presented analysis reveals a significant vulnerability in the negative subnetwork, in which stuck-at faults can lead to false convergence and training failures. An in-depth treatment is provided to show the incorrect convergence of training in the presence of stuck-at faults, highlighting the behavior of the network with faulty neurons. Extensive simulations are presented to evaluate the impact of these faults and propose two innovative fault-tolerant methods: the regularization of the anchor outputs and the modified margin. Simulation shows that false convergence can be very efficiently avoided by utilizing the proposed techniques, and thus the overall accuracy loss of the TN is negligible. These findings contribute to the understanding of fault tolerance in emerging neural networks such as TNs and offer practical solutions for enhancing their robustness against faults.
- 22 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computing
- Vol. 13 (3) , 714-723
Fault Tolerance in Triplet Network Training: Analysis, Evaluation and Protection Methods
This paper investigates the tolerance of Triplet Networks (TNs) with a focus on faults in the training process. For compatibility with the existing literature. So-called stuck-at faults of a functional nature are considered for the operation of the neurons and activation function. While TNs are shown to be generally robust against such faults in the anchor and positive subnetworks, the presented analysis reveals a significant vulnerability in the negative subnetwork, in which stuck-at faults can lead to false convergence and training failures. An in-depth treatment is provided to show the incorrect convergence of training in the presence of stuck-at faults, highlighting the behavior of the network with faulty neurons. Extensive simulations are presented to evaluate the impact of these faults and propose two innovative fault-tolerant methods: the regularization of the anchor outputs and the modified margin. Simulation shows that false convergence can be very efficiently avoided by utilizing the proposed techniques, and thus the overall accuracy loss of the TN is negligible. These findings contribute to the understanding of fault tolerance in emerging neural networks such as TNs and offer practical solutions for enhancing their robustness against faults.
- 21 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Nanotechnology Magazine
- Vol. 18 (6) , 33-42
Are Emerging Machine Learning Models Dependable at the Nanoscales?
Machine learning models by emerging neural networks (NNs) are being developed to solve advanced and complex tasks; for example, multi-branch NNs (such as Siamese and Triplet networks) have been proposed to perform similarity checking tasks, and large-scale models (such as Bidirectional Encoder Representations from Transformers (BERT) and Stable Diffusion) have been developed for nature language and image processing. These emerging NNs that often have a large model size, are potentially resilient to small errors during computation; however, when considering random errors originating from the underlying hardware implementation (such as soft errors on nanoscale circuits induced by radiation or voltage scaling), their complexity poses an obvious question: can these models still operate dependably when implemented by nano-scaled CMOS technologies? This article explores this concern by analyzing and evaluating the dependability of emerging NNs in the presence of hardware induced errors; the results are expected to provide insights for relevant error protection techniques when applying these models in safety-critical applications.
- 21 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Nanotechnology Magazine
- Vol. 18 (6) , 33-42
Are Emerging Machine Learning Models Dependable at the Nanoscales?
Machine learning models by emerging neural networks (NNs) are being developed to solve advanced and complex tasks; for example, multi-branch NNs (such as Siamese and Triplet networks) have been proposed to perform similarity checking tasks, and large-scale models (such as Bidirectional Encoder Representations from Transformers (BERT) and Stable Diffusion) have been developed for nature language and image processing. These emerging NNs that often have a large model size, are potentially resilient to small errors during computation; however, when considering random errors originating from the underlying hardware implementation (such as soft errors on nanoscale circuits induced by radiation or voltage scaling), their complexity poses an obvious question: can these models still operate dependably when implemented by nano-scaled CMOS technologies? This article explores this concern by analyzing and evaluating the dependability of emerging NNs in the presence of hardware induced errors; the results are expected to provide insights for relevant error protection techniques when applying these models in safety-critical applications.
- 20 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Nanotechnology
- Vol. 23 (1536125X) , 208-216
ASIC Design of Nanoscale Artificial Neural Networks for Inference/Training by Floating-Point Arithmetic
Inference and on-chip training of Artificial Neural Networks (ANNs) are challenging computational processes for large datasets; hardware implementations are needed to accelerate this computation, while meeting metrics such as operating frequency, power dissipation and accuracy. In this paper, a high-performance ASIC-based design is proposed to implement both forward and backward propagations of multilayer perceptrons (MLPs) at the nanoscales. To attain a higher accuracy, floating-point arithmetic units for a multiply-andaccumulate (MAC) array are employed in the proposed design; moreover, a hybrid implementation scheme is utilized to achieve flexibility (for networks of different size) and comprehensively low hardware overhead. The proposed design is fully pipelined, and its performance is independent of network size, except for the number of cycles and latency. The efficiency of the proposed nanoscale MLP-based design for inference (as taking place over multiple steps) and training (due to the complex processing in backward propagation by eliminating many redundant calculations) is analyzed. Moreover, the impact of different floating-point precision formats on the final accuracy and hardware metrics under the same design constraints is studied. A comparative evaluation of the proposed MLP design for different datasets and floating-point precision formats is provided. Results show that compared to current schemes found in the technical literatures, the proposed design has the best operating frequency and accuracy with still good latency and energy dissipation.
- 20 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Nanotechnology
- Vol. 23 (1536125X) , 208-216
ASIC Design of Nanoscale Artificial Neural Networks for Inference/Training by Floating-Point Arithmetic
Inference and on-chip training of Artificial Neural Networks (ANNs) are challenging computational processes for large datasets; hardware implementations are needed to accelerate this computation, while meeting metrics such as operating frequency, power dissipation and accuracy. In this paper, a high-performance ASIC-based design is proposed to implement both forward and backward propagations of multilayer perceptrons (MLPs) at the nanoscales. To attain a higher accuracy, floating-point arithmetic units for a multiply-andaccumulate (MAC) array are employed in the proposed design; moreover, a hybrid implementation scheme is utilized to achieve flexibility (for networks of different size) and comprehensively low hardware overhead. The proposed design is fully pipelined, and its performance is independent of network size, except for the number of cycles and latency. The efficiency of the proposed nanoscale MLP-based design for inference (as taking place over multiple steps) and training (due to the complex processing in backward propagation by eliminating many redundant calculations) is analyzed. Moreover, the impact of different floating-point precision formats on the final accuracy and hardware metrics under the same design constraints is studied. A comparative evaluation of the proposed MLP design for different datasets and floating-point precision formats is provided. Results show that compared to current schemes found in the technical literatures, the proposed design has the best operating frequency and accuracy with still good latency and energy dissipation.
- 31 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Reliability
- Vol. 73 (4) , 1782-1791
Concurrent Classifier Error Detection (CCED) in Large Scale Machine Learning Systems
The complexity of machine learning (ML) systems increases each year. As these systems are widely utilized, ensuring their reliable operation is becoming a design requirement. Traditional error detection mechanisms introduce circuit or time redundancy that significantly impacts system performance. An alternative is the use of concurrent error detection (CED) schemes that operate in parallel with the system and exploit their properties to detect errors. CED is attractive for large ML systems because it can potentially reduce the cost of error detection. In this article, we introduce concurrent classifier error detection (CCED), a scheme to implement CED in ML systems using a concurrent ML classifier to detect errors. CCED identifies a set of check signals in the main ML system and feed them to the concurrent ML classifier that is trained to detect errors. The proposed CCED scheme has been implemented and evaluated on two widely used large-scale ML models: Contrastive language-image pretraining (CLIP) used for image classification and bidirectional encoder representations from transformers (BERT) used for natural language applications. The results show that more than 95% of the errors are detected when using a simple Random Forest classifier that is orders of magnitude simpler than CLIP or BERT.
- 31 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Reliability
- Vol. 73 (4) , 1782-1791
Concurrent Classifier Error Detection (CCED) in Large Scale Machine Learning Systems
The complexity of machine learning (ML) systems increases each year. As these systems are widely utilized, ensuring their reliable operation is becoming a design requirement. Traditional error detection mechanisms introduce circuit or time redundancy that significantly impacts system performance. An alternative is the use of concurrent error detection (CED) schemes that operate in parallel with the system and exploit their properties to detect errors. CED is attractive for large ML systems because it can potentially reduce the cost of error detection. In this article, we introduce concurrent classifier error detection (CCED), a scheme to implement CED in ML systems using a concurrent ML classifier to detect errors. CCED identifies a set of check signals in the main ML system and feed them to the concurrent ML classifier that is trained to detect errors. The proposed CCED scheme has been implemented and evaluated on two widely used large-scale ML models: Contrastive language-image pretraining (CLIP) used for image classification and bidirectional encoder representations from transformers (BERT) used for natural language applications. The results show that more than 95% of the errors are detected when using a simple Random Forest classifier that is orders of magnitude simpler than CLIP or BERT.
- 14 January 2024
- Published by Springer Nature
- p. 115-141
Emerging Machine Learning Using Siamese and Triplet Neural Networks
- 14 January 2024
- Published by Springer Nature
- p. 115-141
Emerging Machine Learning Using Siamese and Triplet Neural Networks