MDMarco Donato

Marco Donato

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Marco Donato has a total of 22 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Heterogeneous Memory Integration and Optimization for Energy-Efficient Multi-Task NLP Edge Inference

    by Zirui Fu, Aleksandre Avaliani, Marco Donato

    Current solutions for deploying natural language processing (NLP) tasks on mobile hardware combine specialized hardware architectures and aggressive model compression to enhance energy, latency, and area (ELA) efficiency. Nonetheless, there are still challenges in addressing multi-task inference (MTI) scenarios, wherein fine-tuned variants of model parameters need to be retrieved from main memory for each individual task, resulting in prohibitive data movement costs. This paper explores the co-optimization of heterogeneous scratchpad memories and NLP model architectures for maximal inter-task parameter reuse. The DNN model performance and its resilience to data compression methods are evaluated across a range of tasks using the GLUE benchmark. By storing the model parameters into the heterogeneous memory architecture, we demonstrate notable improvements in ELA utilization.


  • A 16-nm SoC for Noise-Robust Speech and NLP Edge AI Inference With Bayesian Sound Source Separation and Attention-Based DNNs

    by Thierry Tambe, En-Yu Yang, Glenn G. Ko, Yuji Chai, Coleman Hooper, Marco Donato, Paul N. Whatmough, Alexander M. Rush, David Brooks, Gu-Yeon Wei

    The proliferation of personal artificial intelligence (AI) -assistant technologies with speech-based conversational AI interfaces is driving the exponential growth in the consumer Internet of Things (IoT) market. As these technologies are being applied to keyword spotting (KWS), automatic speech recognition (ASR), natural language processing (NLP), and text-to-speech (TTS) applications, it is of paramount importance that they provide uncompromising performance for context learning in long sequences, which is a key benefit of the attention mechanism, and that they work seamlessly in polyphonic environments. In this work, we present a 25-mm $^2$ system-on-chip (SoC) in 16-nm FinFET technology, codenamed SM6, which executes end-to-end speech-enhancing attention-based ASR and NLP workloads. The SoC includes: 1) FlexASR, a highly reconfigurable NLP inference processor optimized for whole-model acceleration of bidirectional attention-based sequence-to-sequence (seq2seq) deep neural networks (DNNs); 2) a Markov random field source separation engine (MSSE), a probabilistic graphical model accelerator for unsupervised inference via Gibbs sampling, used for sound source separation; 3) a dual-core Arm Cortex A53 CPU cluster, which provides on-demand single Instruction/multiple data (SIMD) fast fourier transform (FFT) processing and performs various application logic (e.g., expectation–maximization (EM) algorithm and 8-bit floating-point (FP8) quantization); and 4) an always-on M0 subsystem for audio detection and power management. Measurement results demonstrate the efficiency ranges of 2.6–7.8 TFLOPs/W and 4.33–17.6 Gsamples/s/W for FlexASR and MSSE, respectively; MSSE denoising performance allowing 6 $\times$ smaller ASR model to be stored on-chip with negligible accuracy loss; and 2.24-mJ energy consumption while achieving real-time throughput, end-to-end, and per-frame ASR latencies of 18 ms.


  • SMIV: A 16-nm 25-mm² SoC for IoT With Arm Cortex-A53, eFPGA, and Coherent Accelerators

    by Sae Kyu Lee, Paul N. Whatmough, Marco Donato, Glenn G. Ko, David Brooks, Gu-Yeon Wei

    Emerging Internet of Things (IoT) devices necessitate system-on-chips (SoCs) that can scale from ultralow power always-on (AON) operation, all the way up to less frequent high-performance tasks at high energy efficiency. Specialized accelerators are essential to help meet these needs at both ends of the scale, but maintaining workload flexibility remains an important goal. This article presents a 25-mm 2 SoC in 16-nm FinFET technology which demonstrates targeted, flexible acceleration of key compute-intensive kernels spanning machine learning (ML), DSP, and cryptography. The SMIV SoC includes a dedicated AON sub-system, a dual-core Arm Cortex-A53 CPU cluster, an SoC-attached embedded field-programmable gate array (eFPGA) array, and a quad-core cache-coherent accelerator (CCA) cluster. Measurement results demonstrate: 1) 1236 $\times $ power envelope, from 1.1 mW (only AON cluster), up to 1.36 W (whole SoC at maximum throughput); 2) 5.5–28.9 $\times $ energy efficiency gain from offloading compute kernels from A53 to eFPGA; 3) 2.94 $\times $ latency improvement using coherent memory access (CCA cluster); and 4) 55 $\times $ MobileNetV1 energy per inference improvement on CCA compared to the CPU baseline. The overall flexibility-efficiency range on SMIV spans measured energy efficiencies of 1 $\times $ (dual-core A53), 3.1 $\times $ (A53 with SIMD), 16.5 $\times $ (eFPGA), 54.9 $\times $ (CCA), and 256 $\times $ (AON) at a peak efficiency of 4.8 TOPS/W.


  • Open Access

    CHIPKIT: An agile, reusable open-source framework for rapid test chip development

    by Paul Whatmough, Marco Donato, Glenn Ko, Sae-Kyu Lee, David Brooks, Gu-Yeon Wei

    The current trend for domain-specific architectures (DSAs) has led to renewed interest in research test chips to demonstrate new specialized hardware. Tape-outs also offer huge pedagogical value garnered from real hands-on exposure to the whole system stack. However, successful tape-outs demand hard-earned experience, and the design process is time consuming and fraught with challenges. Therefore, custom chips have remained the preserve of a small number of research groups, typically focused on circuit design research. This paper describes the CHIPKIT framework. We describe a reusable SoC subsystem which provides basic IO, an on-chip programmable host, memory and peripherals. This subsystem can be readily extended with new IP blocks to generate custom test chips. We also present an agile RTL development flow, including a code generation tool calledVGEN. Finally, we outline best practices for full-chip validation across the entire design cycle.

  • Fundamental Thermal Limits on Data Retention in Low-Voltage CMOS Latches and SRAM

    by Elahe Rezaei, Marco Donato, William R. Patterson, Alexander Zaslavsky, R. Iris Bahar

    Ultra-low-power systems with substantial computing capacity require latches and SRAMs to operate at extremely low supply voltages. However, with aggressive technology scaling, reliability becomes a major challenge due to unavoidable process variations and the presence of multiple noise sources, including intrinsic thermal noise. This paper provides a quantitative measure of reliability by calculating the probability distribution function (PDF) of errors induced by thermal noise in latches and SRAMs operating in subthreshold conditions. Implemented in a novel simulation tool for thermal-noise analysis of CMOS circuits (), our algorithm uses a stochastic differential equation circuit model that preserves the proper Poissonian statistics for thermal-noise-driven current fluctuations in MOSFETs. Our probabilistic error model can handle error rate analysis for arrays of latches or full SRAMs on time scales from seconds to years without excessive computational overhead. We demonstrate that the time-to-error (TTE) statistics of subthreshold SRAMs obey log-normal distributions that depend on parameters such as node and device capacitance, device threshold variations and operating conditions of supply voltage and temperature. This makes it possible to quantitatively evaluate the asymptotic behavior of extremely rare error events that are inaccessible to standard SPICE-based simulations.


  • CHIPKIT: An Agile, Reusable Open-Source Framework for Rapid Test Chip Development

    by Paul N. Whatmough, Marco Donato, Glenn G. Ko, Sae Kyu Lee, David Brooks, Gu-Yeon Wei

    The current trend for domain-specific architectures (DSAs) has led to renewed interest in research test chips to demonstrate new specialized hardware. Tape-outs also offer huge pedagogical value garnered from real hands-on exposure to the whole system stack. However, success with tape-outs requires hard-earned experience, and the design process is time consuming and fraught with challenges. Therefore, custom chips have remained the preserve of a small number of research groups, typically focused on circuit design research. This paper describes the CHIPKIT framework: a reusable SoC subsystem which provides basic IO, an on-chip programmable host, off-chip hosting, memory and peripherals. This subsystem can be readily extended with new IP blocks to generate custom test chips. Central to CHIPKIT, is an agile RTL development flow, including a code generation tool called VGEN. Finally, we discuss best practices for full-chip validation across the entire design cycle.


  • MEMTI: Optimizing On-Chip Nonvolatile Storage for Visual Multitask Inference at the Edge

    by Marco Donato, Lillian Pentecost, David Brooks, Gu-Yeon Wei

    The combination of specialized hardware and embedded non-volatile memories (eNVM) holds promise for energy-efficient DNN inference at the edge. However, integrating DNN hardware accelerators with eNVMs still presents several challenges. Multi-level programming is desirable for achieving maximal storage density on chip, but the stochastic nature of eNVM writes makes them prone to errors and further increases the write energy and latency. We present MEMTI, a memory architecture that leverages a multi-task learning technique for maximal reuse of DNN parameters across multiple visual tasks. We show that by retraining and updating only 10% of all DNN parameters, we can achieve efficient model adaptation across a variety of visual inference tasks. The system performance is evaluated by integrating the memory with the open-source NVIDIA Deep Learning Architecture (NVDLA).


  • Open Access

    A Sub-Threshold Noise Transient Simulator Based on Integrated Random Telegraph and Thermal Noise Modeling

    by Marco Donato, R. Iris Bahar, William R. Patterson, Alexander Zaslavsky

    Near-threshold and sub-threshold voltage designs have been identified as possible solutions to overcome the limitations introduced by energy consumption in modern very large scale integration circuits. However, as we approach sub-10 nm transistor technology, aggressive voltage, and gate length scaling will reduce the reliability of logic circuits due to the increasing impact of noise and variability effects. Therefore, designers need new tools to simulate logic circuits in the presence of noise. Time-domain analysis helps understand how transient faults affect a circuit and can guide designers in producing noise-resistant circuitry. However, standard approaches to modeling intrinsic noise sources in the time domain are computationally expensive. Moreover, small noise-driven fluctuations in electron occupation of circuit nodes introduce time-varying biasing point fluctuations, increasing the modeling complexity. To address these challenges, this paper introduces a new approach to modeling thermal noise and random telegraph signal noise directly in the time domain by developing and solving a series of stochastic differential equations. In comparisons to traditional SPICE-based simulations, our approach can provide three orders of magnitude speedup in simulation time without sacrificing accuracy. Moreover, we introduce a novel, iterative threshold-crossing algorithm, aimed at the efficient sampling of rare noise transients. We show that Monte-Carlo simulations based on this approach can detect rare high-amplitude single event transients that would be impossible to uncover with standard transient simulators.