Sergey Serebryakov

Resistive Random Access Memory (RRAM) based in-memory computing (IMC) accelerators offer significant performance and energy advantages for deep neural networks (DNNs), but face three major limitations: (1) they support only static dot-product operations and cannot accelerate arbitrary non-linear functions or data-dependent multiplications essential to modern LLMs; (2) they demand large, power-hungry analog-to-digital converter (ADC) circuits; and (3) mapping model weights to device conductance introduces errors from cell nonidealities. These challenges hinder scalable and accurate IMC acceleration as models grow. We propose NL-DPE, a Non-Linear Dot Product Engine that overcomes these barriers. NL-DPE augments crosspoint arrays with RRAM-based Analog Content Addressable Memory (ACAM) to execute arbitrary non-linear functions and data-dependent matrix multiplications in the analog domain by transforming them into decision trees, fully eliminating ADCs. To address device noise, NL-DPE uses software-based Noise Aware Fine-tuning (NAF), requiring no in-device calibration. Experiments show that NL-DPE delivers 28× energy efficiency and 249× speedup over a GPU baseline, and 22× energy efficiency and 245× speedup over existing IMC accelerators, while maintaining high accuracy.

Structured, or tabular, data is the most common format in data science. While deep learning models have proven formidable in learning from unstructured data such as images or speech, they are less accurate than simpler approaches when learning from tabular data. In contrast, modern tree-based Machine Learning (ML) models shine in extracting relevant information from structured data. An essential requirement in data science is to reduce model inference latency in cases where, for example, models are used in a closed loop with simulation to accelerate scientific discovery. However, the hardware acceleration community has mostly focused on deep neural networks and largely ignored other forms of machine learning. Previous work has described the use of an analog content addressable memory (CAM) component for efficiently mapping random forests. In this work, we develop an analog-digital architecture that implements a novel increased precision analog CAM and a programmable chip for inference of state-of-the-art tree-based ML models, such as XGBoost, CatBoost, and others. Thanks to hardware-aware training, X-TIME reaches state-of-the-art accuracy and 119× higher throughput at 9740× lower latency with > 150× improved energy efficiency compared with a state-of-the-art GPU for models with up to 4096 trees and depth of 8, with a 19W peak power consumption.

In this paper we present the capabilities of Common Metadata Framework (CMF) to enable trustworthy AI. CMF is a decentralized framework for tracking metadata and lineages of datasets and machine learning (ML) models in artificial intelligence (AI) pipelines. The framework provides a few unique features for ML practitioners, such as effortless management of distributed AI pipelines that span across the edge, high-performance systems, and public and private clouds. It provides an unbreakable audit trail and model provenance, resulting in trustworthy models. It ensures reproducibility by versioning artifacts and source code. CMF bridges the gap between the pipeline-centric and model-centric views of the AI metadata. This end-to-end approach to metadata logging unlocks a comprehensive understanding of ML workflows, enabling more efficient management and optimization of AI pipelines.

Due to the increasing adoption of Machine Learning (ML) and in particular Deep Learning (DL), many specialized energy efficient accelerators are being proposed by academia and industry. A number of these accelerators are designed to run a single application at a time in exclusive access mode. This approach gives applications maximum performance but reduces resource efficiency, resulting in increased costs over time. Sharing the device among multiple jobs increases resource utilization and amplifies return on investment. This study is driven by a broad investigation of various spatial resource sharing strategies in machine learning hardware accelerators and performance evaluation in a novel memristor-based accelerator called PUMA [1]. Two methods of spatial sharing are discussed: Model Packing and Logical Allocation. Simulations showed that both methods can be implemented on the PUMA accelerator and have advantages in terms of increased resource utilization. The former spatial sharing strategy achieves higher level of parallelism, fitting more models per device (7 models on 11 tiles), but has higher interference overhead (up to 49%), still being in most cases better than the overhead found for GPUs. The latter spatial sharing strategy achieves better isolation with almost no interference overhead (<1%) with the cost of leaving resources unused (same 7 models consumed 16 tiles). Finally, we discuss security implications of resource sharing for ML and other concerns, presenting a novel ML model integrity check and model bias verification.

Deep Learning (DL) has gained wide adoption and different DL models have been deployed for an expanding number of applications. It is being used both for inference at the edge and for training in datacenters. Applications include image recognition, video analytics, pattern recognition in networking traffic, and many others. Different applications rely on different neural network models, and it has proven difficult to predict resource requirements for different models and applications. This leads to the nonsystematic and suboptimal selection of computational resources for DL applications resulting in overpaying for underutilized infrastructure or, even worse, the deployment of models on underpowered hardware and missed service level objectives. In this paper we present the DL Cookbook, a toolset that a) helps with benchmarking models on different hardware, b) guides the use of DL and c) provides reference designs. Automated benchmarking collects performance data for different DL workloads (training and inference with different models) on various hardware and software configurations. A web-based tool guides a choice of optimal hardware and software configuration via analysis of collected performance data and applying performance models. And finally, it offers reference hardware/software stacks for particular classes of deep learning workloads. This way the DL Cookbook helps both customers and hardware vendors match optimal DL models to the available hardware and vice versa, in case of acquisition, specify required hardware to models in question. Finally, DL Cookbook helps with reproducibility of results.