Ryan Kastner

The next generation of particle physics experiments will face an era of challenges in data acquisition, due to unprecedented data rates and volumes along with extreme environments and operational constraints. Harnessing these data for scientific discovery demands real-time inference and decision-making, intelligent data reduction, and efficient processing architectures beyond current capabilities. Crucial to the success of this experimental paradigm are several emerging technologies, such as artificial intelligence and machine learning, silicon microelectronics, and the advent of quantum algorithms and processing. Their intersection includes areas of research such as low-power and low-latency devices for edge computing, heterogeneous accelerator systems, reconfigurable hardware, codesign and synthesis strategies, readout for cryogenic or high-radiation environments, and analog computing. This white paper presents a community-driven vision to identify and prioritize research and development opportunities in hardware-based ML systems and corresponding physics applications, contributing toward a successful transition to the data frontier of fundamental science.

The scientific community increasingly relies on Machine Learning (ML) for near-sensor processing, leveraging its strengths in tasks such as pattern recognition, anomaly detection, and real-time decision-making. These deployments demand accelerators that combine extremely high performance with programmability, ease of integration, and straightforward verification. We present cgra4ml , an open source, modular framework that generates parameterizable CGRA accelerators in synthesizable SystemVerilog RTL, tailored to common ML compute patterns found in scientific applications. The framework supports seamless system integration through AXI-compliant interfaces and open source DMA components, and it includes automatic firmware generation for programming the accelerator. A comprehensive verification suite and a runtime firmware stack further support deployment across diverse SoC platforms. cgra4ml provides a modular, full-stack infrastructure, including a Python API, SystemVerilog hardware, TCL toolflows, and a C runtime, which facilitates easy integration and experimentation, allowing scientists to focus on innovation rather than dealing with the intricacies of hardware design and optimization. We demonstrate the effectiveness of cgra4ml to implement common scientific edge neural networks using ASIC and FPGA design flows.

As neural networks (NNs) are increasingly used to provide edge intelligence, there is a growing need to make the edge devices that run them robust to faults. Edge devices must mitigate the resulting hardware failures while maintaining strict constraints on power, energy, latency, throughput, memory size, and computational resources. Edge NNs require fundamental changes in model architecture, e.g., quantization and fewer, smaller layers. PrioriFI is a more informed fault injection (FI) algorithm that evaluates edge NN robustness by ranking NN bits based on their fault sensitivity. PrioriFI prioritizes finding highly fault-sensitive bits, that is, the bits most critical to an NN's correctness, first. To accomplish this, PrioriFI uses the Hessian for the initial parameter ranking. Then, during an FI campaign, PrioriFI uses the information gained from past FIs as a heuristic so that future FIs target the bits likely to be the next most sensitive. With PrioriFI, designers can quickly evaluate different NNs and better co-design fault-tolerant edge NN systems.

Generative AI and LLMs require significant computing performance. When used in critical applications like UAVs, UGVs, and other edge devices, they additionally need strong safety and security assurances. A zero-trust SoC hardware platform is crucial for delivering the necessary performance to execute advanced ML models while ensuring safe operation and securing data communication and computation. We propose a zero-trust SoC platform to support the deployment and operation of generative AI and LLMs based on (i) a zero-trust system interconnect, (ii) hardware-level fine-grained real-time analysis, (iii) fault-tolerance in on-chip communications and computation; and (iv) advanced mitigations for side-channel attacks. Our proposal is based on several research projects published in top-notch conferences and journals, a collaboration in a relevant SoC project, and collaborations with relevant industrial partners. Our proposal is based on research projects published in top-notch conferences and journals, a collaboration in a relevant SoC project, and partnerships with relevant industrial partners.

Modern System-on-Chip (SoC) architectures are a complex mix of processors, accelerators, memories, and I/O controllers interconnected by on-chip communication networks. Given the complexity of the computation and the requirements mandated in modern applications, several of these IPs are often outsourced as third-party modules. The integration of third-party modules, however, has been demonstrated to raise severe system-level security concerns – undiscovered vulnerabilities, incorrect firmware configurations, malicious code, and hardware trojans undetected in such IPs can produce leaks of confidential information and compromise the integrity of critical components. These challenges are further intensified when the communication infrastructure lacks robust mechanisms to supervise and monitor the interactions of third-party IPs with the rest of the system. Thus, runtime monitoring and supervising of third-party IPs is a crucial aspect for the system-level security of the entire SoC – the computing modules integrated in the SoC and their communication must behave securely. This paper presents Bastion, an open-source framework designed to support the secure integration of third-party IP modules into SoC architectures based on network-on-chip (NoC) communications, with a focus on providing robust security guarantees for NoC-based open-source hardware platforms. Unlike most previous works, which either focus on design or verification, we address the challenge of securely integrating third-party IPs in NoC-based platforms through a holistic design and verification framework based on three pillars: (i) a high-performance security socket that can be seamlessly integrated into NoC tiles; (ii) secure configuration and management of the security sockets via a Hardware Root of Trust; and (iii) an ad-hoc property-based security verification framework to ensure secure system operation. Bastion is integrated on the popular open-source ESP framework and validated through simulations and FPGA emulation of realistic SoCs. By explicitly targeting open-source platforms and releasing the entire project as open-source, we aim to democratize access to robustly secure application-specific SoC platforms for critical applications and foster further advancements in this domain.

Applications like high-energy physics and cybersecurity require extremely high throughput and low latency neural network (NN) inference. Lookup-table-based NNs address these constraints by implementing NNs as lookup tables (LUTs), achieving inference latency on the order of nanoseconds. Since LUTs are a fundamental FPGA building block, LUT-based NNs efficiently map to FPGAs. LogicNets (and its successors) form one class of LUT-based NNs that target FPGAs, mapping neurons directly to LUTs to meet low latency constraints with minimal resources. However, it is difficult to build larger, more performant LUT-based NNs like LogicNets because LUT usage increases exponentially with respect to neuron fan-in (i.e., number of synapses X synapse bitwidth). A large LUT-based NN quickly runs out of LUTs on an FPGA. Our work AmigoLUT addresses this issue by creating ensembles of smaller LUT-based NNs that scale linearly with respect to the number of models. AmigoLUT improves the scalability of LUT-based NNs, reaching higher throughput with up to an order of magnitude fewer LUTs than the largest LUT-based NNs.