GLENFINNAN: SmartNIC-Accelerated Data Processing for Efficient Vision AI Pipelines
Open Access
- 11 August 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1966-1972
Abstract
Modern AI vision deployments behave like continuous dataflow systems: thousands of camera streams require repeated data processing on the CPU before any neural network can run on the GPU. In multi-model DAG pipelines, these data processing steps multiply across stages, consuming significant CPU cycles and leaving GPUs underutilized. The CPU-bound nature of these tasks limits overall throughput, increases latency, and forces costly over-provisioning. GLENFINNAN is a SmartNIC-accelerated data processing system that restructures common image transformations into a streaming, packet-local, lookup-table-driven pipeline. By exploiting pixel locality, memoizing expensive arithmetic, and maintaining only minimal per-packet state, GLENFINNAN performs key data processing operations directly on the NIC. Across six representative pipelines, including CNNs and multi-model DAGs, it reduces CPU utilization by up to 91%, lowers estimated data processing latency by 40–50%, and supports 2.1–3.4× more concurrent streams per node while consuming only 6–14% of NIC logic resources. These results show that programmable NICs can act as first-class compute stages for vision workloads, reshaping system bottlenecks and improving cluster-wide efficiency.Keywords
This publication has 18 references indexed in Scilit:
- Pylot: A Modular Platform for Exploring Latency-Accuracy Tradeoffs in Autonomous Vehicles2021
- U2-Net: Going deeper with nested U-structure for salient object detectionPattern Recognition, 2020
- HPIPE: Heterogeneous Layer-Pipelined and Sparse-Aware CNN Inference for FPGAs2020
- Do Switches Dream of Machine Learning?2019
- Nexus2019
- FINN- RACM Transactions on Reconfigurable Technology and Systems, 2018
- Chameleon2018
- Can the Network be the AI Accelerator?2018
- Mask R-CNN2017
- NetFPGA SUME: Toward 100 Gbps as Research CommodityIEEE Micro, 2014