Abstract
Modern AI vision deployments behave like continuous dataflow systems: thousands of camera streams require repeated data processing on the CPU before any neural network can run on the GPU. In multi-model DAG pipelines, these data processing steps multiply across stages, consuming significant CPU cycles and leaving GPUs underutilized. The CPU-bound nature of these tasks limits overall throughput, increases latency, and forces costly over-provisioning. GLENFINNAN is a SmartNIC-accelerated data processing system that restructures common image transformations into a streaming, packet-local, lookup-table-driven pipeline. By exploiting pixel locality, memoizing expensive arithmetic, and maintaining only minimal per-packet state, GLENFINNAN performs key data processing operations directly on the NIC. Across six representative pipelines, including CNNs and multi-model DAGs, it reduces CPU utilization by up to 91%, lowers estimated data processing latency by 40–50%, and supports 2.1–3.4× more concurrent streams per node while consuming only 6–14% of NIC logic resources. These results show that programmable NICs can act as first-class compute stages for vision workloads, reshaping system bottlenecks and improving cluster-wide efficiency.