RKRajgopal Kannan

Rajgopal Kannan

*This is an algorithmically generated author record based on publicly available information.

Rajgopal Kannan has a total of 68 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Latent Denoising Improves Visual Alignment in Large Multimodal Models

    by Dhruv Parikh, Jacob Fein-Ashley, Rajgopal Kannan, Viktor Prasanna

  • Open Access

    S 2 MM: Scalable FPGA Acceleration of Secure Matrix Multiplication with Homomorphic Encryption

    by Zhihan Xu, Rajgopal Kannan, Viktor Prasanna

    Homomorphic Encryption (HE) enables secure computation on encrypted data, addressing privacy concerns in cloud computing. However, the high computational cost of HE operations, particularly matrix multiplication (MM), remains a major barrier to its practical deployment. Accelerating Homomorphic Encrypted MM (HE MM) is crucial for applications such as privacy-preserving machine learning. In this article, we present S \({}^{2}\) MM, the first scalable FPGA-based accelerator designed for HE MM. We show that HE MM is fundamentally bottlenecked by excessive off-chip memory traffic when operating with practical HE parameter sets. To analyze this bottleneck, we develop a cost model that characterizes the on-chip memory requirements of ciphertexts during HE MM, revealing the scalability limits of CPU-based implementations. Guided by the analysis, we propose a novel datapath for Homomorphic Linear Transformation (HLT), the dominant workload in HE MM. The proposed datapath enables fine-grained on-chip data reuse, substantially reducing both off-chip memory traffic and on-chip memory demand. To further improve scalability, S \({}^{2}\) MM supports tiled HE MM, removing the matrix size limitation imposed by the HE encoding capacity and enabling efficient computation on large matrices. In addition, the architecture is highly configurable, adapting to diverse matrix sizes and HE parameter sets while maximizing throughput within FPGA resource constraints. We also introduce a systematic configuration methodology to determine optimal design parameters for efficient deployment across varying workloads and platforms. We implement S \({}^{2}\) MM on an Alveo U280 and conduct comprehensive evaluations across a wide range of matrix sizes, shapes, and HE parameters. Experimental results demonstrate that S \({}^{2}\) MM achieves average speedups of 219 \(\times\) and 37.4 \(\times\) over state-of-the-art CPU and GPU implementations, respectively. We further evaluate S \({}^{2}\) MM in an application-level encrypted CNN inference pipeline, achieving a 6.08 \(\times\) end-to-end speedup by accelerating the HE MM-dominated fully connected layers.

  • Open Access

    OLA+: Multi-FPGA O ver l ay A ccelerator System for Fully Homomorphic Encryption

    by Yang Yang, Rajgopal Kannan, Viktor K. Prasanna

    Fully Homomorphic Encryption (FHE) is a promising technique for privacy preserving computation. However, homomorphically encrypted operations are orders of magnitude slower than the corresponding unencrypted operations due to high computation and memory bandwidth requirements. FPGAs are attractive platforms for accelerating FHE workloads, but manually programming FPGAs is challenging, as different FHE parameters and operations require different mapping strategies. We propose OLA+, a scalable overlay accelerator system for FHE, designed to run efficiently on multi-FPGA platforms. OLA+ eliminates the need for manual and time-consuming FPGA programming by providing a Python-based interface and a compiler that efficiently maps OLA+ programs to hardware instructions for execution. The hardware architecture and instruction set are co-designed to accelerate common FHE primitives, while the compiler maps all FHE operations to these primitives for efficient execution. We propose a compile-time partitioning strategy that parallelizes computation both across multiple ciphertexts and within individual ciphertexts, enabling efficient execution of OLA+ programs on multiple FPGA accelerators. OLA+ features two optimizations to address memory bandwidth challenges in FHE computation. First, we propose an asynchronous dataflow execution model where the compiler manages data processing order and the architecture enforces it at run time. This approach enables guaranteed data reuse via on-chip SRAMs. Second, we design latency-aware instruction scheduling in the compiler to reduce data reuse distance and overlap data transfers with computation. We implement the overlay accelerator on AMD Alveo U280 FPGAs. We evaluate the effectiveness of the proposed overlay accelerator across a range of FHE parameters by executing various FHE operations, FHE linear algebra benchmarks, and end-to-end ML training and inference. Experimental results show that running end-to-end HE ML training and inference tasks on OLA+ with 8 overlay accelerators achieves speedups of up to \(8835\times\) and \(14.6\times\) over state-of-the-art CPU and GPU implementations, respectively.

  • Open Access

    ConsensusDrop: Fusing Visual and Cross-Modal Saliency for Efficient Vision Language Models

    by Dhruv Parikh, Haoyang Fan, Rajgopal Kannan, Viktor Prasanna

    Vision-Language Models (VLMs) are expensive because the LLM processes hundreds of largely redundant visual tokens. Existing token reduction methods typically exploit \textit{either} vision-encoder saliency (broad but query-agnostic) \textit{or} LLM cross-attention (query-aware but sparse and costly). We show that neither signal alone is sufficient: fusing them consistently improves performance compared to unimodal visual token selection (ranking). However, making such fusion practical is non-trivial: cross-modal saliency is usually only available \emph{inside} the LLM (too late for efficient pre-LLM pruning), and the two signals are inherently asymmetric, so naive fusion underutilizes their complementary strengths. We propose \textbf{ConsensusDrop}, a training-free framework that derives a \emph{consensus} ranking by reconciling vision encoder saliency with query-aware cross-attention, retaining the most informative tokens while compressing the remainder via encoder-guided token merging. Across LLaVA-1.5/NeXT, Video-LLaVA, and other open-source VLMs, ConsensusDrop consistently outperforms prior pruning methods under identical token budgets and delivers a stronger accuracy-efficiency Pareto frontier -- preserving near-baseline accuracy even at aggressive token reductions while reducing TTFT and KV cache footprint. Our code will be open-sourced.

  • Open Access

    FAME: FPGA Acceleration of Secure Matrix Multiplication with Homomorphic Encryption

    by Zhihan Xu, Rajgopal Kannan, Viktor K. Prasanna

    Homomorphic Encryption (HE) enables secure computation on encrypted data, addressing privacy concerns in cloud computing. However, the high computational cost of HE operations, particularly matrix multiplication (MM), remains a major barrier to its practical deployment. Accelerating homomorphic encrypted MM (HE MM) is therefore crucial for applications such as privacy-preserving machine learning. In this paper, we present a bandwidth-efficient FPGA implementation of HE MM. We first develop a cost model to evaluate the on-chip memory requirements for a given set of HE parameters and input matrix sizes. Our analysis shows that optimizing on-chip memory usage is critical for scalable and efficient HE MM. To this end, we design a novel datapath for Homomorphic Linear Transformation (HLT), the primary bottleneck in HE MM. The proposed datapath significantly reduces off-chip memory traffic and on-chip memory demand by enabling fine-grained data reuse. Leveraging this datapath, we introduce FAME, the first FPGA-based accelerator specifically tailored for HE MM. FAME supports arbitrary matrix shapes and is configurable across a wide range of HE parameter sets. We implement FAME on an Alveo U280 FPGA and evaluate its performance across diverse matrix sizes and shapes. Experimental results show that FAME achieves an average speedup of 221x over state-of-the-art CPU-based implementations, demonstrating its scalability and practicality for large-scale consecutive HE MM and real-world workloads.