RSRathijit Sen

Rathijit Sen

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Rathijit Sen has a total of 140 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    CANNON: A CXL-Based Near-Memory Processing Architecture for Approximate Nearest Neighbor Search on Real Hardware

    by Ganghyun Kim, Hyeonjun An, Hyunjoon Kim, Rathijit Sen, Kangkyu Park, Jinho Baek, Kwangsik Shin, Youngpyo Joo, Kwanghyun Park

  • Open Access

    GASP: GPU-Accelerated Shortest Path for Graph Analytics

    by Ian Di Dio Lavore, Rathijit Sen, Yuanyuan Tian

    GPUs have shown significant potential for accelerating database analytical queries, but leveraging them for graph analytics queries involving weighted shortest-path computations remains challenging. This stems from the need to efficiently support diverse query patterns that require multi-source, multi-target computations, tracking all shortest paths, ranking paths based on cost, as well as ease of integration with columnar data processing systems. We present GASP, a GPU-accelerated framework for weighted shortest-path graph analytics queries designed to operate within columnar and tensor-based data processing systems. GASP uses a supernode-based reduction of multi-source shortest-path problems to single-source ones, a multi-column predecessors table for tracking shortest paths, and efficient post-processing with GPU-parallel backtracking and sorting. Experimental results show that GASP, running on an H100 GPU, achieves speedups of orders of magnitude over CPU-only relational and graph database systems for shortest-path computations in LDBC SNB BI queries.

  • Open Access

    CoddSpeed: Hardware Accelerated Query Processing in Microsoft Fabric

    by Matteo Interlandi, Nicolas Bruno, Brandon Haynes, Carlo Curino, Rathijit Sen, Yinan Li, Kaushik Rajan, Bailu Ding, Lukas M. Maas, Wei Cui, et al.

    Over the past three decades, Microsoft has developed several multibillion-dollar data management products. We have now unified them into Microsoft Fabric, a comprehensive SaaS suite for data management based on a modern cloud-native design that delivers best-in-class analytics performance on CPUs. Meanwhile, Microsoft's AI investments have transformed our data centers. Hardware accelerators are now plentiful and surpass traditional CPU servers by orders of magnitude in compute, memory, and networking capabilities. Running analytics on hardware accelerators is therefore a business imperative. In this paper, we describe a multi-year effort to take Fabric to the next level by enabling analytics execution on state-of-the-art hardware accelerators. Given the rapid evolution of hardware, we have made hardware flexibility our guiding principle. We validate our architectural flexibility by demonstrating how we can run Fabric engines (e.g., Data Warehouse) on a variety of compute and network accelerators (e.g., GPUs, FPGAs, ASICs, NVLink, InfiniBand). We present our most mature implementation, a GPU-based execution engine derived from our Tensor Query Processor, and report key results from a novel data movement system leveraging NVLink and InfiniBand. This version of Fabric outperforms its CPU counterpart by over an order of magnitude across a host of production and benchmark scenarios (e.g., delivering up to 30× speedup on TPC-H 1TB). We share the lessons learned in building one of the first systems in the era of accelerated analytics.


  • Open Access

    22 nd International Workshop on Data Management on New Hardware (DaMoN)

    by Jana Giceva, Eric Sedlar, Rathijit Sen

    Modern hardware -- including tile-based architectures, various accelerators (e.g., GPUs, FPGAs, DPUs, NPUs), as well as emerging memory and storage technologies -- introduce both new challenges and significant opportunities for optimizing database systems performance. Effectively leveraging these hardware characteristics has become a central topic of database systems research. For more than two decades, the DaMoN Workshop has served as a premier venue for exploring these ideas. It provides a platform for presenting research on hardware-conscious data management, but also for fostering closer collaboration between the data management and the broader systems community, including computer architecture, compilers, operating systems and hardware/software co-design.

  • Open Access

    DFLOP: A Data-driven Framework for Multimodal LLM Training Pipeline Optimization

    by Hyeonjun An, Sihyun Kim, Chaerim Lim, Hyunjoon Kim, Rathijit Sen, Sangmin Jung, Hyeonsoo Lee, Dongwook Kim, Takki Yu, Jinkyu Jeong, et al.

    Multimodal Large Language Models (MLLMs) have achieved remarkable advances by integrating text, image, and audio understanding within a unified architecture. However, existing distributed training frameworks remain fundamentally data-blind: they parallelize computation without accounting for variations in input data characteristics. This data unawareness leads to severe computation skew across stages and microbatches, where heterogeneous multimodal inputs incur different processing costs. Consequently, GPU resources are unevenly utilized, synchronization delays accumulate, and overall training efficiency degrades. To address this limitation, we present DFLOP, a d ata-driven f ramework for multimodal L LM training pipeline op timization. DFLOP continuously profiles runtime behavior to capture data-induced computation variance and employs predictive scheduling to balance workloads across stages and microbatches. By coupling data characteristics with execution planning, DFLOP substantially improves GPU utilization and throughput. Extensive experiments on large-scale multimodal benchmarks show that DFLOP achieves up to 3.6× faster training compared to state-of-the-art distributed training frameworks.

  • Open Access

    Rockhopper: A Robust Optimizer for Spark Configuration Tuning in Production Environment

    by Yiwen Zhu, Rathijit Sen, Brian Kroth, Sergiy Matusevych, Andreas Mueller, Tengfei Huang, Rahul Challapalli, Weihan Tang, Xin He, Mo Liu, et al.


  • Open Access

    Performance or Efficiency? A Tale of Two Cores for DB Workloads

    by Rathijit Sen

    We study the performance, power, and thermal profiles for database workloads on hybrid P-core and E-core CPUs. We find that E-cores run cooler than P-cores, use less power, and are more energy-efficient, but need to be provisioned more than 3x the number of P-cores for equivalent performance on multi-threaded analytics workloads. For latency-critical transactional workloads, E-cores can significantly extend very short tail latencies but have less impact for workloads with longer latencies. Asymmetric work distribution policies skewed towards P-cores can improve grouped aggregation performance compared to symmetric policies.


  • Open Access

    JoinBoost: Grow Trees Over Normalized Data Using Only SQL

    by Zezhou Huang, Rathijit Sen, Jiaxiang Liu, Eugene Wu

    Although dominant for tabular data, ML libraries that train tree models over normalized databases (e.g., LightGBM, XGBoost) require the data to be denormalized as a single table, materialized, and exported. This process is not scalable, slow, and poses security risks. In-DB ML aims to train models within DBMSes to avoid data movement and provide data governance. Rather than modify a DBMS to support In-DB ML, is it possible to offer competitive tree training performance to specialized ML libraries...with only SQL? We present JoinBoost, a Python library that rewrites tree training algorithms over normalized databases into pure SQL. It is portable to any DBMS, offers performance competitive with specialized ML libraries, and scales with the underlying DBMS capabilities. JoinBoost extends prior work from both algorithmic and systems perspectives. Algorithmically, we support factorized gradient boosting, by updating the $Y$ variable to the residual in the non-materialized join result. Although this view update problem is generally ambiguous, we identify addition-to-multiplication preserving, the key property of variance semi-ring to support rmse, the most widely used criterion. System-wise, we identify residual updates as a performance bottleneck. Such overhead can be natively minimized on columnar DBMSes by creating a new column of residual values and adding it as a projection. We validate this with two implementations on DuckDB, with no or minimal modifications to its internals for portability. Our experiment shows that JoinBoost is 3x (1.1x) faster for random forests (gradient boosting) compared to LightGBM, and over an order magnitude faster than state-of-the-art In-DB ML systems. Further, JoinBoost scales well beyond LightGBM in terms of the # features, DB size (TPC-DS SF=1000), and join graph complexity (galaxy schemas).

  • Open Access

    Random Forests over normalized data in CPU-GPU DBMSes

    by Zezhou Huang, Pavan Kalyan Damalapati, Rathijit Sen, Eugene Wu

    This short paper studies query execution based on message passing on CPU-GPU systems, using random forests training as the workload. We investigate different data placement and query execution strategies and find that the unique properties of training ML models using message passing necessitates different design decisions. We show that with proper data placement and CPU-GPU co-execution, training random forest models using pure SQL can outperform the leading LightGBM ML library by 1.5 × on SSB SF=10.