SAIA: A Seamless Slurm-Native Solution for HPC-Based Services
Preprint
- 29 July 2025
- preprint
- Published by Springer Nature in Research Square
Abstract
Recent developments indicate a shift toward web services that employ ever larger AI models, e.g., Large Language Models (LLMs), requiring powerful hardware for inference. High-Performance Computing (HPC) systems are commonly equipped with such hardware for the purpose of large scale computation tasks. However, HPC infrastructure is inherently unsuitable for hosting real-time web services due to network, security and scheduling constraints. While various efforts exist to integrate external scheduling solutions, these often require compromises in terms of security or usability for existing HPC users. In this paper, we present SAIA, a Slurm-native platform consisting of a scheduler and a proxy. The scheduler interacts with Slurm to ensure the availability and scalability of services, while the proxy provides external access, which is secured via confined SSH commands. We have demonstrated SAIA’s applicability by deploying a large-scale LLM web service that has served over 50,000 users.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- Large language models in education: A focus on the complementary relationship between human teachers and ChatGPTEducation and Information Technologies, 2023
- NVIDIA’s Cloud Native Supercomputing2022
- MS-QuAAF: A generic evaluation framework for monitoring software architecture qualityInformation and Software Technology, 2021
- Highly accurate protein structure prediction with AlphaFoldNature, 2021
- Large-scale HPC deployment of Scalable CyberInfrastructure for Artificial Intelligence and Likelihood Free Inference (SCAILFIN)EPJ Web of Conferences, 2020
- Deploying AI Frameworks on Secure HPC Systems with Containers2019
- Artificial Intelligence Platform for Mobile Service ComputingJournal of Signal Processing Systems, 2019
- Improving Mandatory Access Control for HPC clustersFuture Generation Computer Systems, 2013
- Introduction of a web service for cloud computing with the integrated hydrologic simulation platform ParFlowComputers & Geosciences, 2012
- SLURM: Simple Linux Utility for Resource Management2003