Kamran Razavi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Kamran Razavi has a total of 16 co-authors. Here are the top co-authors by number of shared publications:
- Lin Wang
Paderborn University, Paderborn, Germany
4 shared publications - Max Mühlhäuser
Technical University of Darmstadt, Darmstadt, Germany
3 shared publications - Mehran Salmani
Computer Science and Automation, TU Ilmenau, Ilmenau, Germany
3 shared publications - Pooyan Jamshidi
University of South Carolina
3 shared publications - Saeid Ghafouri
Queen's University Belfast, Belfast, United Kingdom
3 shared publications - Alireza Sanaee
University of Cambridge, Cambridge, United Kingdom and Huawei, Cambridge, United Kingdom
2 shared publications
Latest Publications
- Open Access
- 27 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 163-170
Bridging CPU and GPU Autoscaling for Cost-Efficient Inference Serving
Autoscaling mechanisms have been used by inference serving systems to maintain service quality and efficiently use cloud resources (CPUs/GPUs). Horizontal scaling-based mechanisms have a responsiveness issue due to cold-start delays. In-place CPU vertical scaling adopted by recent works avoids cold-start; however, it is constrained by hardware limits and model parallelizability, and is limited to CPUs, which are efficient for low workloads but cost-inefficient for large workloads. Thus, combining CPUs and GPUs under dynamic workloads is worthwhile. However, enabling responsive switching between them is challenging. We propose VCG, an inference serving system that utilizes different resources based on the workload, and enables a smooth transition from CPUs to GPUs and vertically scaling GPUs (adjusting the GPU percentage) by forking a new process with enough GPU compute resources using CUDA MPS, and delivering inference requests to the child process through inter-process communication pipes. When the capacity of service is (nearly) exhausted, VCG uses horizontal scaling to provide scalability headroom for future steps. We evaluate a prototype of VCG under two different workloads and show that it reduces SLO violations up to 8× compared to a baseline using horizontal scaling of GPU instances under a bursty workload. It can also reduce the cost by around 60% compared to two CPU-based autoscaling baselines under a dynamic (non-bursty) workload.
- 27 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 163-170
Open AccessBridging CPU and GPU Autoscaling for Cost-Efficient Inference Serving
Autoscaling mechanisms have been used by inference serving systems to maintain service quality and efficiently use cloud resources (CPUs/GPUs). Horizontal scaling-based mechanisms have a responsiveness issue due to cold-start delays. In-place CPU vertical scaling adopted by recent works avoids cold-start; however, it is constrained by hardware limits and model parallelizability, and is limited to CPUs, which are efficient for low workloads but cost-inefficient for large workloads. Thus, combining CPUs and GPUs under dynamic workloads is worthwhile. However, enabling responsive switching between them is challenging. We propose VCG, an inference serving system that utilizes different resources based on the workload, and enables a smooth transition from CPUs to GPUs and vertically scaling GPUs (adjusting the GPU percentage) by forking a new process with enough GPU compute resources using CUDA MPS, and delivering inference requests to the child process through inter-process communication pipes. When the capacity of service is (nearly) exhausted, VCG uses horizontal scaling to provide scalability headroom for future steps. We evaluate a prototype of VCG under two different workloads and show that it reduces SLO violations up to 8× compared to a baseline using horizontal scaling of GPU instances under a bursty workload. It can also reduce the cost by around 60% compared to two CPU-based autoscaling baselines under a dynamic (non-bursty) workload.
- Open Access
- 5 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 2-3
IPA: Inference Pipeline Adaptation to Achieve High Accuracy and Cost-Efficiency
- 5 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 2-3
Open AccessIPA: Inference Pipeline Adaptation to Achieve High Accuracy and Cost-Efficiency
- 13 September 2024
- Published by Springer Nature
- p. 245-265
Towards Democratic Computing
- Open Access
- 22 April 2024
- Published by Association for Computing Machinery (ACM)
- p. 184-191
Sponge
Mobile and IoT applications increasingly adopt deep learning inference to provide intelligence. Inference requests are typically sent to a cloud infrastructure over a wireless network that is highly variable, leading to the challenge of dynamic Service Level Objectives (SLOs) at the request level. This paper presents Sponge, a novel deep learning inference serving system that maximizes resource efficiency while guaranteeing dynamic SLOs. Sponge achieves its goal by applying in-place vertical scaling, dynamic batching, and request reordering. Specifically, we introduce an Integer Programming formulation to capture the resource allocation problem, providing a mathematical model of the relationship between latency, batch size, and resources. We demonstrate the potential of Sponge through a prototype implementation and preliminary experiments and discuss future works.
- 22 April 2024
- Published by Association for Computing Machinery (ACM)
- p. 184-191
Open AccessSponge
Mobile and IoT applications increasingly adopt deep learning inference to provide intelligence. Inference requests are typically sent to a cloud infrastructure over a wireless network that is highly variable, leading to the challenge of dynamic Service Level Objectives (SLOs) at the request level. This paper presents Sponge, a novel deep learning inference serving system that maximizes resource efficiency while guaranteeing dynamic SLOs. Sponge achieves its goal by applying in-place vertical scaling, dynamic batching, and request reordering. Specifically, we introduce an Integer Programming formulation to capture the resource allocation problem, providing a mathematical model of the relationship between latency, batch size, and resources. We demonstrate the potential of Sponge through a prototype implementation and preliminary experiments and discuss future works.
- Open Access
- 8 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 78-86
Reconciling High Accuracy, Cost-Efficiency, and Low Latency of Inference Serving Systems
The use of machine learning (ML) inference for various applications is growing drastically. ML inference services engage with users directly, requiring fast and accurate responses. Moreover, these services face dynamic workloads of requests, imposing changes in their computing resources. Failing to right-size computing resources results in either latency service level objectives (SLOs) violations or wasted computing resources. Adapting to dynamic workloads considering all the pillars of accuracy, latency, and resource cost is challenging. In response to these challenges, we propose InfAdapter, which proactively selects a set of ML model variants with their resource allocations to meet latency SLO while maximizing an objective function composed of accuracy and cost. InfAdapter decreases SLO violation and costs up to 65% and 33%, respectively, compared to a popular industry autoscaler (Kubernetes Vertical Pod Autoscaler).
- 8 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 78-86
Open AccessReconciling High Accuracy, Cost-Efficiency, and Low Latency of Inference Serving Systems
The use of machine learning (ML) inference for various applications is growing drastically. ML inference services engage with users directly, requiring fast and accurate responses. Moreover, these services face dynamic workloads of requests, imposing changes in their computing resources. Failing to right-size computing resources results in either latency service level objectives (SLOs) violations or wasted computing resources. Adapting to dynamic workloads considering all the pillars of accuracy, latency, and resource cost is challenging. In response to these challenges, we propose InfAdapter, which proactively selects a set of ML model variants with their resource allocations to meet latency SLO while maximizing an objective function composed of accuracy and cost. InfAdapter decreases SLO violation and costs up to 65% and 33%, respectively, compared to a popular industry autoscaler (Kubernetes Vertical Pod Autoscaler).
- Open Access
Operator as a Service: Stateful Serverless Complex Event Processing
Complex Event Processing (CEP) is a powerful paradigm for scalable data management that is employed in many real-world scenarios such as detecting credit card fraud in banks. The so-called complex events are expressed using a specification language that is typically implemented and executed on a specific runtime system. While the tight coupling of these two components has been regarded as the key for supporting CEP at high performance, such dependencies pose several inherent challenges as follows. (1) Application development atop a CEP system requires extensive knowledge of how the runtime system operates, which is typically highly complex in nature. (2) The specification language dependence requires the need of domain experts and further restricts and steepens the learning curve for application developers. In this paper, we propose CEPLESS, a scalable data management system that decouples the specification from the runtime system by building on the principles of serverless computing. CEPLESS provides operator as a service and offers flexibility by enabling the development of CEP application in any specification language while abstracting away the complexity of the CEP runtime system. As part of CEPLESS, we designed and evaluated novel mechanisms for in-memory processing and batching that enables the stateful processing of CEP operators even under high rates of ingested events. Our evaluation demonstrates that CEPLESS can be easily integrated into existing CEP systems like Apache Flink while attaining similar throughput under a high scale of events (up to 100K events per second) and dynamic operator update in up to 238 ms.
Open AccessOperator as a Service: Stateful Serverless Complex Event Processing
Complex Event Processing (CEP) is a powerful paradigm for scalable data management that is employed in many real-world scenarios such as detecting credit card fraud in banks. The so-called complex events are expressed using a specification language that is typically implemented and executed on a specific runtime system. While the tight coupling of these two components has been regarded as the key for supporting CEP at high performance, such dependencies pose several inherent challenges as follows. (1) Application development atop a CEP system requires extensive knowledge of how the runtime system operates, which is typically highly complex in nature. (2) The specification language dependence requires the need of domain experts and further restricts and steepens the learning curve for application developers. In this paper, we propose CEPLESS, a scalable data management system that decouples the specification from the runtime system by building on the principles of serverless computing. CEPLESS provides operator as a service and offers flexibility by enabling the development of CEP application in any specification language while abstracting away the complexity of the CEP runtime system. As part of CEPLESS, we designed and evaluated novel mechanisms for in-memory processing and batching that enables the stateful processing of CEP operators even under high rates of ingested events. Our evaluation demonstrates that CEPLESS can be easily integrated into existing CEP systems like Apache Flink while attaining similar throughput under a high scale of events (up to 100K events per second) and dynamic operator update in up to 238 ms.