LibPreemptible: Enabling Fast, Adaptive, and Hardware-Assisted User-Space Scheduling
Authors:
- 2 March 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 922-936
Abstract
Modern cloud applications are prone to high tail latencies since their requests typically follow highly-dispersive distributions. Prior work has proposed both OS- and systemlevel solutions to reduce tail latencies for microsecond-scale workloads through better scheduling. Unfortunately, existing approaches like customized dataplane OSes, require significant OS changes, experience scalability limitations, or do not reach the full performance capabilities hardware offers. We propose LibPreemptible, a preemptive user-level threading library that is flexible, lightweight, and scalable. LibPreemptible is based on three key techniques: 1) a fast and lightweight hardware mechanism for delivery of timed interrupts, 2) a general-purpose user-level scheduling interface, and 3) an API for users to express adaptive scheduling policies tailored to the needs of their applications. Compared to the prior state-of-the-art scheduling system Shinjuku, our system achieves significant tail latency and throughput improvements for various workloads without the need to modify the kernel. We also demonstrate the flexibility of LibPreemptible across scheduling policies for real applications experiencing varying load levels and characteristics.Keywords
This publication has 32 references indexed in Scilit:
- ALTOCUMULUS: Scalable Scheduling for Nanosecond-Scale Remote Procedure Calls2022
- Slite: OS Support for Near Zero-Cost, Configurable Scheduling2020
- The Impact of Thread-Per-Core Architecture on Application Tail Latency2019
- Express-Lane Scheduling and Multithreading to Minimize the Tail Latency of Microservices2019
- Join-Idle-Queue: A novel load balancing algorithm for dynamically scalable web servicesPerformance Evaluation, 2011
- Optimal Selection of Preemption Points to Minimize Preemption Overhead2011
- Performance Evaluation with Heavy Tailed Distributions2000
- Estimating the Heavy Tail Index from Scaling PropertiesMethodology and Computing in Applied Probability, 1999
- Scheduler activationsACM Transactions on Computer Systems, 1992
- First-class user-level threadsACM SIGOPS Operating Systems Review, 1991