Robust Batch-Level Query Routing for Large Language Models under Cost and Capacity Constraints
Open Access
- 26 May 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 890-916
Abstract
No abstract availableAll Related Versions
This publication has 11 references indexed in Scilit:
- OmniRouter: Budget and Performance Controllable Multi-LLM RoutingACM SIGKDD Explorations Newsletter, 2025
- Distributionally robust optimizationActa Numerica, 2025
- Proteus: A High-Throughput Inference-Serving System with Accuracy Scaling2024
- Efficient Memory Management for Large Language Model Serving with PagedAttention2023
- SMDP-Based Dynamic Batching for Efficient Inference on GPU-Based Platforms2023
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks2019
- XGBoost2016
- Robust Optimization2009
- Optimization of conditional value-at-riskJournal of Risk, 2000
- Bootstrap Methods: Another Look at the Jackknife1992