Marlin: Efficient Coordination for Autoscaling Cloud DBMS
Open Access
- 22 September 2025
- journal article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Management of Data
- Vol. 3 (4) , 1-28
Abstract
Modern cloud databases are shifting from converged architectures to storage disaggregation, enabling independent scaling and billing of compute and storage. However, cloud databases still rely on external, converged coordination services (e.g., ZooKeeper) for their control planes. These services are effectively lightweight databases optimized for low-volume metadata. As the control plane scales in the cloud, this approach faces similar limitations as converged databases did before storage disaggregation: scalability bottlenecks, low cost efficiency, and increased operational burden. We propose to disaggregate the cluster coordination to achieve the same benefits that storage disaggregation brought to modern cloud DBMSs. We present Marlin, a cloud-native coordination mechanism that fully embraces storage disaggregation. Marlin eliminates the need for external coordination services by consolidating coordination functionality into the existing cloud-native database it manages. To achieve failover without an external coordination service, Marlin allows cross-node modifications on coordination states. To ensure data consistency, Marlin employs transactions to manage both coordination and application states and introduces MarlinCommit, an optimized commit protocol that ensures strong transactional guarantees even under cross-node modifications. Our evaluations demonstrate that Marlin improves cost efficiency by up to 4.4x and reduces reconfiguration duration by up to 4.9x compared to converged coordination solutions.Keywords
Funding Information
- National Science Foundation (2144588)
This publication has 40 references indexed in Scilit:
- Tourism promotion during emergency response to Omicron subvariant outbreakHeliyon, 2024
- Zab: High-performance broadcast for primary-backup systems2011
- BigtableACM Transactions on Computer Systems, 2008
- Workload Analysis and Demand Prediction of Enterprise Data Center Applications2007
- Paxos made live2007
- SWIM: scalable weakly-consistent infection-style process group membership protocol2003
- Peer-to-peer membership management for gossip-based protocolsIEEE Transactions on Computers, 2003
- Improving the Performance of an Optimistic Concurrency Control Algorithm Through Timestamps and VersionsIEEE Transactions on Software Engineering, 1987
- Observations on optimistic concurrency control schemesInformation Systems, 1984
- Optimistic versus pessimistic concurrency control mechanisms in database management systemsInformation Systems, 1982