DAOS and Friends: A Proposal for an Exascale Storage System

Authors:
  • JLJay Lofstead
    Jay Lofstead
  • CMCarlos Maltzahn
    Carlos Maltzahn
  • QKQuincey Koziol
    Quincey Koziol
  • JBJohn Bent
    John Bent
  • EBEric Barton
    Eric Barton

Abstract
The DOE Extreme-Scale Technology Acceleration Fast Forward Storage and IO Stack project is going to have significant impact on storage systems design within and beyond the HPC community. With phase two of the project starting, it is an excellent opportunity to explore the complete design and how it will address the needs of extreme scale platforms. This paper examines each layer of the proposed stack in some detail along with cross-cutting topics, such as transactions and metadata management. This paper not only provides a timely summary of important aspects of the design specifications but also captures the underlying reasoning that is not available elsewhere. We encourage the broader community to understand the design, intent, and future directions to foster discussion guiding phase two and the ultimate production storage stack based on this work. An initial performance evaluation of the early prototype implementation is also provided to validate the presented design.