Design and Implementation of an Asymmetric Block-Based Parallel File System
- 11 January 2013
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 63 (7) , 1723-1735
Abstract
Existing block-based parallel file systems, which are deployed in the storage area network (SAN), blend metadata with data in underlying disks. Unfortunately, such symmetric architecture is prone to system-level failures, as metadata on shared disks can be damaged by a malfunctioning client. In this paper, we present an asymmetric block-based parallel file system, Redbud, which isolates the metadata storage in the metadata server (MDS) access domain. Although centralized metadata management can effectively improve the reliability of the system, it faces some challenges in providing high performance and availability. Towards this end, we introduce an embedded directory mechanism to explore the disk bandwidth of the metadata storage; we also introduces adaptive layout operations to deliver high I/O throughput for various file access pattern. Besides, by taking the MDS's load into consideration, we propose an adaptive timeout algorithm to make the MDS failure detection adaptive to the evolving workloads, improving the system availability. Measurements of a wide range of workloads demonstrate the benefit of our design and that Redbud gains good scalability.Keywords
This publication has 8 references indexed in Scilit:
- Tolerating hardware device failures in software2009
- A Novel network request scheduler for a large scale storage systemComputer Science - Research and Development, 2009
- On the design of a new Linux readahead frameworkACM SIGOPS Operating Systems Review, 2008
- Improving the reliability of commodity operating systemsACM Transactions on Computer Systems, 2005
- The Google file system2003
- A cost-effective, high-bandwidth storage architecture1998
- Performance of Firefly RPCACM SIGOPS Operating Systems Review, 1989
- Implementing remote procedure callsACM Transactions on Computer Systems, 1984