In-kernel integration of operating system and infiniband functions for high performance computing clusters: a DSM example
Authors:
- 1 August 2005
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 16 (9) , 830-840
Abstract
The infiniband (IB) system area network (SAN) enables applications to access hardware directly from user level, reducing the overhead of user-kernel crossings during data transfer. However, distributed applications that exhibit close coupling between network and OS services may benefit from accessing IB from the kernel through IB's native verbs interface, which permits tight integration of these services. We assess this approach using a sequential-consistency distributed shared memory (DSM) system as an example. We first develop primitives that abstract the low-level communication and kernel details, and efficiently serve the application's communication, memory, and scheduling needs. Next, we combine the primitives to form a kernel DSM protocol. The approach is evaluated using our full-fledged Linux kernel DSM implementation over infiniband. We show that overheads are reduced substantially, and overall application performance is improved in terms of both absolute execution time and scalability relative to an entirely user level implementation.Keywords
This publication has 14 references indexed in Scilit:
- Lazy Release Consistency for Software Distributed Shared Memory2005
- Home-based SVM protocols for SMP clusters: Design and performance2002
- The SPLASH-2 programs: characterization and methodological considerations2002
- Transparent adaptation of sharing granularity in multiview-based DSM systems2002
- Experiences with VI communication for database storageACM SIGARCH Computer Architecture News, 2002
- Using network interface support to avoid asynchronous protocol processing in shared virtual memory systemsACM SIGARCH Computer Architecture News, 1999
- Fast messages: efficient, portable communication for workstation clusters and MPPsIEEE Concurrency, 1997
- SoftFLASH1996
- The Nas Parallel BenchmarksThe International Journal of Supercomputing Applications, 1991
- Memory coherence in shared virtual memory systemsACM Transactions on Computer Systems, 1989