Empirical Guide to Use of Persistent Memory for Large-Scale In-Memory Graph Analysis
Authors:
- 1 October 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 316-320
Abstract
We investigate runtime environment characteristics and explore the challenges of conventional in-memory graph processing. This system-level analysis includes empirical results and observations, which are opposite to the existing expectations of graph application users. Specifically, since raw graph data are not the same as the in-memory graph data, processing a billion-scale graph exhausts all system resources and makes the target system unavailable due to out-of-memory at runtime.To address a lack of memory space problem for big-scale graph analysis, we configure real persistent memory devices (PMEMs) with different operation modes and system software frameworks. In this work, we introduce PMEM to a representative in-memory graph system, Ligra, and perform an in-depth analysis uncovering the performance behaviors of different PMEM-applied in-memory graph systems. Based on our observations, we modify Ligra to improve the graph processing performance with a solid level of data persistence. Our evaluation results reveal that Ligra, with our simple modification, exhibits 4.41× and 3.01× better performance than the original Ligra running on a virtual memory expansion and conventional persistent memory, respectively.Keywords
This publication has 10 references indexed in Scilit:
- ScaleMine: Scalable Parallel Frequent Subgraph Mining in a Single Large Graph2016
- One trillion edgesProceedings of the VLDB Endowment, 2015
- GraphMatProceedings of the VLDB Endowment, 2015
- Seraph2014
- Trinity2013
- LigraACM SIGPLAN Notices, 2013
- Fast and Efficient Graph Traversal Algorithm for CPUs: Maximizing Single-Node Efficiency2012
- Crawling Facebook for social network analysis purposes2011
- Massive Social Network Analysis: Mining Twitter for Social Good2010
- Parallel and distributed graph cuts by dual decomposition2010