Fast Compression of Large Semantic Web Data Using X10
- 30 October 2015
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 27 (9) , 2603-2617
Abstract
The Semantic Web comprises enormous volumes of semi-structured data elements. For interoperability, these elements are represented by long strings. Such representations are not efficient for the purposes of applications that perform computations over large volumes of such information. A common approach to alleviate this problem is through the use of compression methods that produce more compact representations of the data. The use of dictionary encoding is particularly prevalent in Semantic Web database systems for this purpose. However, centralized implementations present performance bottlenecks, giving rise to the need for scalable, efficient distributed encoding schemes. In this paper, we propose an efficient algorithm for fast encoding large Semantic Web data. Specially, we present the detailed implementation of our approach based on the state-of-art asynchronous partitioned global address space (APGAS) parallel programming model. We evaluate performance on a cluster of up to 384 cores and datasets of up to 11 billion triples (1.9 TB). Compared to the state-of-art approach, we demonstrate a speed-up of $2.6 - 7.4\times$ and excellent scalability. In the meantime, these results also illustrate the significant potential of the APGAS model for efficient implementation of dictionary encoding and contributes to the engineering of more efficient, larger scale Semantic Web applications.
Keywords
Funding Information
- DFG
- DIAMOND (KR 4381/1-1)
- HAEC (CRC 912)
This publication has 33 references indexed in Scilit:
- Scalable RDF data compression with MapReduceConcurrency and Computation: Practice and Experience, 2012
- PLDSACM Transactions on Architecture and Code Optimization, 2012
- CassandraACM SIGOPS Operating Systems Review, 2010
- Linked Data - The Story So FarInternational Journal on Semantic Web and Information Systems, 2009
- MapReduceCommunications of the ACM, 2008
- Parallel Programmability and the Chapel LanguageThe International Journal of High Performance Computing Applications, 2007
- LUBM: A benchmark for OWL knowledge base systemsJournal of Web Semantics, 2005
- UniProt: the Universal Protein knowledgebaseNucleic Acids Research, 2004
- Co-array Fortran for parallel programmingACM SIGPLAN Fortran Forum, 1998
- OpenMP: an industry standard API for shared-memory programmingIEEE Computational Science and Engineering, 1998