Exploiting Key-Value Data Stores Scalability for HPC
- 1 August 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 85-94
Abstract
BigData revolutionised the IT industry. It first interested the OLTP systems. Distributed Hash Tables replaced Traditional SQL databases as they guaranteed low response time on simple read/write requests. The second wave recast the data warehousing: map-reduce systems spread as they proved to scale linearly long-running computational workloads on commodity servers. The focus now is on real-time analytics. Being able to analyse massive quantities of data in a short time enables multiple HPC applications and interactive analysis and visualization. In this paper, we study the performance of a system that employs the DHT architecture to achieve fast in local analysis on indexed data. We observed that the number of keys, nodes, and the hardware characteristics strongly influence the actual scalability of the system. Therefore, we developed a mathematical model that allows finding the right system configuration to meet desired performance for each kind of query type. We also show how our model can be used to find the right architecture for each distributed application.Keywords
This publication has 8 references indexed in Scilit:
- Knights Landing: Second-Generation Intel Xeon Phi ProductIEEE Micro, 2016
- D8-tree2016
- BIGNASim: a NoSQL database structure and analysis portal for nucleic acids simulation dataNucleic Acids Research, 2015
- Experiences of Using Cassandra for Molecular Dynamics Simulations2015
- Aeneas: A Tool to Enable Applications to Effectively Use Non-relational DatabasesProcedia Computer Science, 2013
- GFSCommunications of the ACM, 2010
- Balanced Allocations: The Heavily Loaded CaseSIAM Journal on Computing, 2006
- The Google file system2003