Data transformation and migration in polystores
- 1 December 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
Ever increasing data size and new requirements in data processing has fostered the development of many new database systems. The result is that many data-intensive applications are underpinned by different engines. To enable data mobility there is a need to transfer data between systems easily and efficiently. We analyze the state-of-the-art of data migration and outline research opportunities for a rapid data transfer. Our experiments explore data migration between a diverse set of databases, including PostgreSQL, SciDB, S-Store and Accumulo. Each of the systems excels at specific application requirements, such as transactional processing, numerical computation, streaming data, and large scale text processing. Providing an efficient data migration tool is essential to take advantage of superior processing from that specialized databases. Our goal is to build such a data migration framework that will take advantage of recent advancement in hardware and software.Keywords
This publication has 9 references indexed in Scilit:
- PipeGen2016
- Live datastore transformation for optimizing big data applications in cloud environments2015
- MISO2014
- Optimizing Database Load and Extract for Big Data EraLecture Notes in Computer Science, 2014
- Instant loading for main memory databasesProceedings of the VLDB Endowment, 2013
- Split query processing in polybase2013
- HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots2011
- HadoopDBProceedings of the VLDB Endowment, 2009
- BigtableACM Transactions on Computer Systems, 2008