Towards a Semantic Extract-Transform-Load (ETL) Framework for Big Data Integration
Author:
- Srividya K. BansalSBSrividya K. Bansal
- 1 June 2014
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 23797703,p. 522-529
Abstract
Big Data has become the new ubiquitous term used to describe massive collection of datasets that are difficult to process using traditional database and software techniques. Most of this data is inaccessible to users, as we need technology and tools to find, transform, analyze, and visualize data in order to make it consumable for decision-making. One aspect of Big Data research is dealing with the Variety of data that includes various formats such as structured, numeric, unstructured text data, email, video, audio, stock ticker, etc. Managing, merging, and governing a variety of data is the focus of this paper. This paper proposes a semantic Extract-Transform-Load (ETL) framework that uses semantic technologies to integrate and publish data from multiple sources as open linked data. This includes - creation of a semantic data model to provide a basis for integration and understanding of knowledge from multiple sources, creation of a distributed Web of data using Resource Description Framework (RDF) as the graph data model, extraction of useful knowledge and information from the combined data using SPARQL as the semantic query language.Keywords
This publication has 18 references indexed in Scilit:
- Techniques for Graph Analytics on Big Data2013
- Data Abstraction and Visualisation in Next Step: Experiences from a Government Services Delivery Trial2013
- Revealing the Causes of Dynamic Change in Protein-Protein Interaction Network2013
- Online Association Rule Mining over Fast Data2013
- Learning Classifiers from Chains of Multiple Interlinked RDF Data Stores2013
- Milieu: Lightweight and Configurable Big Data Provenance for Science2013
- A semantic approach to ETL technologiesData & Knowledge Engineering, 2011
- QoX-driven ETL design2009
- Ontology-Based Conceptual Design of ETL Processes for Both Structured and Semi-Structured DataInternational Journal on Semantic Web and Information Systems, 2007
- A UML Based Approach for Modeling ETL Processes in Data Warehouses2003