Challenges and Solutions of Real-Time Data Integration Techniques by ETL Application
- 29 December 2023
- book chapter
- Published by IGI Global Scientific Publishing
- p. 348-371
Abstract
Business organizations are trying to focus from the traditional extract-transform-load (ETL) system towards real-time implementation of the ETL process. Traditional ETL process upgrades new data to the data warehouse (DW) at predefined time intervals when the DW is in off-line mode. Modern organizations want to capture and respond to business events faster than ever. Accessing fresh data is not possible using traditional ETL. Real-time ETL can reflect fresh data on the warehouse immediately at the occurrence of an event in the operational data store. Therefore, the key tool for business trade lies in real-time enterprise DW enabled with Business Intelligence. This study provides an overview of ETL process and its evolution towards real-time ETL. This chapter will represent the real-time ETL characteristics, its technical challenges, and some popular real-time ETL implementation methods. Finally, some future directions towards real-time ETL technology are explored. Request access from your librarian to read this chapter's full text.Keywords
This publication has 34 references indexed in Scilit:
- Problems and available solutions on the stage of Extract, Transform, and Loading in near real-time data warehousing (a literature study)2015
- ELTA: New Approach in Designing Business Intelligence Solutions in Era of Big DataProcedia Technology, 2014
- Scheduling strategies for efficient ETL executionInformation Systems, 2013
- On-Demand ELT Architecture for Right-Time BIInternational Journal of Data Warehousing and Mining, 2013
- Near real-time with traditional data warehouse architectures2013
- Maintaining Internal Consistency of Report for Real-Time OLAP with Layer-Based ViewLecture Notes in Computer Science, 2011
- Near Real-Time Data Warehousing with Multi-stage Trickle and Flip2011
- A Survey of Extract–Transform–Load TechnologyInternational Journal of Data Warehousing and Mining, 2009
- Synchronization Options for Data Warehouse DesignsComputer, 2006
- Arktos: towards the modeling, design, control and execution of ETL processesInformation Systems, 2001