Big Data Quality: A Quality Dimensions Evaluation
Authors:
- Ikbal TalebITIkbal Taleb
- Hadeel T. El KassabiHKHadeel T. El Kassabi
- Mohamed Adel SerhaniMSMohamed Adel Serhani
- Rachida DssouliRDRachida Dssouli
- Chafik BouhaddiouiCBChafik Bouhaddioui
- 1 July 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 759-765
Abstract
Data is the most valuable asset companies are proud of. When its quality degrades, the consequences are unpredictable, can lead to complete wrong insights. In Big Data context, evaluating the data quality is challenging, must be done prior to any Big data analytics by providing some data quality confidence. Given the huge data size, its fast generation, it requires mechanisms, strategies to evaluate, assess data quality in a fast, efficient way. However, checking the Quality of Big Data is a very costly process if it is applied on the entire data. In this paper, we propose an efficient data quality evaluation scheme by applying sampling strategies on Big data sets. The Sampling will reduce the data size to a representative population samples for fast quality evaluation. The evaluation targeted some data quality dimensions like completeness, consistency. The experimentations have been conducted on Sleep disorder's data set by applying Big data bootstrap sampling techniques. The results showed that the mean quality score of samples is representative for the original data, illustrate the importance of sampling to reduce computing costs when Big data quality evaluation is concerned. We applied the Quality results generated as quality proposals on the original data to increase its quality.Keywords
This publication has 19 references indexed in Scilit:
- A Survey on Data Quality: Classifying Poor Data2015
- Evaluating the Quality of Social Media Data in Big Data ArchitectureIEEE Access, 2015
- Big Data Pre-processing: A Quality Framework2015
- An Empirical Study on Quality Issues of Production Big Data Platform2015
- Seven V's of Big Data understanding Big Data to extract value2014
- Process-Driven Data Quality Management -- An Application of the Combined Conceptual Life Cycle Model2014
- A Data Quality in Use Model for Big DataLecture Notes in Computer Science, 2014
- Elastic algorithms for guaranteeing quality monotonicity in big data mining2013
- Survey on data quality2012
- Data quality: A survey of data quality dimensions2012