Profiler
Authors:
Open Access
- 21 May 2012
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 547-554
Abstract
Data quality issues such as missing, erroneous, extreme and duplicate values undermine analysis and are time-consuming to find and fix. Automated methods can help identify anomalies, but determining what constitutes an error is context-dependent and so requires human judgment. While visualization tools can facilitate this process, analysts must often manually construct the necessary views, requiring significant expertise. We present Profiler, a visual analysis tool for assessing quality issues in tabular data. Profiler applies data mining methods to automatically flag problematic data and suggests coordinated summary visualizations for assessing the data in context. The system contributes novel methods for integrated statistical and visual analysis, automatic view suggestion, and scalable visual summaries that support real-time interaction with millions of data points. We present Profiler's architecture --- including modular components for custom data types, anomaly detection routines and summary visualizations --- and describe its application to motion picture, natural disaster and water quality data sets.Keywords
Funding Information
- Division of Computing and Communication Foundations (CCF-0964173)
This publication has 25 references indexed in Scilit:
- Anomaly detectionACM Computing Surveys, 2009
- A Survey of Outlier Detection MethodologiesArtificial Intelligence Review, 2004
- GGobi: evolving from XGobi into an extensible framework for interactive data visualizationComputational Statistics & Data Analysis, 2003
- Polaris: a system for query, analysis, and visualization of multidimensional relational databasesIEEE Transactions on Visualization and Computer Graphics, 2002
- Information visualization and visual data miningIEEE Transactions on Visualization and Computer Graphics, 2002
- HD-Eye: visual mining of high-dimensional dataIEEE Computer Graphics and Applications, 1999
- Tane: An Efficient Algorithm for Discovering Functional and Approximate DependenciesThe Computer Journal, 1999
- Outliers, level shifts, and variance changes in time seriesJournal of Forecasting, 1988
- Scatterplot Matrix Techniques for Large NJournal of the American Statistical Association, 1987
- Modeling by shortest data descriptionAutomatica, 1978