Controlling False Discoveries During Interactive Data Exploration
Authors:
Open Access
- 9 May 2017
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 527-540
Abstract
Recent tools for interactive data exploration significantly increase the chance that users make false discoveries. They allow users to (visually) examine many hypotheses and make inference with simple interactions, and thus incur the issue commonly known in statistics as the "multiple hypothesis testing error." In this work, we propose a solution to integrate the control of multiple hypothesis testing into interactive data exploration systems. A key insight is that existing methods for controlling the false discovery rate (such as FDR) are not directly applicable to interactive data exploration. We therefore discuss a set of new control procedures that are better suited for this task and integrate them in our system, QUDE. Via extensive experiments on both real-world and synthetic data sets we demonstrate how QUDE can help experts and novice users alike to efficiently control false discoveries.Keywords
All Related Versions
Funding Information
- Defense Sciences Office, DARPA (16-43-D3M-FP-040)
- National Science Foundation (IIS-1453171)
- Air Force Office of Scientific Research (FA9550-15-1-0144)
This publication has 26 references indexed in Scilit:
- Cross-Validation2009
- α-Investing: a Procedure for Sequential Control of Expected False DiscoveriesJournal of the Royal Statistical Society Series B: Statistical Methodology, 2008
- Why Most Published Research Findings Are FalsePLOS Medicine, 2005
- The control of the false discovery rate in multiple testing under dependencyThe Annals of Statistics, 2001
- Bayesian perspectives on multiple comparisonsJournal of Statistical Planning and Inference, 1999
- Multiple Hypothesis TestingAnnual Review of Psychology, 1995
- A sharper Bonferroni procedure for multiple tests of significanceBiometrika, 1988
- An improved Bonferroni procedure for multiple tests of significanceBiometrika, 1986
- Efficiency of Human Visual Signal DiscriminationScience, 1981
- Rectangular Confidence Regions for the Means of Multivariate Normal DistributionsJournal of the American Statistical Association, 1967