Discovering influential variables: A method of partitions
Authors:
- Herman ChernoffHCHerman Chernoff
- Shaw-Hwa LoSLShaw-Hwa Lo
- Tian ZhengTZTian Zheng
Open Access
- 1 December 2009
- journal article
- Published by Institute of Mathematical Statistics in The Annals of Applied Statistics
- Vol. 3 (4) , 1335-1369
Abstract
A trend in all scientific disciplines, based on advances in technology, is the increasing availability of high dimensional data in which are buried important information. A current urgent challenge to statisticians is to develop effective methods of finding the useful information from the vast amounts of messy and noisy data available, most of which are noninformative. This paper presents a general computer intensive approach, based on a method pioneered by Lo and Zheng for detecting which, of many potential explanatory variables, have an influence on a dependent variable Y. This approach is suited to detect influential variables, where causal effects depend on the confluence of values of several variables. It has the advantage of avoiding a difficult direct analysis, involving possibly thousands of variables, by dealing with many randomly selected small subsets from which smaller subsets are selected, guided by a measure of influence I. The main objective is to discover the influential variables, rather than to measure their effects. Once they are detected, the problem of dealing with a much smaller group of influential variables should be vulnerable to appropriate analysis. In a sense, we are confining our attention to locating a few needles in a haystack.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- Discovering interactions among BRCA1 and other candidate genes associated with sporadic breast cancerProceedings of the National Academy of Sciences, 2008
- Autoimmune diseases in a Danish cohort of 4,866 carriers of constitutional structural chromosomal rearrangementsArthritis & Rheumatism, 2007
- Backward Genotype-Trait Association (BGTA)-Based Dissection of Complex Traits in Case-Control DesignsHuman Heredity, 2006
- Multilocus Linkage Analysis of Affected Sib PairsHuman Heredity, 2005
- A genome‐wide scan for juvenile rheumatoid arthritis in affected sibpair families provides evidence of linkageArthritis & Rheumatism, 2004
- Dense genome‐wide linkage analysis of rheumatoid arthritis, including covariatesArthritis & Rheumatism, 2004
- Whole-Genome Scan, in a Complex Disease, Using 11,245 Single-Nucleotide Polymorphisms: Comparison with MicrosatellitesThe American Journal of Human Genetics, 2004
- Backward Haplotype Transmission Association (BHTA) Algorithm – A Fast Multiple-Marker Screening MethodHuman Heredity, 2002
- The Corn–Guano ModelMetroeconomica, 2001
- Association between EcoRI fragment-length polymorphism of the immunoglobulin lambda variable 8 (IGLV8) gene family with rheumatoid arthritis and systemic lupus erythematosusBrazilian Journal of Medical and Biological Research, 2001