Erin LeDell
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Erin LeDell has a total of 4 co-authors. Here are the top co-authors by number of shared publications:
Mark J. van der Laan
University of California Berkeley, Division of Biostatistics, Berkeley, CA 94720, USA
1 shared publication- Mark J. van der Laan
University of California at Berkeley Berkeley California USA
1 shared publication - Maya L. Petersen
University of California Berkeley
1 shared publication Maya Petersen
University of California Berkeley, Division of Biostatistics, Berkeley, CA 94720, USA
1 shared publication
Latest Publications
- Open Access
- 8 December 2015
- Published by Walter de Gruyter GmbH in The International Journal of Biostatistics
- Vol. 12 (1) , 203-218
AUC-Maximizing Ensembles through Metalearning
Area Under the ROC Curve (AUC) is often used to measure the performance of an estimator in binary classification problems. An AUC-maximizing classifier can have significant advantages in cases where ranking correctness is valued or if the outcome is rare. In a Super Learner ensemble, maximization of the AUC can be achieved by the use of an AUC-maximining metalearning algorithm. We discuss an implementation of an AUC-maximization technique that is formulated as a nonlinear optimization problem. We also evaluate the effectiveness of a large number of different nonlinear optimization algorithms to maximize the cross-validated AUC of the ensemble fit. The results provide evidence that AUC-maximizing metalearners can, and often do, out-perform non-AUC-maximizing metalearning methods, with respect to ensemble AUC. The results also demonstrate that as the level of imbalance in the training data increases, the Super Learner ensemble outperforms the top base algorithm by a larger degree.
- 8 December 2015
- Published by Walter de Gruyter GmbH in The International Journal of Biostatistics
- Vol. 12 (1) , 203-218
Open AccessAUC-Maximizing Ensembles through Metalearning
Area Under the ROC Curve (AUC) is often used to measure the performance of an estimator in binary classification problems. An AUC-maximizing classifier can have significant advantages in cases where ranking correctness is valued or if the outcome is rare. In a Super Learner ensemble, maximization of the AUC can be achieved by the use of an AUC-maximining metalearning algorithm. We discuss an implementation of an AUC-maximization technique that is formulated as a nonlinear optimization problem. We also evaluate the effectiveness of a large number of different nonlinear optimization algorithms to maximize the cross-validated AUC of the ensemble fit. The results provide evidence that AUC-maximizing metalearners can, and often do, out-perform non-AUC-maximizing metalearning methods, with respect to ensemble AUC. The results also demonstrate that as the level of imbalance in the training data increases, the Super Learner ensemble outperforms the top base algorithm by a larger degree.
- Open Access
- 1 January 2015
- Published by Institute of Mathematical Statistics in Electronic Journal of Statistics
- Vol. 9 (1) , 1583-1607
Computationally efficient confidence intervals for cross-validated area under the ROC curve estimates
In binary classification problems, the area under the ROC curve (AUC) is commonly used to evaluate the performance of a prediction model. Often, it is combined with cross-validation in order to assess how the results will generalize to an independent data set. In order to evaluate the quality of an estimate for cross-validated AUC, we obtain an estimate of its variance. For massive data sets, the process of generating a single performance estimate can be computationally expensive. Additionally, when using a complex prediction method, the process of cross-validating a predictive model on even a relatively small data set can still require a large amount of computation time. Thus, in many practical settings, the bootstrap is a computationally intractable approach to variance estimation. As an alternative to the bootstrap, we demonstrate a computationally efficient influence curve based approach to obtaining a variance estimate for cross-validated AUC.
- 1 January 2015
- Published by Institute of Mathematical Statistics in Electronic Journal of Statistics
- Vol. 9 (1) , 1583-1607
Open AccessComputationally efficient confidence intervals for cross-validated area under the ROC curve estimates
In binary classification problems, the area under the ROC curve (AUC) is commonly used to evaluate the performance of a prediction model. Often, it is combined with cross-validation in order to assess how the results will generalize to an independent data set. In order to evaluate the quality of an estimate for cross-validated AUC, we obtain an estimate of its variance. For massive data sets, the process of generating a single performance estimate can be computationally expensive. Additionally, when using a complex prediction method, the process of cross-validating a predictive model on even a relatively small data set can still require a large amount of computation time. Thus, in many practical settings, the bootstrap is a computationally intractable approach to variance estimation. As an alternative to the bootstrap, we demonstrate a computationally efficient influence curve based approach to obtaining a variance estimate for cross-validated AUC.