Lung Cancer Survival Prediction using Ensemble Data Mining on Seer Data
Open Access
- 1 January 2012
- journal article
- Published by Wiley in Scientific Programming
- Vol. 20 (1) , 29-42
Abstract
We analyze the lung cancer data available from the SEER program with the aim of developing accurate survival prediction models for lung cancer. Carefully designed preprocessing steps resulted in removal/modification/splitting of several attributes, and 2 of the 11 derived attributes were found to have significant predictive power. Several supervised classification methods were used on the preprocessed data along with various data mining optimizations and validations. In our experiments, ensemble voting of five decision tree based classifiers and meta-classifiers was found to result in the best prediction performance in terms of accuracy and area under the ROC curve. We have developed an on-line lung cancer outcome calculator for estimating the risk of mortality after 6 months, 9 months, 1 year, 2 year and 5 years of diagnosis, for which a smaller non-redundant subset of 13 attributes was carefully selected using attribute selection techniques, while trying to retain the predictive power of the original set of attributes. Further, ensemble voting models were also created for predicting conditional survival outcome for lung cancer (estimating risk of mortality after 5 years of diagnosis, given that the patient has already survived for a period of time), and included in the calculator. The on-line lung cancer outcome calculator developed as a result of this study is available at http://info.eecs.northwestern.edu:8080/LungCancerOutcomeCalculator/.Keywords
Funding Information
- National Science Foundation (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- National Science Foundation (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- National Science Foundation (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- National Science Foundation (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- National Science Foundation (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- National Science Foundation (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- National Science Foundation (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- National Science Foundation (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- National Science Foundation (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- National Science Foundation (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- National Science Foundation (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- National Science Foundation (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- U.S. Department of Energy (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- U.S. Department of Energy (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- U.S. Department of Energy (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- U.S. Department of Energy (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
- U.S. Department of Energy (CCF-0621443, OCI-0724599, CCF-0833131, CNS-0830927, IIS-0905205, OCI-0956311, CCF-0938000, CCF-1043085, CCF-1029, OCI-1144061, CNS-0551639, IIS-0536994, DE-FC02-07ER25808, DE-FG02-08ER25848, DE-SC0001283, DE-SC0005309, DE-SC0005340)
This publication has 15 references indexed in Scilit:
- Data Mining: Practical Machine Learning Tools and Techniques2011
- Developing Prognostic Systems of Cancer Patients by Ensemble ClusteringBioMed Research International, 2009
- One Hundred Years After “Carcinoid”: Epidemiology of and Prognostic Factors for Neuroendocrine Tumors in 35,825 Cases in the United StatesJournal of Clinical Oncology, 2008
- Conditional survival in ovarian cancer: Results from the SEER dataset 1988–2001Gynecologic Oncology, 2008
- Conditional survival in gastric cancer: a SEER database analysisGastric Cancer, 2007
- Predicting breast cancer survivability: a comparison of three data mining methodsArtificial Intelligence in Medicine, 2005
- Medical diagnosis with c4.5 rule preceded by artificial neural network ensembleIEEE Transactions on Information Technology in Biomedicine, 2003
- Artificial Neural Networks Applied to Survival Prediction in Breast CancerOncology, 1999
- Combining classifiers: A theoretical frameworkPattern Analysis and Applications, 1998
- The random subspace method for constructing decision forestsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998