A Machine Learning Approach to Reduce Dimensional Space in Large Datasets
Open Access
- 29 July 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 148181-148192
Abstract
Large datasets computing is a research problem as well as a huge challenge due to massive amounts of data that are mined and crunched in order to successfully analyze these massive datasets because they constitute a valuable source of information over different and cross-folded domains, and therefore it represents an irreplaceable opportunity. Hence, the increasing number of environments that use data-intensive computations need more complex calculations than the ones applied to grid-based infrastructures. In this way, this paper analyzes the most commonly used algorithms regarding to this complex problem of handling large datasets whose part of research efforts are focused on reducing dimensional space. Consequently, we present a novel machine learning method that reduces dimensional space in large datasets. This approach is carried out by developing different phases: merging all datasets as a huge one, performing the Extract, Transform and Load (ETL) process, applying the Principal Component Analysis (PCA) algorithm to machine learning techniques, and finally displaying the data results by means of dashboards. The major contribution in this paper is the development of a novel architecture divided into five phases that presents an hybrid method of machine learning for reducing dimensional space in large datasets. In order to verify the correctness of our proposal, we have presented a case study with a complex dataset, specifically an epileptic seizure recognition database. The experiments carried out are very promising since they present very encouraging results to be applied to a great number of different domains.Keywords
Funding Information
- ECLIPSEUA Spanish Ministry of Education and Science (RTI2018-094283-B-C32)
This publication has 67 references indexed in Scilit:
- Selecting the number of components in principal component analysis using cross-validation approximationsComputational Statistics & Data Analysis, 2012
- A hybrid recommender system for the selective dissemination of research resources in a Technology Transfer OfficeInformation Sciences, 2012
- Intrusion detection in computer networks by a modular ensemble of one-class classifiersInformation Fusion, 2008
- Support vector clusteringScholarpedia, 2008
- Understanding diagnostic tests 1: sensitivity, specificity and predictive valuesActa Paediatrica, 2007
- Fusion of multiple classifiers for intrusion detection in computer networksPattern Recognition Letters, 2003
- Benchmarking state-of-the-art classification algorithms for credit scoringJournal of the Operational Research Society, 2003
- Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain statePhysical Review E, 2001
- Learn++: an incremental learning algorithm for supervised neural networksIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2001
- The data warehouse and data miningCommunications of the ACM, 1996