EaSFE: Scalable and Efficient Feature Engineering for Boosting Machine Learning Performance
Open Access
- 3 June 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Knowledge Discovery From Data
- Vol. 20 (5) , 1-19
Abstract
Feature engineering plays a critical role in machine learning (ML), but existing methods often struggle with high computational cost and limited scalability when applied to large-scale and sparse datasets. In this article, we propose EaSFE, an efficient and scalable feature engineering framework that unifies feature generation, filtering, and evaluation in an end-to-end manner. EaSFE is designed to efficiently construct and select informative features while explicitly considering computational and memory constraints. To achieve scalability, EaSFE incorporates parallel and distributed execution mechanisms, as well as a chunk-based data processing strategy that enables memory-efficient feature engineering on large datasets. In addition, EaSFE adopts tailored storage and execution strategies to handle high-dimensional sparse data effectively. Extensive experiments on multiple real-world datasets demonstrate that EaSFE consistently improves predictive performance (e.g., 5% accuracy improvement in poker ) while substantially enhancing efficiency (i.e., over 10x speedup) compared to existing feature engineering methods. In addition, EaSFE is demonstrated to scale to large and sparse datasets, successfully handling datasets with over 119 million training instances and 54 million features.Keywords
Funding Information
- National Natural Science Foundation of China (62376099)
- Postdoctoral Fellowship Program of CPSF (GZC20251042)
- China Postdoctoral Science Foundation (2024M760785)
- Key Research Project Plan of Higher Education Institutions in Henan (26A520002)
This publication has 12 references indexed in Scilit:
- A Feature Engineering Method for Whole-Genome DNA Sequence with Nucleotide ResolutionInternational Journal of Molecular Sciences, 2025
- Modelling bank customer behaviour using feature engineering and classification techniquesResearch in International Business and Finance, 2023
- QFC: A Parallel Software Tool for Feature Construction, Based on Grammatical EvolutionAlgorithms, 2022
- Multi-label feature selection based on label correlations and feature redundancyKnowledge-Based Systems, 2022
- An enhanced black widow optimization algorithm for feature selectionKnowledge-Based Systems, 2022
- Prediction of stock price direction using a hybrid GA-XGBoost algorithm with a three-stage feature engineering processExpert Systems with Applications, 2021
- DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems2021
- Hadoop neural network for parallel and distributed feature selectionNeural Networks, 2016
- Iterative feature construction for improving inductive learning algorithmsExpert Systems with Applications, 2009
- Open MPI: A Flexible High Performance MPI2006