Heterogeneous Multi-Agent Reinforcement Learning with Attention for Cooperative and Scalable Feature Transformation
Open Access
- 20 April 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2010-2019
Abstract
Feature transformation enhances downstream task performance by generating informative features through mathematical feature crossing. Despite the advancements in deep learning, feature transformation remains essential, particularly for structured data, where deep models often struggle to capture complex feature interactions effectively. Prior literature on automated feature transformation has achieved notable success but often relies on heuristics or exhaustive searches, leading to inefficient and time-consuming processes. Recent works employ reinforcement learning (RL) to enhance traditional approaches through a more effective trial-and-error way. However, two key limitations remain: 1) Dynamic feature expansion during the transformation process, which introduces instability and increases the time complexity of the learning procedure for RL agents; 2) Insufficient cooperation and communication between agents, which results in suboptimal feature crossing operations and degraded model performance. To address them, we propose a novel heterogeneous multi-agent RL framework to enable cooperative and scalable feature transformation. The framework comprises three heterogeneous agents, grouped into two types, each designed to select essential features and operations for feature crossing. To enhance communication among these agents, we implement a shared critic mechanism that facilitates information exchange during the feature transformation process. This collaboration enables the agents to learn more intelligent and effective transformation policies. To handle the dynamically expanding feature space, we tailor multi-head attention-based feature agents to select suitable features for feature crossing. This design facilitates scalable decision-making and effective candidate selection based on comprehensive global feature space information. Additionally, we introduce a state encoding technique during the optimization process to stabilize and enhance the learning dynamics of the RL agents, resulting in more robust and reliable transformation policies. Finally, we conduct extensive experiments to validate the effectiveness, efficiency, robustness, and interpretability of our model. Our code and dataset are publicly available on GitHub.Keywords
All Related Versions
Funding Information
- National Science Foundation (2550105)
- National Science Foundation (2550106)
- National Science Foundation (2242812)
This publication has 22 references indexed in Scilit:
- Beyond Discrete Selection: Continuous Embedding Space Optimization for Generative Feature Selection2023
- Self-optimizing Feature Generation via Categorical Hashing Representation and Hierarchical Reinforcement Crossing2023
- Traceable Automatic Feature Transformation via Cascading Actor-Critic Agents2023
- The autofeat Python Library for Automated Feature Engineering and Selection2020
- Neural Feature Search: A Neural Architecture for Automated Feature Engineering2019
- Cognito: Automated Feature Engineering for Supervised Learning2016
- Genetic programming for feature construction and selection in classification on high-dimensional dataMemetic Computing, 2015
- Deep feature synthesis: Towards automating data science endeavors2015
- Ridge Regression: Biased Estimation for Nonorthogonal ProblemsTechnometrics, 1970
- Nearest neighbor pattern classificationIEEE Transactions on Information Theory, 1967