Automated Machine Learning in the Era of Generative AI: A Comprehensive State-of-the-Art Review
- 1 January 2026
- journal article
- Published by Lviv Polytechnic National University in Mathematical Modeling and Computing
- Vol. 13 (3) , 1013-1023
Abstract
This paper provides a comprehensive review of the state-of-the-art in Automated Machine Learning (AutoML) and Symbolic Regression (SR), focusing on the integration of Large Language Models (LLMs) and intelligent agents. We systematically analyze classical AutoML frameworks, End-to-End multi-agent systems, data-centric AI approaches, and hybrid methods combining LLMs with Bayesian Optimization. Each approach is evaluated based on its autonomy, computational efficiency, reliability, and suitability for highly regulated domains. The study highlights that while LLM agents offer high flexibility, they suffer from hallucinations and deployment complexities. Conversely, data-centric and hybrid methods provide better reliability and mathematical convergence. We conclude that no single universal solution exists and propose a pragmatic hybrid modular pipeline: utilizing data-centric agents for semantic data preparation, followed by robust classical or hybrid optimizers for mathematically guaranteed model selection.Keywords
This publication has 18 references indexed in Scilit:
- LLM-Meta-SR: In-Context Learning for Evolving Selection Operators in Symbolic RegressionIEEE Transactions on Evolutionary Computation, 2026
- A survey on large language models driven meta-optimizers for automated intelligent optimizationArtificial Intelligence Review, 2026
- A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to SolvingIEEE Computational Intelligence Magazine, 2026
- A Comparison of Recent Algorithms for Symbolic Regression to Genetic Programming2025
- Using Bayesian Optimization to Improve Hyperparameter Search in TPOT2024
- AutoFeat: Transitive Feature Discovery over Join Paths2024
- Automated machine learning: past, present and futureArtificial Intelligence Review, 2024
- SymFormer: End-to-End Symbolic Regression Using Transformer-Based ArchitectureIEEE Access, 2024
- Interpretable scientific discovery with symbolic regression: a reviewArtificial Intelligence Review, 2024
- Revisiting OPRO: The Limitations of Small-Scale LLMs as Optimizers2024