The Potential of AutoML for Recommender Systems
Authors:
Preprint
Abstract
Automated Machine Learning (AutoML) has greatly advanced applications of Machine Learning (ML) including model compression, machine translation, and computer vision. Recommender Systems (RecSys) can be seen as an application of ML. Yet, AutoML has found little attention in the RecSys community; nor has RecSys found notable attention in the AutoML community. Only few and relatively simple Automated Recommender Systems (AutoRecSys) libraries exist that adopt AutoML techniques. However, these libraries are based on student projects and do not offer the features and thorough development of AutoML libraries. We set out to determine how AutoML libraries perform in the scenario of an inexperienced user who wants to implement a recommender system. We compared the predictive performance of 60 AutoML, AutoRecSys, ML, and RecSys algorithms from 15 libraries, including a mean predictor baseline, on 14 explicit feedback RecSys datasets. To simulate the perspective of an inexperienced user, the algorithms were evaluated with default hyperparameters. We found that AutoML and AutoRecSys libraries performed best. AutoML libraries performed best for six of the 14 datasets (43%), but it was not always the same AutoML library performing best. The single-best library was the AutoRecSys library Auto-Surprise, which performed best on five datasets (36%). On three datasets (21%), AutoML libraries performed poorly, and RecSys libraries with default parameters performed best. Although, while obtaining 50% of all placements in the top five per dataset, RecSys algorithms fall behind AutoML on average. ML algorithms generally performed the worst.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- EnPSO: An AutoML Technique for Generating Ensemble Recommender SystemArabian Journal for Science and Engineering, 2021
- Auto-Pytorch: Multi-Fidelity MetaLearning for Efficient and Robust AutoDLIEEE Transactions on Pattern Analysis and Machine Intelligence, 2021
- Can AutoML outperform humans? An evaluation on popular OpenML datasets using AutoML Benchmark2020
- Surprise: A Python library for recommender systemsJournal of Open Source Software, 2020
- Implementing AutoML in Educational Data Mining for Prediction TasksApplied Sciences, 2019
- Scaling tree-based automated machine learning to biomedical big data with a feature set selectorBioinformatics, 2019
- GAMA: Genetic Automated Machine learning AssistantJournal of Open Source Software, 2019
- Hyperopt: a Python library for model selection and hyperparameter optimizationComputational Science & Discovery, 2015