Interleaving multi-task neural architecture search
Authors:
Open Access
- 21 June 2026
- journal article
- research article
- Published by Springer Nature in npj Artificial Intelligence
- Vol. 2 (1)
Abstract
Multi-task neural architecture search (MTNAS), which searches for a shared architecture for multiple tasks, has been broadly investigated. In these methods, multiple tasks are learned simultaneously by minimizing the weighted sum of their losses. How to balance these losses by finding the optimal loss weights requires a lot of tuning, which is time-consuming and labor intensive. To address this problem, we propose an interleaving MTNAS framework, where no tuning of loss weights is needed. In our method, a set of tasks (e.g., A, B, C) are performed in an interleaving loop (e.g., ABCABCABC...) where each task transfers its knowledge to the next task. Each task is learned by minimizing its loss function alone, without intervening with losses of other tasks. Loss functions of individual tasks are organized into a multi-level optimization framework which enables all tasks performed end-to-end. The effectiveness of our method is demonstrated in a variety of experiments.Keywords
This publication has 45 references indexed in Scilit:
- MTL-SIMNAS: Task Similarity-Driven Neural Architecture Search for Enhanced Multi-task Learning2025
- Dual-mask: Progressively sparse multi-task architecture learningPattern Recognition, 2024
- Multi-Task Learning for Multi-Objective Evolutionary Neural Architecture Search2021
- Single-Path NAS: Designing Hardware-Efficient ConvNets in Less Than 4 Hours2020
- UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory2017
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences, 2017
- Cross-Stitch Networks for Multi-task Learning2016
- Microsoft COCO: Common Objects in Context2014
- ImageNet: A large-scale hierarchical image database2009
- Multitask Learning1998