A Framework for Controllable Multi-objective Learning with Annealed Stein Variational Hypernetworks

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Pareto Set Learning (PSL) is an efficient approach for approximating the complete set of optimal solutions in Multi-objective Learning (MOL). By learning a set of solutions that map to a dense Pareto front in objective space, PSL aims to recover the underlying Pareto set. However, most existing methods focus heavily on convergence toward optimality, often neglecting solution diversity. To explicitly balance convergence and diversity, we propose Stein Variational Hypernetwork for MOL (SVH-MOL), a novel framework that incorporates Stein variational updates into PSL. SVH-MOL updates solutions via two complementary components: (i) a driving term that guides particles toward the Pareto set, and (ii) a repulsive term that encourages diversity among solutions. These two terms inherently compete, making stable and effective learning challenging. To address this issue, we introduce an annealing schedule that adaptively controls the relative strength of each term during training. Extensive experiments on synthetic multi-objective benchmarks and real-world multi-task learning problems demonstrate that SVH-MOL achieves a better tradeoff between convergence and diversity than existing methods. Our implementation is released at https://github.com/nguyenduc810/SVH-MOL-official .
Funding Information
  • Vingroup Innovation Foundation (VinIF.2024.DA113)

This publication has 20 references indexed in Scilit: