Conformal prediction for multi-label learning: a review of methods and guarantees
Open Access
- 27 August 2026
- journal article
- review article
- Published by The Royal Society in Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
- Vol. 384 (2327)
Abstract
Multi-label learning (MLL) is a machine learning paradigm that aims to predict a set of labels for each instance, rather than a single class. Such tasks arise in a wide range of real-world applications and pose significant challenges, including an exponentially large output space, dependence among labels and often severe label imbalance. These challenges amplify predictive uncertainty, making reliable uncertainty quantification essential. Conformal prediction (CP) is an attractive answer: it converts model outputs into prediction regions with distribution-free, finite-sample guarantees under the sole assumption of data exchangeability. Several adaptations of CP to the multi-label setting have been proposed. Yet these vary widely in scoring constructions, output types and targeted guarantees. This review consolidates the landscape of CP adaptations for MLL. It places existing approaches under a unified framework, examining the types of outputs and guarantees they provide, where label dependencies are incorporated, and how inference cost scales with the number of labels. It provides an in-depth analysis of all approaches using common notation, identifying their key characteristics along with their practical implications and assessing their strengths and limitations. Finally, it compares approaches side-by-side, highlighting trade-offs among guarantee types, precision of regions, compactness of outputs and scalability. This article is part of the theme issue ‘Advancing uncertainty quantification in AI systems’.Keywords
This publication has 11 references indexed in Scilit:
- Copula-based conformal prediction for prioritized heterogeneous multi-task learningPattern Recognition, 2025
- Well-calibrated confidence measures for multi-label text classification with a large number of labelsPattern Recognition, 2022
- Algorithmic Learning in a Random World2022
- Using Conformal Prediction for Multi-label Document Classification in e-Mail Support Systems2019
- Multi-target prediction: a unifying view on problems and methodsData Mining and Knowledge Discovery, 2018
- Binary Relevance Multi-label Conformal Predictor2016
- A Comparison of Three Implementations of Multi-Label Conformal Prediction2015
- Random k-Labelsets: An Ensemble Method for Multilabel Classification2007
- Transductive Confidence Machines for Pattern Recognition2002
- Inductive Confidence Machines for Regression2002