Semi-Supervised Multi-View Multi-Label Learning with View-Specific Transformer and Enhanced Pseudo-Label
Open Access
- 11 April 2025
- journal article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (17) , 18430-18438
Abstract
Multi-view multi-label learning has become a research focus for describing objects with rich expressions and annotations. However, real-world data often contains numerous unlabeled instances, due to the high cost and technical limitations of manual labeling. This crucial problem involves three main challenges: i) How to extract advanced semantics from available views? ii) How to build a refined classification framework with limited labeled space? iii) How to provide more high-quality supervisory information? To address these problems, we propose a Semi-Supervised Multi-View Multi-Label Learning Method with View-Specific Transformer and Enhanced Pseudo-Label named SMVTEP. Specifically, Generative Adversarial Networks are employed to extract informative shared and specific representations and their consistency and distinctiveness are ensured through the adversarial mechanism and information theory based contrastive learning. Then we build specific classifiers for each extracted feature and apply instance-level manifold constraints to reduce bias across classifiers. Moreover, we design a transformer-style fusion approach that simultaneously captures the imbalance of expressive power among views, mapping effects on specific labels, and label dependencies by incorporating confidence scores and category semantics into the self-attention mechanism. Furthermore, after using Mixup for data augmentation, category-enhanced pseudo-labels are leveraged to improve the reliability of additional annotations by aligning the label distribution of unlabeled samples with the true distribution. Finally, extensive experimental results validate the effectiveness of SMVTEP against state-of-the-art methods.Keywords
This publication has 32 references indexed in Scilit:
- A Deep Model for Partial Multi-label Image Classification with Curriculum-based DisambiguationMachine Intelligence Research, 2024
- Vision-based multi-label detection framework for capturing occupant action and clothing information using large-scale datasetBuilding and Environment, 2024
- A new multi-view multi-label model with privileged information learningInformation Sciences, 2023
- Multi-view multi-label learning with double orders manifold preservingApplied Intelligence, 2022
- Learning view-specific labels and label-feature dependence maximization for multi-view multi-label classificationApplied Soft Computing, 2022
- Multi granularity based label propagation with active learning for semi-supervised classificationExpert Systems with Applications, 2022
- Modeling of Tumor Growth with Input from Patient-Specific Metabolomic DataAnnals of Biomedical Engineering, 2022
- Two-step multi-view and multi-label learning with missing label via subspace learningApplied Soft Computing, 2021
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary2002