Robust Multi-View Clustering With Noisy Correspondence
Top Cited Papers
- 11 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 36 (12) , 9150-9162
Abstract
Deep multi-view clustering leverages deep neural networks to achieve promising performance, but almost all existing methods implicitly assume that all views are aligned correctly. This assumption is unrealistic in many real-world scenarios, where noise, occlusion, or sensor differences can inevitably cause misaligned data. Based on this observation, we reveal and study a practical but understudied problem in multi-view clustering (MVC), i.e., noisy correspondence (NC). Considering this problem, we argue that the main challenge is to prevent the model from overfiting NC. To this end, we propose a novel Robust Multi-view Clustering with Noisy Correspondence (RMCNC) method, which alleviates the influence of the misaligned pairs from multi-view data. To be specific, we first compute a united probability with all positive pairs to learn cross-view alignment consistency, thereby alleviating the adverse impact of the individual false positives. To further mitigate the overfitting problem, we propose a noise-tolerance multi-view contrastive loss that avoids overemphasizing noisy data. Moreover, RMCNC is a unified framework, which can deal with both partially view-aligned and NC problems in multi-view clustering. To the best of our knowledge, it could be the first study on NC in multi-view clustering. The experimental results on eight benchmark datasets indicate our RMCNC achieves competitive performance and robustness. The code of RMCNC is released at https://github.com/sunyuan-cs/2024-TKDE-RMCNC .Keywords
Funding Information
- National Natural Science Foundation of China (U21B2040, 62176171, 62102274, 62372315)
- Sichuan Science and Technology Planning (2024NSFTD0038, 2024NSFTD0047, 24ZDZX0007, 2024NSFTD0049, 2023ZYD0143, 2024YFHZ0144, 2024YFHZ0089)
- Fundamental Research Funds for the Central Universities (CJ202303, CJ202403)
This publication has 54 references indexed in Scilit:
- Robust Object Re-identification with Coupled Noisy LabelsInternational Journal of Computer Vision, 2024
- Discrete aggregation hashing for image set classificationExpert Systems with Applications, 2023
- Cross-view Graph Matching Guided Anchor Alignment for Incomplete Multi-view ClusteringInformation Fusion, 2023
- Multi-view subspace clustering via adaptive graph learning and late fusion alignmentNeural Networks, 2023
- Auto-weighted Tensor Schatten p-Norm for Robust Multi-view Graph ClusteringPattern Recognition, 2022
- Momentum Contrast for Unsupervised Visual Representation Learning2020
- AE2-Nets: Autoencoder in Autoencoder Networks2019
- Deep Supervised Cross-Modal Retrieval2019
- Latent Multi-view Subspace Clustering2017
- A Bayesian Hierarchical Model for Learning Natural Scene Categories2005