Great Models Think Alike: Improving Model Reliability via Inter-Model Latent Agreement
Authors:
- Ailin DengADAilin Deng
- Miao XiongMXMiao Xiong
- Bryan HooiBHBryan Hooi
Preprint
Abstract
Reliable application of machine learning is of primary importance to the practical deployment of deep learning methods. A fundamental challenge is that models are often unreliable due to overconfidence. In this paper, we estimate a model's reliability by measuring \emph{the agreement between its latent space, and the latent space of a foundation model}. However, it is challenging to measure the agreement between two different latent spaces due to their incoherence, \eg, arbitrary rotations and different dimensionality. To overcome this incoherence issue, we design a \emph{neighborhood agreement measure} between latent spaces and find that this agreement is surprisingly well-correlated with the reliability of a model's predictions. Further, we show that fusing neighborhood agreement into a model's predictive confidence in a post-hoc way significantly improves its reliability. Theoretical analysis and extensive experiments on failure detection across various datasets verify the effectiveness of our method on both in-distribution and out-of-distribution settings.Keywords
All Related Versions
This publication has 2 references indexed in Scilit:
- Probabilistic Forecasts, Calibration and SharpnessJournal of the Royal Statistical Society Series B: Statistical Methodology, 2007
- Cumulated gain-based evaluation of IR techniquesACM Transactions on Information Systems, 2002