Few-Shot Learning Based on Information Geometry
- 17 November 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 8311-8316
Abstract
Few-shot classification, as the name suggests, is a very challenging task where only a few training examples are available for each category. Extensive research has been done to solve the problem of the high cost of labeling a large number of samples. A recent research strategy is to find similarity measures between query sets and support sets for judgment. Recently, information geometry techniques have been applied in deep learning research, which explores the relationships and properties between probability distributions by treating them as geometric objects and using geometric tools. Based on information geometry, this paper characterizes differences in the geometric properties of images as differences in probability distributions. Assume that the data satisfies a multivariate Gaussian distribution and that different images are considered as different points on the manifold according to the parameters that characterize the probability distribution of the images. We measure the similarity between different images by the geodesic and validate three methods for estimating the geodesic on a neural network model with 64 neurons at four levels and improve the model performance by Firth bias. The GM, GMF, KM, and KMF methods are validated in the resnet-12 and resnet-18 frameworks using a meta-learning approach on the miniImageNet and CUB datasets, respectively. The experiments show that the models trained by the above four algorithms have good performance and have advantages over the classical algorithms.Keywords
Funding Information
- National Natural Science Foundation of China (62032016,61972016)
- Beijing Nova Program (20220484106,20230484451)
This publication has 11 references indexed in Scilit:
- Disentangled Feature Representation for Few-Shot Image ClassificationIEEE Transactions on Neural Networks and Learning Systems, 2023
- An Information Geometric Perspective to Adversarial Attacks and Defenses2022
- Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification2022
- FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning2022
- Few-Shot Classification with Feature Map Reconstruction Networks2021
- Asymmetric Distribution Measure for Few-shot Learning2020
- Information Geometry and Its Applications2016
- Natural Gradient Works Efficiently in LearningNeural Computation, 1998
- Information geometry of the EM and em algorithms for neural networksNeural Networks, 1995
- Jeffreys' prior for logit modelsJournal of Econometrics, 1994