Assessing Image and Text Generation with Topological Analysis and Fuzzy Logic
Authors:
- 1 January 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 13, 2012-2021
Abstract
Objective and interpretable metrics to evaluate current artificial intelligent systems are of great importance, not only to analyze the current state of such systems but also to objectively measure progress in the future. We propose a novel metric, called Fuzzy Topology Impact (FTI), that assesses both the quality and diversity of a generated set using topological representations combined with fuzzy logic. In our synthetic experiments, FTI consistently outperforms current evaluation methods in terms of stability and sensitivity to detect drops in quality and diversity in the generated set, both on image and text generation tasks. Moreover, FTI shows a high degree of correlation to human evaluation on unconditional language generation.Keywords
This publication has 9 references indexed in Scilit:
- microbatchGAN: Stimulating Diversity with Multi-Adversarial Discrimination2020
- Mark-Evaluate: Assessing Language Generation using Population Estimation Methods2020
- A Style-Based Generator Architecture for Generative Adversarial Networks2019
- How Good Is My GAN?2018
- Universal Sentence Encoder for English2018
- DeLiGAN: Generative Adversarial Networks for Diverse and Limited Data2017
- Rethinking the Inception Architecture for Computer Vision2016
- ImageNet: A large-scale hierarchical image database2009
- Laplacian Eigenmaps for Dimensionality Reduction and Data RepresentationNeural Computation, 2003