Proof of biased behavior of Normalized Mutual Information
Authors:
Top Cited Papers
Open Access
- 19 April 2024
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 14 (1) , 1-17
Abstract
The Normalized Mutual Information (NMI) metric is widely utilized in the evaluation of clustering and community detection algorithms. This study explores the performance of NMI, specifically examining its performance in relation to the quantity of communities, and uncovers a significant drawback associated with the metric's behavior as the number of communities increases. Our findings reveal a pronounced bias in the NMI as the number of communities escalates. While previous studies have noted this biased behavior, they have not provided a formal proof and have not addressed the causation of this problem, leaving a gap in the existing literature. In this study, we fill this gap by employing a mathematical approach to formally demonstrate why NMI exhibits biased behavior, thereby establishing its unsuitability as a metric for evaluating clustering and community detection algorithms. Crucially, our study exposes the vulnerability of entropy-based metrics that employ logarithmic functions to similar bias.Keywords
Funding Information
- Narodowe Centrum Nauki (2020/38/A/HS6/00066)
This publication has 44 references indexed in Scilit:
- On the Modeling and Simulation of Portfolio Allocation Schemes: an Approach Based on Network Community DetectionComputational Economics, 2022
- Maximum Entropy Networks Applied on Twitter Disinformation Datasets2022
- How the Far-Right Polarises Twitter: ‘Hashjacking’ as a Disinformation Strategy in Times of COVID-192022
- Community Detection and Design of Recommendation System Based on Criminal Incidents2020
- Describing Alt-Right Communities and Their Discourse on Twitter During the 2018 US Mid-term Elections2019
- Beyond misinformation: Understanding and coping with the “post-truth” era.Journal of Applied Research in Memory and Cognition, 2017
- A Novel Approach to Evaluate Community Detection Algorithms on Ground Truth2016
- On the Use of the Adjusted Rand Index as a Metric for Evaluating Supervised Classification2009
- Comparing clusterings—an information based distanceJournal of Multivariate Analysis, 2007
- Comparing Clusterings by the Variation of Information2003