Hamming Diversification Index: A New Clustering-Based Metric to Understand and Visualize Time Evolution of Patterns in Multi-Dimensional Datasets
Open Access
- 10 July 2025
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 15 (14) , 7760
Abstract
One of the most challenging problems in data analysis is visualizing patterns and extracting insights from multi-dimensional datasets that vary over time. The complexity of data and variations in the correlations between different features adds further difficulty to the analysis. In this paper, we provide a framework to analyze the temporal dynamics of such datasets. We use machine learning clustering techniques and examine the time evolution of data patterns by constructing the corresponding cluster trajectories. These trajectories allow us to visualize the patterns and the changing nature of correlations over time. The similarity and correlations of features are reflected in common cluster membership, whereas the historical dynamics are described by a trajectory in the corresponding (cluster, time) space. This allows an effective visualization of multi-dimensional data over time. We introduce several statistical metrics to measure duration, volatility, and inertia of changes in patterns. Using the Hamming distance of trajectories over multiple time periods, we propose a novel metric, the Hamming diversification index, to measure the spread between trajectories. The novel metric is easy to compute, has a simple machine learning implementation, and provides additional insights into the temporal dynamics of data. This parsimonious diversification index can be used to examine changes in pattern similarities over aggregated time periods. We demonstrate the efficacy of our approach by analyzing a complex multi-year dataset of multiple worldwide economic indicators.Keywords
This publication has 27 references indexed in Scilit:
- Uncovering temporal patterns in visualizations of high-dimensional dataMachine Learning, 2025
- A user‐friendly guide to using distance measures to compare time series in ecologyEcology and Evolution, 2023
- A general methodology for n-dimensional trajectory clusteringExpert Systems with Applications, 2015
- Data Mining2015
- Ward’s Hierarchical Agglomerative Clustering Method: Which Algorithms Implement Ward’s Criterion?Journal of Classification, 2014
- Algorithms for hierarchical clustering: an overviewWIREs Data Mining and Knowledge Discovery, 2011
- A review on time series data miningEngineering Applications of Artificial Intelligence, 2011
- Data clustering: 50 years beyond K-meansPattern Recognition Letters, 2010
- Clustering of time series data—a surveyPattern Recognition, 2005
- Error Detecting and Error Correcting CodesBell System Technical Journal, 1950