On the Attractive and Repulsive Forces of Generalized Stochastic Neighbor Embedding With Alpha-Divergence
Open Access
- 28 June 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 90380-90394
Abstract
Stochastic neighbor embedding (SNE) performs nonlinear transformation from high-dimensional observation space to low-dimensional latent space which preserves neighbor affinities. Data pairs in latent space tend to be crowded due to the dimensionality reduction. To mitigate the crowding problem, certain characteristics are favorable in the design of SNE setting. This study presents a fundamental analysis of SNE that not only generalizes the previous SNEs but also provides a systematic way to understand the intrinsic properties. From the perspective of theoretical connection, we are able to conceive a new generalized SNE ( g -SNE) by introducing a regularized power-law distribution with the α-divergence for manifold learning. The proposed method generalizes and incorporates various favorable features for the clustering process. In addition, the proposed method provides high flexibility, admitting tailored realizations to reflect the similarity properly between original and dimension-reduced samples. Experiments are performed to analyze the proposed method, and its effectiveness is demonstrated with several learning tasks.Keywords
This publication has 27 references indexed in Scilit:
- Revealing hidden patterns in deep neural network feature space continuum via manifold learningNature Communications, 2023
- Enhance explainability of manifold learningNeurocomputing, 2022
- CO-SNE: Dimensionality Reduction and Visualization for Hyperbolic Data2022
- Manifold learning with structured subspace for multi-label feature selectionPattern Recognition, 2021
- Semisupervised Feature Selection via Structured Manifold LearningIEEE Transactions on Cybernetics, 2021
- Structured optimal graph based sparse feature extraction for semi-supervised learningSignal Processing, 2020
- Stochastic neighbor embedding (SNE) for dimension reduction and visualization using arbitrary divergencesNeurocomputing, 2012
- Spherical Stochastic Neighbor Embedding of Hyperspectral DataIEEE Transactions on Geoscience and Remote Sensing, 2012
- Convex Divergence ICA for Blind Source SeparationIEEE Transactions on Audio, Speech, and Language Processing, 2011
- Interpretable dimension reductionJournal of Applied Statistics, 2005