Measuring Distances in High Dimensional Spaces
Open Access
- 22 January 2025
- journal article
- letter
- Published by Cambridge University Press (CUP) in Political Analysis
- Vol. 33 (3) , 266-273
Abstract
Analysts often seek to compare representations in high-dimensional space, e.g., embedding vectors of the same word across groups. We show that the distance measures calculated in such cases can exhibit considerable statistical bias, that stems from uncertainty in the estimation of the elements of those vectors. This problem applies to Euclidean distance, cosine similarity, and other similar measures. After illustrating the severity of this problem for text-as-data applications, we provide and validate a bias correction for the squared Euclidean distance. This same correction also substantially reduces bias in ordinary Euclidean distance and cosine similarity estimates, but corrections for these measures are not quite unbiased and are (non-intuitively) bimodal when distances are close to zero. The estimators require obtaining the variance of the latent positions. We (will) implement the estimator in free software, and we offer recommendations for related work.Keywords
This publication has 12 references indexed in Scilit:
- Lexical Ambiguity in Political Rhetoric: Why Morality Doesn't Fit in a Bag of WordsBritish Journal of Political Science, 2023
- Embedding Regression: Models for Context-Specific Description and InferenceAmerican Political Science Review, 2023
- A Statistical Test for Legal Interpretation: Theory and ApplicationsThe Journal of Law, Economics, and Organization, 2021
- Measuring Agenda Setting in Interactive Political CommunicationAmerican Journal of Political Science, 2021
- Using Administrative Records and Survey Data to Construct Samples of Tweeters and TweetsPublic Opinion Quarterly, 2021
- Matching with Text Data: An Experimental Evaluation of Methods for Matching Documents and of Measuring Match QualityPolitical Analysis, 2020
- The Uptick in Income Segregation: Real Trend or Random Sampling Variation?American Journal of Sociology, 2018
- Reliability of dissimilarity measures for multi-voxel pattern analysisNeuroImage, 2016
- THE ROLE OF ECOLOGICAL CONSTRAINT IN DRIVING THE EVOLUTION OF AVIAN SONG FREQUENCY ACROSS A LATITUDINAL GRADIENTEvolution, 2012
- The Bootstrap Variance of the Square of a Sample MeanThe American Statistician, 2007