A Polynomial Space Lower Bound for Diameter Estimation in Dynamic Streams
Preprint
Abstract
We study the space complexity of estimating the diameter of a subset of points in an arbitrary metric space in the dynamic (turnstile) streaming model. The input is given as a stream of updates to a frequency vector $x \in \mathbb{Z}_{\geq 0}^n$, where the support of $x$ defines a multiset of points in a fixed metric space $M = ([n], \mathsf{d})$. The goal is to estimate the diameter of this multiset, defined as $\max\{\mathsf{d}(i,j) : x_i, x_j > 0\}$, to a specified approximation factor while using as little space as possible. In insertion-only streams, a simple $O(\log n)$-space algorithm achieves a 2-approximation. In sharp contrast to this, we show that in the dynamic streaming model, any algorithm achieving a constant-factor approximation to diameter requires polynomial space. Specifically, we prove that a $c$-approximation to the diameter requires $n^{Ω(1/c)}$ space. Our lower bound relies on two conceptual contributions: (1) a new connection between dynamic streaming algorithms and linear sketches for {\em scale-invariant} functions, a class that includes diameter estimation, and (2) a connection between linear sketches for diameter and the {\em minrank} of graphs, a notion previously studied in index coding. We complement our lower bound with a nearly matching upper bound, which gives a $c$-approximation to the diameter in general metrics using $n^{O(1/c)}$ space.
Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- New streaming algorithms for high dimensional EMD and MST2022
- Markov Chains and Mixing Times2017
- Nonlinear Index Coding Outperforming the Linear OptimumIEEE Transactions on Information Theory, 2009
- Metric structures in : dimension, snowflakes, and average distortionEuropean Journal of Combinatorics, 2004
- On embedding expanders into ℓ p spacesIsrael Journal of Mathematics, 1997
- On the distortion required for embedding finite metric spaces into normed spacesIsrael Journal of Mathematics, 1996
- On lipschitz embedding of finite metric spaces in Hilbert spaceIsrael Journal of Mathematics, 1985