Search Results

Estimating the second frequency moment F₂ of a data stream up to a (1 ± ε) factor is a central problem in the streaming literature. For errors ε > Ω(1/√n), the tight bound Θ(log(ε² n)/ε²) was recently established by Braverman and Zamir. In this work, we complete the picture by resolving the remaining regime of small error, ε < 1/√n, showing that the optimal space complexity is Θ(min(n, 1/ε²)⋅(1 + |log(ε² n)|)) bits for all ε ≥ 1/n², assuming a sufficiently large universe. This closes the gap between the best known Ω(n) lower bound and the straightforward O(n log n) upper bound in that range, and shows that essentially storing the entire stream is necessary for high-precision estimation. To derive this bound, we fully characterize the two-party communication complexity of estimating the size of a set intersection up to an arbitrary additive error ε n. In particular, we prove a tight Ω(n log n) lower bound for one-way communication protocols when ε < n^{-1/2-Ω(1)}, in contrast to classical O(n)-bit protocols that use two-way communication. Motivated by this separation, we present a two-pass streaming algorithm that computes the exact histogram of a stream with high probability using only O(n log log n) bits of space, in contrast to the Θ(n log n) bits required in one pass even to approximate F₂ with small error. This yields the first asymptotic separation between one-pass and O(1)-passes space complexity for small frequency moment estimation.