Document Open Access Logo

Tight Better-Than-Worst-Case Bounds for Element Distinctness and Set Intersection

Authors Ivor van der Hoog , Eva Rotenberg , Daniel Rutschmann

PDF

File

Thumbnail PDF
PDF
LIPIcs.ESA.2026.101.pdf
  • Filesize: 0.85 MB
  • 21 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Design and analysis of algorithms
Keywords
  • Comparison-based analysis
  • set intersection
  • universal optimality

Metrics

Abstract

The element distinctness problem takes as input a list I of n values from a totally ordered universe, where pairwise comparisons between values are allowed, and the goal is to decide whether I contains any duplicates. It is a well-studied problem with a classical worst-case Ω(n log n) comparison-based lower bound by Fredman [TCS'76]. At first glance, this lower bound appears to rule out any algorithm more efficient than the naive approach of sorting I and comparing adjacent elements. However, upon closer inspection, the Ω(n log n) bound is overly pessimistic. For instance, if I contains n/2 identical elements, a median-finding algorithm will, regardless of the input order, find a duplicate in linear time. This raises a natural question: Are there comparison-based lower bounds for element distinctness that are sensitive to the amount of duplicates in the input instance?
To address this question, we derive instance-specific lower bounds. For any input instance I, we represent the combinatorial structure of the duplicates in I by an undirected graph G(I) that connects identical elements. Each such graph G is a union of cliques, and we study algorithms by their worst-case running time over all inputs I' with G(I') ≅ G. We establish an adversarial lower bound showing that, for any deterministic algorithm 𝒜, there exists a graph G and an algorithm 𝒜' that, for all inputs I with G(I) ≅ G, is a factor O(log log n) faster than 𝒜. Consequently, no deterministic algorithm can be o(log log n)-competitive for all graphs G. We complement this with an O(log log n)-competitive deterministic algorithm, thereby obtaining tight bounds for element distinctness that go beyond classical worst-case analysis. Subsequently, we study the related problem of set intersection. We show that no deterministic set intersection algorithm can be o(log n)-competitive, and provide an O(log n)-competitive deterministic algorithm. We find it interesting and surprising to discover tight O(log log n)-competitive bounds for element distinctness. Moreover, we find the separation between element distinctness and the set intersection problem unexpected.

Cite As Get BibTex

Ivor van der Hoog, Eva Rotenberg, and Daniel Rutschmann. Tight Better-Than-Worst-Case Bounds for Element Distinctness and Set Intersection. In 34th Annual European Symposium on Algorithms (ESA 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 388, pp. 101:1-101:21, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.ESA.2026.101

Author Details

Ivor van der Hoog
  • IT University of Copenhagen, Denmark
Eva Rotenberg
  • IT University of Copenhagen, Denmark
Daniel Rutschmann
  • Institute of Science and Technology Austria (ISTA), Klosterneuburg, Austria

Funding

Ivor van der Hoog, Eva Rotenberg, and Daniel Rutschmann thank the VILLUM Foundation grant (VIL37507) "Efficient Recomputations for Changeful Problems" for supporting this work. Daniel Rutschmann is supported by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (No. 101019564).

References

  1. Peyman Afshani, Jérémy Barbay, and Timothy Chan. Instance-optimal geometric algorithms. In IEEE Symposium on Foundations of Computer Science (FOCS), pages 129-138. Association for Computing Machinery, 2009. URL: https://doi.org/10.1145/3046673.
  2. Miklós Ajtai. A lower bound for finding predecessors in Yao’s cell probe model. Combinatorica, 8(3):235-247, 1988. URL: https://doi.org/10.1007/BF02126797.
  3. Paul Beame, Allan Borodin, Michael Saks, and Sven Skyum. Time–space tradeoffs for branching programs. Journal of Computer and System Sciences, 44(2):272-298, 1991. URL: https://doi.org/10.1016/0022-0000(91)90035-2.
  4. Paul Beame, Raphael Clifford, and Widad Machmouchi. Element distinctness, frequency moments, and sliding windows. In Symposium on Foundations of Computer Science (FOCS), FOCS '13, pages 290-299, USA, 2013. IEEE Computer Society. URL: https://doi.org/10.1109/FOCS.2013.39.
  5. Jean Cardinal, Samuel Fiorini, Gwenaël Joret, Raphaël M. Jungers, and J. Ian Munro. Sorting under partial information (without the ellipsoid algorithm). In ACM Symposium on Theory of Computing (STOC), STOC '10, New York, NY, USA, June 2010. Association for Computing Machinery. URL: https://doi.org/10.1145/1806689.1806740.
  6. Amit Chakrabarti, Graham Cormode, and Andrew McGregor. Robust lower bounds for communication and stream computation. In Symposium on Theory of Computing (STOC), STOC '08, pages 641-650, New York, NY, USA, 2008. Association for Computing Machinery. URL: https://doi.org/10.1145/1374376.1374470.
  7. Erik D. Demaine, Dion Harmon, John Iacono, and Mihai Patrascu. Dynamic optimality - Almost. In IEEE Symposium on Foundations of Computer Science (FOCS), pages 484-490, USA, 2004. IEEE Computer Society. URL: https://doi.org/10.1109/FOCS.2004.23.
  8. Michael L. Fredman. How good is the information theory bound in sorting? Theoretical Computer Science, 1(4):355-361, April 1976. URL: https://doi.org/10.1016/0304-3975(76)90078-5.
  9. Bernhard Haeupler, Richard Hladík, Václav Rozhon, Robert E. Tarjan, and Jakub Tetek. Universal optimality of dijkstra via beyond-worst-case heaps. In Annual Symposium on Foundations of Computer Science (FOCS), pages 2099-2130. IEEE, 2024. URL: https://doi.org/10.1109/FOCS61266.2024.00125.
  10. Bernhard Haeupler, Richard Hladík, John Iacono, Václav Rozhoň, Robert E. Tarjan, and Jakub Tětek. Fast and simple sorting using partial information. In Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 3953-3973, 2025. URL: https://doi.org/10.1137/1.9781611978322.134.
  11. Bernhard Haeupler, David Wajc, and Goran Zuzic. Universally-optimal distributed algorithms for known topologies. In ACM SIGACT Symposium on Theory of Computing (STOC), pages 1166-1179, New York, NY, USA, 2021. Association for Computing Machinery. URL: https://doi.org/10.1145/3406325.3451081.
  12. Yijie Han. Deterministic sorting in o(n log log n) time and linear space. In ACM Symposium on Theory of Computing (STOC), STOC '02, pages 602-608, New York, NY, USA, 2002. Association for Computing Machinery. URL: https://doi.org/10.1145/509907.509993.
  13. Yijie Han and Mikkel Thorup. Integer sorting in 0(n sqrt (log log n)) expected time and linear space. In Foundations of Computer Science (FOCS), pages 135-144. IEEE Computer Society, 2002. URL: https://doi.org/10.1109/SFCS.2002.1181890.
  14. Richard Hladík and Jakub Tětek. Near-Universally-Optimal Differentially Private Minimum Spanning Trees. In Mark Bun, editor, Foundations of Responsible Computing (FORC), volume 329 of Leibniz International Proceedings in Informatics (LIPIcs), pages 6:1-6:19, Dagstuhl, Germany, 2025. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.FORC.2025.6.
  15. Jeff Kahn and Jeong Han Kim. Entropy and sorting. In ACM symposium on Theory of Computing (STOC), STOC '92, pages 178-187, New York, NY, USA, July 1992. Association for Computing Machinery. URL: https://doi.org/10.1145/129712.129731.
  16. Jeff Kahn and Michael Saks. Balancing poset extensions. Order, 1(2):113-126, June 1984. URL: https://doi.org/10.1007/BF00565647.
  17. David G Kirkpatrick and Raimund Seidel. The ultimate planar convex hull algorithm? SIAM Journal on Computing, 15(1):287-299, 1986. URL: https://doi.org/10.1137/0215021.
  18. Tsvi Kopelowitz, Seth Pettie, and Ely Porat. Higher lower bounds from the 3sum conjecture. In ACM-SIAM Symposium on Discrete Algorithms (SODA), SODA '16, pages 1272-1287, USA, 2016. Society for Industrial and Applied Mathematics. URL: https://doi.org/10.5555/2884435.2884524.
  19. J. Misra and David Gries. Finding repeated elements. Science of Computer Programming, 2(2):143-152, 1982. URL: https://doi.org/10.1016/0167-6423(82)90012-0.
  20. Anna Ostlin and Rasmus Pagh. Uniform hashing in constant time and linear space. In ACM Symposium on Theory of Computing (STOC), STOC '03, pages 622-628, New York, NY, USA, 2003. Association for Computing Machinery. URL: https://doi.org/10.1145/780542.780633.
  21. Boaz Patt-Shamir and David Peleg. Time-space tradeoffs for set operations. Theoreticl Compututer Science, 110(1):99-129, March 1993. URL: https://doi.org/10.1016/0304-3975(93)90352-T.
  22. Mihai Pătraşcu and Erik D. Demaine. Logarithmic lower bounds in the cell-probe model. SIAM Journal on Computing, 35(4):932-963, 2006. URL: https://doi.org/10.1137/S0097539705447256.
  23. Daniel Dominic Sleator and Robert Endre Tarjan. Self-adjusting binary search trees. Journal of the ACM (JACM), 32(3):652-686, July 1985. URL: https://doi.org/10.1145/3828.3835.
  24. Ivor van der Hoog, Eva Rotenberg, and Daniel Rutschmann. A combinatorial proof of universal optimality for computing a planar convex hull. In Anne Benoit, Haim Kaplan, Sebastian Wild, and Grzegorz Herman, editors, European Symposium on Algorithms (ESA), volume 351 of LIPIcs, pages 102:1-102:13. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2025. URL: https://doi.org/10.4230/LIPIcs.ESA.2025.102.
  25. Ivor van der Hoog, Eva Rotenberg, and Daniel Rutschmann. Simpler optimal sorting from a directed acyclic graph. In Symposium on Simplicity in Algorithms (SOSA). SIAM, 2025. URL: https://doi.org/10.1137/1.9781611978315.26.
  26. Ivor van der Hoog, Eva Rotenberg, and Daniel Rutschmann. Simpler universally optimal dijkstra. In European Symposium on Algorithms (ESA), volume 351 of LIPIcs, pages 71:1-71:9. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2025. URL: https://doi.org/10.4230/LIPIcs.ESA.2025.71.
  27. Ivor van der Hoog and Daniel Rutschmann. Tight bounds for sorting under partial information. In Foundations of Computer Science, FOCS 2024, Chicago, IL, USA, October 27-30, 2024, pages 2243-2252. IEEE, 2024. URL: https://doi.org/10.1109/FOCS61266.2024.00131.
  28. Andrew Chi-Chih Yao. Near-optimal time-space tradeoff for element distinctness. SIAM Journal on Computing, 23(5):966-975, 1994. URL: https://doi.org/10.1137/S0097539788148959.
  29. Goran Zuzic, Gramoz Goranci, Mingquan Ye, Bernhard Haeupler, and Xiaorui Sun. Universally-optimal distributed shortest paths and transshipment via graph-based 𝓁1-oblivious routing. In Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 2549-2579, 2022. URL: https://doi.org/10.1137/1.9781611977073.100.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact