Document Open Access Logo

Systematic Data Structure Lower Bounds via the Query-With-Sketch Model

Authors Sumegha Garg , Songhua He , Yuanzhi Li , Periklis A. Papakonstantinou , Xin Yang

PDF

File

Thumbnail PDF
PDF
LIPIcs.CCC.2026.41.pdf
  • Filesize: 1.08 MB
  • 44 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Theory of computation → Data structures design and analysis
Keywords
  • systematic data structure
  • query-with-sketch model
  • approximate matrix powering

Metrics

Abstract

We study data structure lower bounds for the Approximate Matrix Powering (AMP) problem. Given a substochastic, symmetric matrix 𝐌 ∈ ℝ^{n× n} and parameters k and α, the goal is to preprocess 𝐌 so as to answer entry queries (u,v)↦ 𝐌^{k}[u,v] up to additive error 1/n^{α}. We focus on AMP in the succinct and systematic regime, in which the data structure stores 𝐌 verbatim, uses an additional r bits of redundancy, and must answer queries by probing only a small number of entries of 𝐌.
Our main conceptual contribution is a general framework for proving probe-redundancy trade-offs for systematic data structures. We introduce the query-with-sketch model and develop a min-entropy-based approach that lifts conditional min-entropy bounds in the absence of redundancy to probe lower bounds in the presence of redundancy. We then establish these min-entropy bounds using problem-specific analytic and algebraic tools, for the downstream applications to AMP and its variants. As a consequence, our results provide new unconditional evidence toward a conjecture of Pătraşcu and Roditty on the space required for constant-time set-disjointness queries [Patrascu and Roditty, 2010].

Cite As Get BibTex

Sumegha Garg, Songhua He, Yuanzhi Li, Periklis A. Papakonstantinou, and Xin Yang. Systematic Data Structure Lower Bounds via the Query-With-Sketch Model. In 41st Computational Complexity Conference (CCC 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 383, pp. 41:1-41:44, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.CCC.2026.41

Author Details

Sumegha Garg
  • Rutgers University, New Brunswick, NJ, USA
Songhua He
  • Rutgers University, New Brunswick, NJ, USA
Yuanzhi Li
  • Carnegie Mellon University, Pittsburgh, PA, USA
Periklis A. Papakonstantinou
  • Rutgers University, New Brunswick, NJ, USA
Xin Yang
  • Meta, Seattle, WA, USA

Acknowledgements

We thank the anonymous reviewers for their valuable suggestions.

References

  1. Peyman Afshani and Jesper Sindahl Nielsen. Data structure lower bounds for document indexing problems. In Ioannis Chatzigiannakis, Michael Mitzenmacher, Yuval Rabani, and Davide Sangiorgi, editors, 43rd International Colloquium on Automata, Languages, and Programming, ICALP 2016, Rome, Italy, July 11-15, 2016, volume 55 of LIPIcs, pages 93:1-93:15. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2016. URL: https://doi.org/10.4230/LIPIcs.ICALP.2016.93.
  2. Richard Beigel and Tirza Hirst. One help bit doesn't help. In Jeffrey Scott Vitter, editor, Proceedings of the Thirtieth Annual ACM Symposium on the Theory of Computing, Dallas, Texas, USA, May 23-26, 1998, pages 124-130. ACM, 1998. URL: https://doi.org/10.1145/276698.276720.
  3. Eric David Belsley. Rates of convergence of Markov chains related to association schemes. Harvard University, 1993. Google Scholar
  4. Vladimir I Bogachev. Measures on topological spaces. In Measure theory, pages 476-583. Springer, 2007. Google Scholar
  5. Sergey Brin and Lawrence Page. The anatomy of a large-scale hypertextual web search engine. Computer networks and ISDN systems, 30(1-7):107-117, 1998. URL: https://doi.org/10.1016/S0169-7552(98)00110-X.
  6. Karl Bringmann and Kasper Green Larsen. Succinct sampling from discrete distributions. In Proceedings of the forty-fifth annual ACM symposium on Theory of Computing, pages 775-782, 2013. URL: https://doi.org/10.1145/2488608.2488707.
  7. Diptarka Chakraborty, Lior Kamma, and Kasper Green Larsen. Tight cell probe bounds for succinct boolean matrix-vector multiplication. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, pages 1297-1306, 2018. URL: https://doi.org/10.1145/3188745.3188830.
  8. Hagai Cohen and Ely Porat. Fast set intersection and two-patterns matching. Theoretical Computer Science, 411(40-42):3795-3800, 2010. URL: https://doi.org/10.1016/j.tcs.2010.06.002.
  9. Henry Corrigan-Gibbs and Dmitry Kogan. The function-inversion problem: Barriers and opportunities. In Theory of Cryptography Conference, pages 393-421, 2019. URL: https://doi.org/10.1007/978-3-030-36030-6_16.
  10. Zeev Dvir, Alexander Golovnev, and Omri Weinstein. Static data structure lower bounds imply rigidity. In Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, pages 967-978, 2019. URL: https://doi.org/10.1145/3313276.3316348.
  11. Pavel Dvorák, Michal Koucký, Karel Král, and Veronika Slívová. Data structures lower bounds and popular conjectures. In Petra Mutzel, Rasmus Pagh, and Grzegorz Herman, editors, 29th Annual European Symposium on Algorithms, ESA 2021, Lisbon, Portugal (Virtual Conference), September 6-8, 2021, volume 204 of LIPIcs, pages 39:1-39:15. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2021. URL: https://doi.org/10.4230/LIPIcs.ESA.2021.39.
  12. Jason Fulman and Larry Goldstein. Stein’s method and the rank distribution of random matrices over finite fields. The Annals of Probability, 43(3):1274-1314, 2015. Google Scholar
  13. Anna Gál and Peter Bro Miltersen. The cell probe complexity of succinct data structures. Theoretical computer science, 379(3):405-417, 2007. URL: https://doi.org/10.1016/j.tcs.2007.02.047.
  14. Isaac Goldstein, Tsvi Kopelowitz, Moshe Lewenstein, and Ely Porat. Conditional lower bounds for space/time tradeoffs. In Workshop on Algorithms and Data Structures, pages 421-436, 2017. URL: https://doi.org/10.1007/978-3-319-62127-2_36.
  15. Isaac Goldstein, Moshe Lewenstein, and Ely Porat. On the hardness of set disjointness and set intersection with bounded universe. In Pinyan Lu and Guochuan Zhang, editors, 30th International Symposium on Algorithms and Computation, ISAAC 2019, Shanghai University of Finance and Economics, Shanghai, China, December 8-11, 2019, volume 149 of LIPIcs, pages 7:1-7:22. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2019. URL: https://doi.org/10.4230/LIPIcs.ISAAC.2019.7.
  16. Alexander Golovnev, Siyao Guo, Thibaut Horel, Sunoo Park, and Vinod Vaikuntanathan. Data structures meet cryptography: 3sum with preprocessing. In Proceedings of the 52nd annual ACM SIGACT symposium on theory of computing, pages 294-307, 2020. URL: https://doi.org/10.1145/3357713.3384342.
  17. Gene H Golub and Charles F Van Loan. Matrix computations. JHU press, 2013. Google Scholar
  18. Alexander Golynski. Optimal lower bounds for rank and select indexes. Theoretical Computer Science, 387(3):348-359, 2007. URL: https://doi.org/10.1016/j.tcs.2007.07.041.
  19. Alexander Golynski, Roberto Grossi, Ankur Gupta, Rajeev Raman, and S. Srinivasa Rao. On the size of succinct indices. In Lars Arge, Michael Hoffmann, and Emo Welzl, editors, Algorithms - ESA 2007, 15th Annual European Symposium, Eilat, Israel, October 8-10, 2007, Proceedings, volume 4698 of Lecture Notes in Computer Science, pages 371-382. Springer, 2007. URL: https://doi.org/10.1007/978-3-540-75520-3_34.
  20. Alexander Golynski, Rajeev Raman, and S Srinivasa Rao. On the redundancy of succinct data structures. In Scandinavian Workshop on Algorithm Theory, pages 148-159, 2008. URL: https://doi.org/10.1007/978-3-540-69903-3_15.
  21. Monika Henzinger, Sebastian Krinninger, Danupon Nanongkai, and Thatchaphol Saranurak. Unifying and strengthening hardness for dynamic problems via the online matrix-vector multiplication conjecture. In Proceedings of the forty-seventh annual ACM symposium on Theory of computing, pages 21-30, 2015. URL: https://doi.org/10.1145/2746539.2746609.
  22. Roger A. Horn and Charles R. Johnson. Matrix Analysis, 2nd Ed. Cambridge University Press, 2012. URL: https://doi.org/10.1017/CBO9781139020411.
  23. Guy Joseph Jacobson. Succinct static data structures. Carnegie Mellon University, 1988. Google Scholar
  24. Tsvi Kopelowitz and Virginia Vassilevska Williams. Towards optimal set-disjointness and set-intersection data structures. In Artur Czumaj, Anuj Dawar, and Emanuela Merelli, editors, 47th International Colloquium on Automata, Languages, and Programming, ICALP 2020, Saarbrücken, Germany (Virtual Conference), July 8-11, 2020, volume 168 of LIPIcs, pages 74:1-74:16. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020. URL: https://doi.org/10.4230/LIPIcs.ICALP.2020.74.
  25. Kasper Green Larsen. Higher cell probe lower bounds for evaluating polynomials. In 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science, pages 293-301. IEEE, 2012. URL: https://doi.org/10.1109/FOCS.2012.21.
  26. Peter Bro Miltersen. Lower bounds on the size of selection and rank indexes. In SODA, volume 5, pages 11-12, 2005. URL: http://dl.acm.org/citation.cfm?id=1070432.1070435.
  27. Peter Bro Miltersen, Noam Nisan, Shmuel Safra, and Avi Wigderson. On data structures and asymmetric communication complexity. In Proceedings of the twenty-seventh annual ACM symposium on Theory of computing, pages 103-111, 1995. URL: https://doi.org/10.1145/225058.225093.
  28. Rajeev Motwani and Prabhakar Raghavan. Randomized algorithms. ACM Computing Surveys (CSUR), 28(1):33-37, 1996. URL: https://doi.org/10.1145/234313.234327.
  29. Noam Nisan, Steven Rudich, and Michael Saks. Products and help bits in decision trees. SIAM Journal on Computing, 28(3):1035-1050, 1998. URL: https://doi.org/10.1137/S0097539795282444.
  30. James R. Norris. Markov chains. Cambridge series in statistical and probabilistic mathematics. Cambridge University Press, 1998. Google Scholar
  31. Mihai Patrascu and Liam Roditty. Distance oracles beyond the thorup-zwick bound. In 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, pages 815-823. IEEE, 2010. URL: https://doi.org/10.1109/FOCS.2010.83.
  32. Mihai Patrascu and Emanuele Viola. Cell-probe lower bounds for succinct partial sums. In Moses Charikar, editor, Proceedings of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2010, Austin, Texas, USA, January 17-19, 2010, pages 117-122. SIAM, SIAM, 2010. URL: https://doi.org/10.1137/1.9781611973075.11.
  33. Sivaramakrishnan Natarajan Ramamoorthy and Cyrus Rashtchian. Equivalence of systematic linear data structures and matrix rigidity. In Thomas Vidick, editor, 11th Innovations in Theoretical Computer Science Conference, ITCS 2020, Seattle, Washington, USA, January 12-14, 2020, volume 151 of LIPIcs, pages 35:1-35:20. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020. URL: https://doi.org/10.4230/LIPIcs.ITCS.2020.35.
  34. Stanislaw J Szarek. Spaces with large distance to 𝓁ⁿ_∞ and random matrices. American Journal of Mathematics, 112(6):899-942, 1990. Google Scholar
  35. Robert Endre Tarjan. A class of algorithms which require nonlinear time to maintain disjoint sets. Journal of computer and system sciences, 18(2):110-127, 1979. URL: https://doi.org/10.1016/0022-0000(79)90042-4.
  36. Robert C Thompson. The behavior of eigenvalues and singular values under perturbations of restricted rank. Linear Algebra and its Applications, 13(1-2):69-78, 1976. Google Scholar
  37. Mikkel Thorup and Uri Zwick. Approximate distance oracles. Journal of the ACM (JACM), 52(1):1-24, 2005. URL: https://doi.org/10.1145/1044731.1044732.
  38. Leslie G Valiant. Graph-theoretic arguments in low-level complexity. In International Symposium on Mathematical Foundations of Computer Science, pages 162-176, 1977. URL: https://doi.org/10.1007/3-540-08353-7_135.
  39. Leslie G Valiant. Why is boolean complexity theory difficult. Boolean Function Complexity, 169(84-94):4, 1992. Google Scholar
  40. Roman Vershynin. High-Dimensional Probability: An Introduction with Applications in Data Science. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2018. Google Scholar
  41. Emanuele Viola. Lower bounds for data structures with space close to maximum imply circuit lower bounds. Theory of Computing, 15(1):1-9, 2019. URL: https://doi.org/10.4086/toc.2019.v015a018.
  42. Feng Wei. Upper bound for intermediate singular values of random matrices. Journal of Mathematical Analysis and Applications, 445(2):1530-1547, 2017. Google Scholar
  43. Andrew Chi-Chih Yao. Should tables be sorted? Journal of the ACM (JACM), 28(3):615-628, 1981. URL: https://doi.org/10.1145/322261.322274.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact