Document Open Access Logo

Optimal Static Fully Indexable Dictionaries

Authors Jingxun Liang , Renfei Zhou

PDF

File

Thumbnail PDF
PDF
LIPIcs.ICALP.2025.114.pdf
  • Filesize: 1.09 MB
  • 20 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Data structures design and analysis
Keywords
  • data structures
  • dictionaries
  • space efficiency

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

Fully indexable dictionaries (FID) store sets of integer keys while supporting rank/select queries. They serve as basic building blocks in many succinct data structures. Despite the great importance of FIDs, no known FID is succinct with efficient query time when the universe size U is a large polynomial in the number of keys n, which is the conventional parameter regime for dictionary problems. In this paper, we design an FID that uses log binom(U,n) + n/((log U/t)^{Ω(t)}) bits of space, and answers rank/select queries in O(t + log log n) time in the worst case, for any parameter 1 ≤ t ≤ log n / log log n, provided U = n^{1 + Θ(1)}. This time-space trade-off matches known lower bounds for FIDs [Pǎtraşcu and Thorup, 2006; Pǎtraşcu and Viola, 2010; Viola, 2023] when t ≤ log^{0.99} n.
Our techniques also lead to efficient succinct data structures for the fundamental problem of maintaining n integers each of 𝓁 = Θ(log n) bits and supporting partial-sum queries, with a trade-off between O(t) query time and n𝓁 + n / (log n / t)^{Ω(t)} bits of space. Prior to this work, no known data structure for the partial-sum problem achieves constant query time with n 𝓁 + o(n) bits of space usage.

Cite As Get BibTex

Jingxun Liang and Renfei Zhou. Optimal Static Fully Indexable Dictionaries. In 52nd International Colloquium on Automata, Languages, and Programming (ICALP 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 334, pp. 114:1-114:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.ICALP.2025.114

Author Details

Jingxun Liang
  • Carnegie Mellon University, Pittsburgh, PA, USA
Renfei Zhou
  • Carnegie Mellon University, Pittsburgh, PA, USA

Funding

  • Zhou, Renfei: Partially supported by the MongoDB PhD Fellowship.

Acknowledgements

The authors thank William Kuszmaul and Huacheng Yu for helpful suggestions on paper writing, and anonymous reviewers for pointing out important related works.

References

  1. Sepehr Assadi, Martín Farach-Colton, and William Kuszmaul. Tight bounds for monotone minimal perfect hashing. ACM Transactions on Algorithms, page 3677608, August 2024. See also SODA 2023. URL: https://doi.org/10.1145/3677608.
  2. Jérémy Barbay, Francisco Claude, Travis Gagie, Gonzalo Navarro, and Yakov Nekrich. Efficient fully-compressed sequence representations. Algorithmica, 69(1):232-268, May 2014. URL: https://doi.org/10.1007/s00453-012-9726-3.
  3. Djamal Belazzougui, Paolo Boldi, Rasmus Pagh, and Sebastiano Vigna. Monotone minimal perfect hashing: Searching a sorted table with O(1) accesses. In Proc. 20th ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 785-794, 2009. URL: https://doi.org/10.1137/1.9781611973068.86.
  4. Djamal Belazzougui and Gonzalo Navarro. Optimal lower and upper bounds for representing sequences. ACM Transactions on Algorithms, 11(4):1-21, June 2015. URL: https://doi.org/10.1145/2629339.
  5. Michael A. Bender, Martín Farach-Colton, John Kuszmaul, and William Kuszmaul. Modern hashing made simple. In Proc. 7th Symposium on Simplicity in Algorithms (SOSA), pages 363-373, 2024. URL: https://doi.org/10.1137/1.9781611977936.33.
  6. Michael A. Bender, Martín Farach-Colton, John Kuszmaul, William Kuszmaul, and Mingmou Liu. On the optimal time/space tradeoff for hash tables. In Proc. 54th ACM SIGACT Symposium on Theory of Computing (STOC), pages 1284-1297, 2022. URL: https://doi.org/10.1145/3519935.3519969.
  7. Daniel K. Blandford and Guy E. Blelloch. Compact dictionaries for variable-length keys and data with applications. ACM Transactions on Algorithms, 4(2):17:1-17:25, May 2008. URL: https://doi.org/10.1145/1361192.1361194.
  8. H. Buhrman, P. B. Miltersen, J. Radhakrishnan, and S. Venkatesh. Are bitvectors optimal? SIAM Journal on Computing, 31(6):1723-1744, January 2002. URL: https://doi.org/10.1137/S0097539702405292.
  9. David R. Clark. Compact PAT trees. PhD thesis, University of Waterloo, 1996. Google Scholar
  10. David R. Clark and J. Ian Munro. Efficient suffix trees on secondary storage. In Proc. 7th ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 383-391, 1996. Google Scholar
  11. Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms, fourth edition. MIT Press, April 2022. Google Scholar
  12. P. Ferragina, F. Luccio, G. Manzini, and S. Muthukrishnan. Structuring labeled trees for optimal succinctness, and beyond. In Proc. 46th IEEE Symposium on Foundations of Computer Science (FOCS), pages 184-193, 2005. URL: https://doi.org/10.1109/SFCS.2005.69.
  13. Paolo Ferragina, Roberto Grossi, Ankur Gupta, Rahul Shah, and Jeffrey Scott Vitter. On searching compressed string collections cache-obliviously. In Proc. 27th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS), pages 181-190, 2008. URL: https://doi.org/10.1145/1376916.1376943.
  14. Paolo Ferragina and Giovanni Manzini. Indexing compressed text. Journal of the ACM, 52(4):552-581, July 2005. URL: https://doi.org/10.1145/1082036.1082039.
  15. Paolo Ferragina, Giovanni Manzini, Veli Mäkinen, and Gonzalo Navarro. Compressed representations of sequences and full-text indexes. ACM Transactions on Algorithms, 3(2):20-es, May 2007. URL: https://doi.org/10.1145/1240233.1240243.
  16. Michael L. Fredman, János Komlós, and Endre Szemerédi. Storing a sparse table with O(1) worst case access time. Journal of the ACM, 31(3):538-544, June 1984. URL: https://doi.org/10.1145/828.1884.
  17. Richard F. Geary, Naila Rahman, Rajeev Raman, and Venkatesh Raman. A simple optimal representation for balanced parentheses. Theoretical Computer Science, 368(3):231-246, December 2006. URL: https://doi.org/10.1016/j.tcs.2006.09.014.
  18. Richard F. Geary, Rajeev Raman, and Venkatesh Raman. Succinct ordinal trees with level-ancestor queries. ACM Transactions on Algorithms, 2(4):510-534, October 2006. URL: https://doi.org/10.1145/1198513.1198516.
  19. Alexander Golynski, J. Ian Munro, and Srinivasa Rao Satti. Rank/select operations on large alphabets: a tool for text indexing. In Proc. 17th ACM-SIAM Symposium on Discrete Algorithm (SODA), pages 368-373, 2006. URL: https://doi.org/10.1145/1109557.1109599.
  20. Alexander Golynski, Rajeev Raman, and Srinivasa Rao Satti. On the redundancy of succinct data structures. In Proc. 11th Scandinavian Workshop on Algorithm Theory (SWAT), pages 148-159, 2008. URL: https://doi.org/10.1007/978-3-540-69903-3_15.
  21. Roberto Grossi, Ankur Gupta, and Jeffrey Scott Vitter. High-order entropy-compressed text indexes. In Proc. 14th ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 841-850, 2003. URL: http://dl.acm.org/citation.cfm?id=644108.644250.
  22. Roberto Grossi, Alessio Orlandi, and Rajeev Raman. Optimal trade-offs for succinct string indexes. In Proc. 37th International Colloquium Conference on Automata, Languages and Programming (ICALP), pages 678-689, 2010. URL: https://doi.org/10.1007/978-3-642-14165-2_57.
  23. Roberto Grossi, Alessio Orlandi, Rajeev Raman, and Srinivasa Rao Satti. More haste, less waste: lowering the redundancy in fully indexable dictionaries. In Proc. 26th International Symposium on Theoretical Aspects of Computer Science (STACS), pages 517-528, 2009. URL: https://doi.org/10.4230/LIPICS.STACS.2009.1847.
  24. Roberto Grossi and Jeffrey Scott Vitter. Compressed suffix arrays and suffix trees with applications to text indexing and string matching. SIAM Journal on Computing, 35(2):378-407, January 2005. URL: https://doi.org/10.1137/S0097539702402354.
  25. Ankur Gupta, Wing-Kai Hon, Rahul Shah, and Jeffrey Scott Vitter. Compressed data structures: Dictionaries and data-aware measures. Theoretical Computer Science, 387(3):313-331, 2007. URL: https://doi.org/10.1016/j.tcs.2007.07.042.
  26. Wing-Kai Hon, Kunihiko Sadakane, and Wing-Kin Sung. Breaking a time-and-space barrier in constructing full-text indices. SIAM Journal on Computing, 38(6):2162-2178, January 2009. URL: https://doi.org/10.1137/070685373.
  27. Yang Hu, Jingxun Liang, Huacheng Yu, Junkai Zhang, and Renfei Zhou. Optimal static dictionary with worst-case constant query time. In Proc. 57th ACM Symposium on Theory of Computing (STOC), 2025. Google Scholar
  28. Guy Joseph Jacobson. Succinct static data structures. PhD thesis, Carnegie Mellon University, 1988. Google Scholar
  29. Dmitry Kosolobov. Simplified tight bounds for monotone minimal perfect hashing. In Proc. 35th Symposium on Combinatorial Pattern Matching (CPM), pages 19:1-19:13, 2024. URL: https://doi.org/10.4230/LIPICS.CPM.2024.19.
  30. Tianxiao Li, Jingxun Liang, Huacheng Yu, and Renfei Zhou. Dynamic "succincter". In Proc. 64th IEEE Symposium on Foundations of Computer Science (FOCS), pages 1715-1733, 2023. URL: https://doi.org/10.1109/FOCS57990.2023.00104.
  31. Tianxiao Li, Jingxun Liang, Huacheng Yu, and Renfei Zhou. Tight cell-probe lower bounds for dynamic succinct dictionaries. In Proc. 64th IEEE Symposium on Foundations of Computer Science (FOCS), pages 1842-1862, 2023. URL: https://doi.org/10.1109/FOCS57990.2023.00112.
  32. Tianxiao Li, Jingxun Liang, Huacheng Yu, and Renfei Zhou. Dynamic dictionary with subconstant wasted bits per key. In Proc. 35th ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 171-207, 2024. URL: https://doi.org/10.1137/1.9781611977912.9.
  33. J. Ian Munro. Tables. In Proc. 16th Conference on Foundations of Software Technology and Theoretical Computer Science (FSTTCS), pages 37-42, 1996. URL: https://doi.org/10.1007/3-540-62034-6_35.
  34. J. Ian Munro, Rajeev Raman, Venkatesh Raman, and Srinivasa Rao Satti. Succinct representations of permutations and functions. Theoretical Computer Science, 438:74-88, June 2012. URL: https://doi.org/10.1016/j.tcs.2012.03.005.
  35. J. Ian Munro and Venkatesh Raman. Succinct representation of balanced parentheses and static trees. SIAM Journal on Computing, 31(3):762-776, 2001. URL: https://doi.org/10.1137/S0097539799364092.
  36. J.Ian Munro, Venkatesh Raman, and S.Srinivasa Rao. Space efficient suffix trees. Journal of Algorithms, 39(2):205-222, 2001. URL: https://doi.org/10.1006/jagm.2000.1151.
  37. Gonzalo Navarro and Veli Mäkinen. Compressed full-text indexes. ACM Computing Surveys, 39(1):2-es, April 2007. URL: https://doi.org/10.1145/1216370.1216372.
  38. Gonzalo Navarro and Javiel Rojas-Ledesma. Predecessor search. ACM Computing Surveys, 53(5):1-35, September 2021. URL: https://doi.org/10.1145/3409371.
  39. Mihai Pǎtraşcu. Succincter. In Proc. 49th IEEE Symposium on Foundations of Computer Science (FOCS), pages 305-313, 2008. Google Scholar
  40. Mihai Pǎtraşcu and Mikkel Thorup. Time-space trade-offs for predecessor search. In Proc. 38th ACM Symposium on Theory of Computing (STOC), pages 232-240, 2006. URL: https://doi.org/10.1145/1132516.1132551.
  41. Mihai Pǎtraşcu and Emanuele Viola. Cell-probe lower bounds for succinct partial sums. In Proc. 21st ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 117-122, 2010. Google Scholar
  42. Giulio Ermanno Pibiri and Rossano Venturini. Dynamic Elias-Fano representation. In Proc. 28th Annual Symposium on Combinatorial Pattern Matching (CPM), pages 30:1-30:14, 2017. URL: https://doi.org/10.4230/LIPIcs.CPM.2017.30.
  43. Rajeev Raman, Venkatesh Raman, and Srinivasa Rao Satti. Succinct dynamic data structures. In Proc. 7th International Workshop on Algorithms and Data Structures (WADS), pages 426-437, 2001. Google Scholar
  44. Rajeev Raman, Venkatesh Raman, and Srinivasa Rao Satti. Succinct indexable dictionaries with applications to encoding k-ary trees, prefix sums and multisets. ACM Transactions on Algorithms, 3(4):43, November 2007. See also SODA 2002. URL: https://doi.org/10.1145/1290672.1290680.
  45. Rajeev Raman and Srinivasa Rao Satti. Succinct dynamic dictionaries and trees. In Proc. 30th International Colloquium on Automata, Languages and Programming (ICALP), pages 357-368, 2003. Google Scholar
  46. Emanuele Viola. New sampling lower bounds via the separator. In Proc. 38th Computational Complexity Conference (CCC), pages 26:1-26:23, 2023. URL: https://doi.org/10.4230/LIPIcs.CCC.2023.26.
  47. Huacheng Yu. Optimal succinct rank data structure via approximate nonnegative tensor decomposition. In Proc. 51st ACM SIGACT Symposium on Theory of Computing (STOC), pages 955-966, 2019. URL: https://doi.org/10.1145/3313276.3316352.
  48. Huacheng Yu. Nearly optimal static Las Vegas succinct dictionary. In Proc. 52nd ACM SIGACT Symposium on Theory of Computing (STOC), pages 1389-1401, 2020. URL: https://doi.org/10.1145/3357713.3384274.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact