Document Open Access Logo

Compressed Index with Construction in Compressed Space

Author Dmitry Kosolobov

PDF

File

Thumbnail PDF
PDF
LIPIcs.CPM.2026.25.pdf
  • Filesize: 1.08 MB
  • 24 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Design and analysis of algorithms
Keywords
  • compressed index
  • pattern matching
  • string complexity
  • grammar
  • block tree

Metrics

Abstract

Suppose that we are given a string s of length n over an alphabet {0,1,…,n^O(1)} and δ is the string complexity of s, a known compression measure. We describe an index on s with O(δlog(n/δ)) space, measured in O(log n)-bit machine words, which can search in s any string of length m in O(m + (occ + 1)log^ε n) time, where occ is the number of occurrences and ε > 0 is any fixed constant (the big-O in the space bound hides factor 1/ε). Crucially, the index can be built in O(n log n) expected time by one left-to-right pass on the string s in a streaming fashion with O(δlog(n/δ)) construction space. The index does not use the Karp-Rabin fingerprints, and the randomization in the construction time can be eliminated by using deterministic dictionaries instead of hash tables (with a slowdown). The search time matches currently best results and the space is almost optimal (the known optimum is O(δlog n/(δα)), where α = log_σ n and σ is the alphabet size, and it coincides with O(δlog(n/δ)) when δ = O(n/α²)). This is the first index that can be constructed within such space and with such time guarantees. To avoid uninteresting marginal cases, all above bounds are stated for δ ≥ Ω(log log n).

Cite As Get BibTex

Dmitry Kosolobov. Compressed Index with Construction in Compressed Space. In 37th Annual Symposium on Combinatorial Pattern Matching (CPM 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 369, pp. 25:1-25:24, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.CPM.2026.25

Author Details

Dmitry Kosolobov
  • St. Petersburg State University, Russia

Funding

Supported by the Russian Science Foundation (RSF), project 24-71-00062.

References

  1. A. R. Adudodla and D. Kempa. Engineering fast and space-efficient recompression from SLP-compressed text. arXiv preprint arXiv:2506.12011, 2025. Google Scholar
  2. S. Alstrup, T. Husfeldt, and T. Rauhe. Marked ancestor problems. In Proc. 39th Annual Symposium on Foundations of Computer Science (Cat. no. 98CB36280), pages 534-543. IEEE, 1998. URL: https://doi.org/10.1109/SFCS.1998.743504.
  3. A. Andersson and M. Thorup. Dynamic ordered sets with exponential search trees. Journal of the ACM (JACM), 54(3):13-es, 2007. URL: https://doi.org/10.1145/1236457.1236460.
  4. D. Belazzougui, P. Boldi, R. Pagh, and S. Vigna. Monotone minimal perfect hashing: searching a sorted table with O(1) accesses. In Proc. 20th ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 785-794. SIAM, 2009. URL: https://doi.org/10.1137/1.9781611973068.86.
  5. D. Belazzougui, P. Boldi, and S. Vigna. Dynamic z-fast tries. In Proc. SPIRE, volume 6393, pages 159-172. Springer, 2010. URL: https://doi.org/10.1007/978-3-642-16321-0_15.
  6. D. Belazzougui, M. Cáceres, T. Gagie, P. Gawrychowski, J. Kärkkäinen, G. Navarro, A. Ordóñez, S. J. Puglisi, and Y. Tabei. Block trees. Journal of Computer and System Sciences, 117:1-22, 2021. URL: https://doi.org/10.1016/j.jcss.2020.11.002.
  7. D. Belazzougui, T. Gagie, P. Gawrychowski, J. Kärkkäinen, A. Ordónez, S. J. Puglisi, and Y. Tabei. Queries on LZ-bounded encodings. In Proc. DCC, pages 83-92. IEEE, 2015. URL: https://doi.org/10.1109/DCC.2015.69.
  8. M. A. Bender, R. Cole, E. D. Demaine, M. Farach-Colton, and J. Zito. Two simplified algorithms for maintaining order in a list. In Algorithms-ESA 2002, volume 2461 of LNCS, pages 152-164. Springer, 2002. URL: https://doi.org/10.1007/3-540-45749-6_17.
  9. M. A. Bender, M. Farach-Colton, J. Kuszmaul, and W. Kuszmaul. Modern hashing made simple. In Proc. the 2024 Symposium on Simplicity in Algorithms (SOSA), pages 363-373. SIAM, 2024. URL: https://doi.org/10.1137/1.9781611977936.33.
  10. P. Bille, I. L. Gørtz, M. B. T. Knudsen, M. Lewenstein, and H. W. Vildhøj. Longest common extensions in sublinear space. In Proc. 26th Annual Symposium on Combinatorial Pattern Matching (CPM), volume 9133 of LNCS, pages 65-76. Springer, 2015. URL: https://doi.org/10.1007/978-3-319-19929-0_6.
  11. P. Bille, I. L. Gørtz, and F. R. Skjoldjensen. Deterministic indexing for packed strings. In Proc. CPM, volume 78 of LIPIcs, pages 6:1-6:11, Dagstuhl, Germany, 2017. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.CPM.2017.6.
  12. O. Birenzwige, S. Golan, and E. Porat. Locally consistent parsing for text indexing in small space. In Proc. 14th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 607-626. SIAM, 2020. URL: https://doi.org/10.1137/1.9781611975994.37.
  13. G. E. Blelloch. Space-efficient dynamic orthogonal point location, segment intersection, and range reporting. In SODA, volume 8, pages 894-903, 2008. Google Scholar
  14. G. S. Brodal, P. Davoodi, and S. S. Rao. Path minima queries in dynamic weighted trees. In Workshop on Algorithms and Data Structures, volume 6844 of LNTCS, pages 290-301. Springer, 2011. URL: https://doi.org/10.1007/978-3-642-22300-6_25.
  15. T. M. Chan, K. G. Larsen, and M. Pătraşcu. Orthogonal range searching on the RAM, revisited. In Proc. 27th Annual Symposium on Computational Geometry (SoCG), pages 1-10. ACM, 2011. URL: https://doi.org/10.1145/1998196.1998198.
  16. A. R. Christiansen and M. B. Ettienne. Compressed indexing with signature grammars. In Proc. 13th Latin American Symposium on Theoretical Informatics (LATIN), volume 10807 of LNCS, pages 331-345. Springer, 2018. URL: https://doi.org/10.1007/978-3-319-77404-6_25.
  17. A. R. Christiansen, M. B. Ettienne, T. Kociumaka, G. Navarro, and N. Prezza. Optimal-time dictionary-compressed indexes. ACM Transactions on Algorithms (TALG), 17(1):1-39, 2020. URL: https://doi.org/10.1145/3426473.
  18. F. Claude and G. Navarro. Self-indexed grammar-based compression. Fundamenta Informaticae, 111(3):313-337, 2011. URL: https://doi.org/10.3233/FI-2011-565.
  19. R. Cole and U. Vishkin. Deterministic coin tossing with applications to optimal parallel list ranking. Information and Control, 70(1):32-53, 1986. URL: https://doi.org/10.1016/S0019-9958(86)80023-7.
  20. K. J. Conrad and P. R. Wilson. Grammatical Ziv-Lempel compression: Achieving PPM-class text compression ratios with LZ-class decompression speed. In Proc. Data Compression Conference (DCC), page 586. IEEE, 2016. URL: https://doi.org/10.1109/DCC.2016.119.
  21. P. Dietz and D. Sleator. Two algorithms for maintaining order in a list. In Proc. 19th annual ACM Symposium on Theory of Computing (STOC), pages 365-372. ACM, 1987. URL: https://doi.org/10.1145/28395.28434.
  22. M. Dietzfelbinger, J. Gil, Y. Matias, and N. Pippenger. Polynomial hash functions are reliable. In Proc. 19th International Colloquium on Automata, Languages and Programming (ICALP), volume 623 of LNCS, pages 235-246. Springer, 1992. URL: https://doi.org/10.1007/3-540-55719-9_77.
  23. T. Gagie, P. Gawrychowski, J. Kärkkäinen, Y. Nekrich, and S. J. Puglisi. LZ77-based self-indexing with faster pattern matching. In Proc. 11th Latin American Symposium on Theoretical Informatics (LATIN), volume 8392 of LNCS, pages 731-742. Springer, 2014. URL: https://doi.org/10.1007/978-3-642-54423-1_63.
  24. T. Gagie, G. Navarro, and N. Prezza. Fully functional suffix trees and optimal text searching in BWT-runs bounded space. Journal of the ACM (JACM), 67(1):1-54, 2020. URL: https://doi.org/10.1145/3375890.
  25. P. Gawrychowski, M. Lewenstein, and P.K. Nicholson. Weighted ancestors in suffix trees. In Proc. 22nd Annual European Symposium on Algorithms (ESA), volume 8737 of LNCS, pages 455-466. Springer, 2014. URL: https://doi.org/10.1007/978-3-662-44777-2_38.
  26. A. Jeż. A really simple approximation of smallest grammar. Theoretical Computer Science, 616:141-150, 2016. URL: https://doi.org/10.1016/j.tcs.2015.12.032.
  27. R. M. Karp and M. O. Rabin. Efficient randomized pattern-matching algorithms. IBM Journal of research and development, 31(2):249-260, 1987. URL: https://doi.org/10.1147/rd.312.0249.
  28. D. Kempa and B. Langmead. Fast and space-efficient construction of AVL grammars from the LZ77 parsing. In Proc. 29th Annual European Symposium on Algorithms (ESA), volume 204 of LIPIcs, pages 56:1-56:14. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2021. URL: https://doi.org/10.4230/LIPIcs.ESA.2021.56.
  29. D. Kempa and N. Prezza. At the roots of dictionary compression: string attractors. In Proc. 50th Annual ACM Symposium on Theory of Computing (STOC), pages 827-840. ACM, 2018. URL: https://doi.org/10.1145/3188745.3188814.
  30. T. Kida, T. Matsumoto, Y. Shibata, M. Takeda, A. Shinohara, and S. Arikawa. Collage system: a unifying framework for compressed pattern matching. Theoretical Computer Science, 298(1):253-272, 2003. URL: https://doi.org/10.1016/S0304-3975(02)00426-7.
  31. J. C. Kieffer and E.-H. Yang. Grammar-based codes: A new class of universal lossless source codes. IEEE Transactions on Information Theory, 46(3):737-754, 2000. URL: https://doi.org/10.1109/18.841160.
  32. T. Kociumaka, G. Navarro, and F. Olivares. Near-optimal search time in δ-optimal space, and vice versa. In Proc. 15th Latin American Symposium on Theoretical Informatics (LATIN), volume 13568 of LNCS, pages 88-103. Springer, 2022. URL: https://doi.org/10.1007/978-3-031-20624-5.
  33. T. Kociumaka, G. Navarro, and F. Olivares. Near-optimal search time in δ-optimal space, and vice versa. Algorithmica, 86(4):1031-1056, 2024. URL: https://doi.org/10.1007/s00453-023-01186-0.
  34. T. Kociumaka, G. Navarro, and N. Prezza. Towards a definitive measure of repetitiveness. In Proc. 14th Latin American Symposium on Theoretical Informatics (LATIN), volume 12118 of LNCS, pages 207-219. Springer, 2020. URL: https://doi.org/10.1007/978-3-030-61792-9_17.
  35. T. Kociumaka, G. Navarro, and N. Prezza. Toward a definitive compressibility measure for repetitive sequences. IEEE Transactions on Information Theory, 69(4):2074-2092, 2022. URL: https://doi.org/10.1109/TIT.2022.3224382.
  36. T. Kopelowitz and M. Lewenstein. Dynamic weighted ancestors. In Proc. 18th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 565-574. ACM-SIAM, 2007. URL: https://doi.org/10.5555/1283383.1283444.
  37. D. Köppl, F. Kurpicz, and D. Meyer. Faster block tree construction. In Proc. 31st Annual European Symposium on Algorithms (ESA), volume 274 of LIPIcs, pages 74:1-74:20, Dagstuhl, Germany, 2023. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.ESA.2023.74.
  38. D. Kosolobov. Compressed index with construction in compressed space. arXiv preprint, 2026. URL: https://arxiv.org/abs/2602.13735.
  39. D. Kosolobov and N. Sivukhin. Construction of sparse suffix trees and LCE indexes in optimal time and space. In Proc. 35th Annual Symposium on Combinatorial Pattern Matching (CPM), volume 296 of LIPIcs, pages 20:1-20:18. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2024. URL: https://doi.org/10.4230/LIPIcs.CPM.2024.20.
  40. K. Mehlhorn, R. Sundar, and C. Uhrig. Maintaining dynamic sequences under equality tests in polylogarithmic time. Algorithmica, 17(2):183-198, 1997. URL: https://doi.org/10.1007/BF02522825.
  41. J. Ian Munro, Gonzalo Navarro, and Yakov Nekrich. Text indexing and searching in sublinear time. In Proc. 31st Annual Symposium on Combinatorial Pattern Matching (CPM), volume 161 of LIPIcs, pages 24:1-24:15, Dagstuhl, Germany, 2020. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.CPM.2020.24.
  42. G. Navarro. A self-index on block trees. In Proc. 24th International Symposium on String Processing and Information Retrieval (SPIRE), volume 10508 of LNCS, pages 278-289. Springer, 2017. URL: https://doi.org/10.1007/978-3-319-67428-5_24.
  43. G. Navarro. Indexing highly repetitive string collections, part i: Repetitiveness measures. ACM Computing Surveys (CSUR), 54(2):1-31, 2021. URL: https://doi.org/10.1145/3434399.
  44. Gonzalo Navarro and Nicola Prezza. Universal compressed text indexing. Theoretical Computer Science, 762:41-50, 2019. URL: https://doi.org/10.1016/j.tcs.2018.09.007.
  45. Y. Nekrich. Orthogonal range searching in linear and almost-linear space. Computational Geometry, 42(4):342-351, 2009. URL: https://doi.org/10.1016/j.comgeo.2008.09.001.
  46. T. Nishimoto, T. I, S. Inenaga, H. Bannai, and M. Takeda. Dynamic index and LZ factorization in compressed space. Discrete Applied Mathematics, 274:116-129, 2020. URL: https://doi.org/10.1016/j.dam.2019.01.014.
  47. S. Raskhodnikova, D. Ron, R. Rubinfeld, and A. Smith. Sublinear algorithms for approximating string compressibility. Algorithmica, 65(3):685-709, 2013. URL: https://doi.org/10.1007/s00453-012-9618-6.
  48. M. Ružić. Constructing efficient dictionaries in close to sorting time. In Proc. International Colloquium on Automata, Languages, and Programming (ICALP), volume 5125 of LNTCS, pages 84-95. Springer, 2008. URL: https://doi.org/10.1007/978-3-540-70575-8_8.
  49. S. C. Sahinalp and U. Vishkin. Symmetry breaking for suffix tree construction. In Proc. 26th Annual ACM Symposium on Theory of Computing (STOC), pages 300-309. ACM, 1994. URL: https://doi.org/10.1145/195058.195164.
  50. Y. Takabatake, T. I, and H. Sakamoto. A space-optimal grammar compression. In Proc. 25th Annual European Symposium on Algorithms (ESA), volume 87 of LIPIcs, pages 67:1-67:15, Dagstuhl, Germany, 2017. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.ESA.2017.67.
  51. F. Zhang, W. Wan, C. Zhang, J. Zhai, Y. Chai, H. Li, and X. Du. CompressDB: Enabling efficient compressed data direct processing for various databases. In Proc. 2022 International Conference on Management of Data, pages 1655-1669, 2022. URL: https://doi.org/10.1145/3514221.3526130.
  52. J. Ziv and A. Lempel. A universal algorithm for sequential data compression. IEEE Transactions on Information Theory, 23(3):337-343, 1977. URL: https://doi.org/10.1109/TIT.1977.1055714.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact