Document Open Access Logo

Dynamic Grammar-Compressed Self-Index in δ-Optimal Space

Authors Takaaki Nishimoto , Yasuo Tabei

PDF

File

Thumbnail PDF
PDF
LIPIcs.ESA.2026.6.pdf
  • Filesize: 1.3 MB
  • 24 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Pattern matching
Keywords
  • Compressed string indexes
  • grammar-compression
  • dynamic data structures

Metrics

Abstract

A compressed self-index stores a string in compressed form while supporting locate queries without decompression. For highly repetitive strings - arising in web crawls, versioned documents, and genomic collections - static self-indexes can match the δ-optimal lower bound of Ω(δ log(n log σ / (δ log n)) log n) bits up to constant factors, where n is the string length, σ is the alphabet size, and δ is the substring complexity. Their dynamic counterparts, however, remain scarce: every existing dynamic self-index either fails to attain δ-optimal space, pays Ω(log n) time per reported occurrence during locate, or has an update time that grows with the maximum value in the longest common prefix (LCP) array of the text. We present the dynamic RR-index, a dynamic grammar-compressed self-index built on the restricted recompression run-length straight-line program (RLSLP). To our knowledge, it is the first dynamic self-index to attain δ-optimal space. The index occupies expected 𝒪(δ log(n log σ / (δ log n)) log n) bits, answers locate queries in expected 𝒪(m + log m log² n + occ (log n / log log n)) time - where m is the pattern length and occ is the number of occurrences - and supports insertion of a length-m' string and deletion of a length-m' substring in expected amortized 𝒪(m' log² n + log³ n) time, with no dependence on the maximum LCP value. On eleven highly repetitive corpora, including a 37 GB Wikipedia dump and a 59 GB human-chromosome collection, the dynamic RR-index is up to 77× faster than the dynamic r-index on updates and up to 11× faster than other dynamic indexes on locate.

Cite As Get BibTex

Takaaki Nishimoto and Yasuo Tabei. Dynamic Grammar-Compressed Self-Index in δ-Optimal Space. In 34th Annual European Symposium on Algorithms (ESA 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 388, pp. 6:1-6:24, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.ESA.2026.6

Author Details

Takaaki Nishimoto
  • RIKEN Center for Advanced Intelligence Project, Tokyo, Japan
Yasuo Tabei
  • RIKEN Center for Advanced Intelligence Project, Tokyo, Japan

Funding

This work was supported by JST, ACT-X Grant Number JPMJAX24CL, Japan.

Supplementary Materials

References

  1. Stephen Alstrup, Gerth Stølting Brodal, and Theis Rauhe. Pattern matching in dynamic texts. In Proceedings of the Eleventh Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 819-828, 2000. URL: https://dl.acm.org/doi/10.5555/338219.338645.
  2. Amos Beimel, Haim Kaplan, Yishay Mansour, Kobbi Nissim, Thatchaphol Saranurak, and Uri Stemmer. Dynamic algorithms against an adaptive adversary: generic constructions and lower bounds. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing (STOC), pages 1671-1684, 2022. URL: https://doi.org/10.1145/3519935.3520064.
  3. Djamal Belazzougui, Manuel Cáceres, Travis Gagie, Pawel Gawrychowski, Juha Kärkkäinen, Gonzalo Navarro, Alberto Ordóñez Pereira, Simon J. Puglisi, and Yasuo Tabei. Block trees. Journal of Computer and System Sciences, 117:1-22, 2021. URL: https://doi.org/10.1016/j.jcss.2020.11.002.
  4. Nico Bertram, Johannes Fischer, and Lukas Nalbach. Move-r: Optimizing the r-index. In Proceedings of 22nd International Symposium on Experimental Algorithms (SEA), pages 1:1-1:19, 2024. URL: https://doi.org/10.4230/LIPIcs.SEA.2024.1.
  5. Guy E. Blelloch. Space-efficient dynamic orthogonal point location, segment intersection, and range reporting. In Proceedings of the Nineteenth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2008. URL: https://dl.acm.org/doi/10.5555/1347082.1347180.
  6. Michael Burrows and David J. Wheeler. A block-sorting lossless data compression algorithm. Technical Report SRC-RR-124, Digital Equipment Corporation, Systems Research Center, 1994. Google Scholar
  7. Anders Roy Christiansen and Mikko Berggren Ettienne. Compressed indexing with signature grammars. In Proceedings of the 13th Latin American Symposium on Theoretical Informatics (LATIN), pages 331-345, 2018. URL: https://doi.org/10.1007/978-3-319-77404-6_25.
  8. Anders Roy Christiansen, Mikko Berggren Ettienne, Tomasz Kociumaka, Gonzalo Navarro, and Nicola Prezza. Optimal-time dictionary-compressed indexes. ACM Transactions on Algorithms, 17:8:1-8:39, 2021. URL: https://doi.org/10.1145/3426473.
  9. Francisco Claude, Gonzalo Navarro, and Alberto Ordóñez Pereira. The wavelet matrix: An efficient wavelet tree for large alphabets. Information Systems, 47:15-32, 2015. URL: https://doi.org/10.1016/J.IS.2014.06.002.
  10. Paul F. Dietz. Optimal algorithms for list indexing and subset rank. In Proceedings of the Workshop on Algorithms and Data Structures (WADS), pages 39-46, 1989. URL: https://doi.org/10.1007/3-540-51542-9_5.
  11. Paul F. Dietz and Daniel Dominic Sleator. Two algorithms for maintaining order in a list. In Proceedings of the Nineteenth Annual ACM Symposium on Theory of Computing (STOC), pages 365-372, 1987. URL: https://doi.org/10.1145/28395.28434.
  12. Martin Dietzfelbinger, Anna R. Karlin, Kurt Mehlhorn, Friedhelm Meyer auf der Heide, Hans Rohnert, and Robert Endre Tarjan. Dynamic perfect hashing: Upper and lower bounds. SIAM Journal on Computing, 23:738-761, 1994. URL: https://doi.org/10.1137/S0097539791194094.
  13. Anouk Duyster and Tomasz Kociumaka. Logarithmic-time internal pattern matching queries in compressed and dynamic texts. Theory of Computing Systems, 70(1):11, 2026. URL: https://doi.org/10.1007/s00224-026-10266-x.
  14. Andrzej Ehrenfeucht, Ross M. McConnell, and Sung-Whan Woo. Contracted suffix trees: A simple and dynamic text indexing data structure. In Proceedings of the 20th Annual Symposium on Combinatorial Pattern Matching (CPM), pages 41-53, 2009. URL: https://doi.org/10.1007/978-3-642-02441-2_5.
  15. Paolo Ferragina and Giovanni Manzini. On compressing the textual web. In Proceedings of the 3rd Web Search and Data Mining (WSDM), pages 391-400, 2010. URL: https://doi.org/10.1145/1718487.1718536.
  16. Travis Gagie, Gonzalo Navarro, and Nicola Prezza. Fully functional suffix trees and optimal text searching in BWT-runs bounded space. Journal of the ACM, 67, 2020. URL: https://doi.org/10.1145/3375890.
  17. Pawel Gawrychowski, Adam Karczmarz, Tomasz Kociumaka, Jakub Lacki, and Piotr Sankowski. Optimal dynamic strings. CoRR, abs/1511.02612, 2015. URL: https://doi.org/10.48550/arXiv.1511.02612.
  18. Pawel Gawrychowski, Adam Karczmarz, Tomasz Kociumaka, Jakub Lacki, and Piotr Sankowski. Optimal dynamic strings. In Proceedings of the 2018 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 1509-1528, 2018. URL: https://doi.org/10.1137/1.9781611975031.99.
  19. Torben Hagerup. Sorting and searching on the word RAM. In Proceedings of the 15th Annual Symposium on Theoretical Aspects of Computer Science (STACS), pages 366-398, 1998. URL: https://doi.org/10.1007/BFb0028575.
  20. Tomohiro I. Longest common extensions with recompression. In Proceedings of the 28th Annual Symposium on Combinatorial Pattern Matching (CPM), pages 18:1-18:15, 2017. URL: https://doi.org/10.4230/LIPIcs.CPM.2017.18.
  21. Artur Jez. Faster fully compressed pattern matching by recompression. ACM Transactions on Algorithms, 11:20:1-20:43, 2015. URL: https://doi.org/10.1145/2631920.
  22. Marek Karpinski, Wojciech Rytter, and Ayumi Shinohara. An efficient pattern-matching algorithm for strings with short descriptions. Nordic Journal of Computing, 4:172-186, 1997. Google Scholar
  23. Dominik Kempa and Tomasz Kociumaka. Resolution of the Burrows-Wheeler transform conjecture. In Proceedings of the 61st IEEE Annual Symposium on Foundations of Computer Science (FOCS), pages 1002-1013, 2020. URL: https://doi.org/10.1109/FOCS46700.2020.00097.
  24. Dominik Kempa and Tomasz Kociumaka. Dynamic suffix array with polylogarithmic queries and updates. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing (STOC), pages 1657-1670, 2022. URL: https://doi.org/10.1145/3519935.3520061.
  25. Dominik Kempa and Tomasz Kociumaka. Collapsing the hierarchy of compressed data structures: Suffix arrays in optimal compressed space. CoRR, abs/2308.03635, 2023. URL: https://doi.org/10.48550/arXiv.2308.03635.
  26. Dominik Kempa and Tomasz Kociumaka. Collapsing the hierarchy of compressed data structures: Suffix arrays in optimal compressed space. In Proceedings of the 64th IEEE Annual Symposium on Foundations of Computer Science (FOCS), pages 1877-1886, 2023. URL: https://doi.org/10.1109/FOCS57990.2023.00114.
  27. Dominik Kempa and Nicola Prezza. At the roots of dictionary compression: String attractors. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing (STOC), pages 827-840, 2018. URL: https://doi.org/10.1145/3188745.3188814.
  28. Tomasz Kociumaka, Gonzalo Navarro, and Francisco Olivares. Near-optimal search time in δ-optimal space, and vice versa. Algorithmica, 86(4):1031-1056, 2024. URL: https://doi.org/10.1007/s00453-023-01186-0.
  29. Tomasz Kociumaka, Gonzalo Navarro, and Nicola Prezza. Toward a definitive compressibility measure for repetitive sequences. IEEE Transactions on Information Theory, 69:2074-2092, 2023. URL: https://doi.org/10.1109/TIT.2022.3224382.
  30. Tomasz Kociumaka, Jakub Radoszewski, Wojciech Rytter, and Tomasz Walen. Optimal data structure for internal pattern matching queries in a text and applications. CoRR, abs/1311.6235, 2013. URL: https://doi.org/10.48550/arXiv.1311.6235.
  31. Dmitry Kosolobov. Compressed index with construction in compressed space. CoRR, abs/2602.13735, 2026. URL: https://doi.org/10.48550/arXiv.2602.13735.
  32. Abraham Lempel and Jacob Ziv. On the complexity of finite sequences. IEEE Transactions on Information Theory, 22(1):75-81, 1976. URL: https://doi.org/10.1109/TIT.1976.1055501.
  33. Martine Léonard, Laurent Mouchard, and Mikaël Salson. On the number of elements to reorder when updating a suffix array. Journal of Discrete Algorithms, 11:87-99, 2012. URL: https://doi.org/10.1016/j.jda.2011.01.002.
  34. Shirou Maruyama, Masaya Nakahara, Naoya Kishiue, and Hiroshi Sakamoto. ESP-index: A compressed index based on edit-sensitive parsing. Journal of Discrete Algorithms, 18:100-112, 2013. URL: https://doi.org/10.1016/j.jda.2012.07.009.
  35. Gonzalo Navarro. Wavelet trees for all. Journal of Discrete Algorithms, 25:2-20, 2014. URL: https://doi.org/10.1016/j.jda.2013.07.004.
  36. Takaaki Nishimoto, Tomohiro I, Shunsuke Inenaga, Hideo Bannai, and Masayuki Takeda. Fully dynamic data structure for LCE queries in compressed space. In Proceedings of the 41st International Symposium on Mathematical Foundations of Computer Science (MFCS), pages 72:1-72:14, 2016. URL: https://doi.org/10.4230/LIPIcs.MFCS.2016.72.
  37. Takaaki Nishimoto, Tomohiro I, Shunsuke Inenaga, Hideo Bannai, and Masayuki Takeda. Dynamic index and LZ factorization in compressed space. Discrete Applied Mathematics, 274:116-129, 2020. URL: https://doi.org/10.1016/j.dam.2019.01.014.
  38. Takaaki Nishimoto and Yasuo Tabei. Dynamic RLSLP Index. Software, (visited on 2026-08-13). URL: https://github.com/TNishimoto/dynamic-rlslp-index
    Software Heritage Logo archived version
    full metadata available at: https://doi.org/10.4230/artifacts.27657
  39. Takaaki Nishimoto and Yasuo Tabei. Optimal-time queries on BWT-runs compressed indexes. In Proceedings of the 48th International Colloquium on Automata, Languages, and Programming (ICALP), pages 101:1-101:15, 2021. URL: https://doi.org/10.4230/LIPIcs.ICALP.2021.101.
  40. Takaaki Nishimoto and Yasuo Tabei. Dynamic suffix array in optimal compressed space. CoRR, abs/2404.07510, 2024. URL: https://doi.org/10.48550/arXiv.2404.07510.
  41. Takaaki Nishimoto and Yasuo Tabei. Dynamic grammar-compressed self-index in δ-optimal space. CoRR, abs/2604.24080, 2026. URL: https://doi.org/10.48550/arXiv.2604.24080.
  42. Takaaki Nishimoto and Yasuo Tabei. Dynamic r-index: An updatable self-index for highly repetitive strings. In Proceedings of Data Compression Conference (DCC), pages 401-410, 2026. URL: https://doi.org/10.1109/DCC66757.2026.00048.
  43. Takaaki Nishimoto, Yoshimasa Takabatake, and Yasuo Tabei. A compressed dynamic self-index for highly repetitive text collections. Information and Computation, 273:104518, 2020. URL: https://doi.org/10.1016/j.ic.2020.104518.
  44. Pizza&Chili repetitive corpus. URL: http://pizzachili.dcc.uchile.cl/repcorpus.html.
  45. Molly Przeworski, Richard R. Hudson, and Anna Di Rienzo. Adjusting the focus on human variation. Trends in Genetics, 16:296-302, 2000. Google Scholar
  46. Sofya Raskhodnikova, Dana Ron, Ronitt Rubinfeld, and Adam D. Smith. Sublinear algorithms for approximating string compressibility. Algorithmica, 65:685-709, 2013. URL: https://doi.org/10.1007/s00453-012-9618-6.
  47. Mikaël Salson, Thierry Lecroq, Martine Léonard, and Laurent Mouchard. Dynamic extended suffix arrays. Journal of Discrete Algorithms, 8:241-257, 2010. URL: https://doi.org/10.1016/j.jda.2009.02.007.
  48. The 1000 Genomes Project Consortium. An integrated map of genetic variation from 1,092 human genomes. Nature, 491:56-65, 2012. Google Scholar
  49. Enwiki dump progress on 20241201: All pages with complete edit history (xml-p1p812). URL: https://dumps.wikimedia.org/enwiki/.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact