Document Open Access Logo

Text Indexing for Simple Regular Expressions

Authors Hideo Bannai , Philip Bille , Inge Li Gørtz , Gad M. Landau , Gonzalo Navarro , Nicola Prezza , Teresa Anna Steiner , Simon Rumle Tarnow

PDF

File

Thumbnail PDF
PDF
LIPIcs.CPM.2025.20.pdf
  • Filesize: 0.84 MB
  • 16 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Theory of computation → Pattern matching
Keywords
  • Text indexing
  • regular expressions
  • data structures

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

We study the problem of indexing a text T[1..n] ∈ Σⁿ so that, later, given a query regular expression pattern R of size m = |R|, we can report all the occ substrings T[i..j] of T matching R. The problem is known to be hard for arbitrary patterns R, so in this paper, we consider the following two types of patterns. (1) Character-class Kleene-star patterns of the form P₁ D^* P₂, where P₁ and P₂ are strings and D = {c₁, …, c_k} ⊂ Σ is a character-class (shorthand for the regular expression (c₁ | c₂ | ⋯ | c_k)) and (2) String Kleene-star patterns of the form P₁ P^* P₂ where P, P₁ and P₂ are strings. In case (1), we describe an index of O(nlog^{1+ε}n) space (for any constant ε > 0) solving queries in time O(m + log n/log log n + occ) on constant-sized alphabets. We also describe a general solution for any alphabet size. This result is conditioned on the existence of an anchor: a character of P₁P₂ that does not belong to D. We justify this assumption by proving that no efficient indexing solution can exist if an anchor is not present unless the Set Disjointness Conjecture fails. In case (2), we describe an index of size O(n) answering queries in time O(m + (occ+1)log^{ε}n) on any alphabet size.

Cite As Get BibTex

Hideo Bannai, Philip Bille, Inge Li Gørtz, Gad M. Landau, Gonzalo Navarro, Nicola Prezza, Teresa Anna Steiner, and Simon Rumle Tarnow. Text Indexing for Simple Regular Expressions. In 36th Annual Symposium on Combinatorial Pattern Matching (CPM 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 331, pp. 20:1-20:16, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.CPM.2025.20

Author Details

Hideo Bannai
  • M&D Data Science Center, Institute of Integrated Research, Institute of Science Tokyo, Japan
Philip Bille
  • Technical University of Denmark, Lyngby, Denmark
Inge Li Gørtz
  • Technical University of Denmark, Lyngby, Denmark
Gad M. Landau
  • Department of Computer Science, University of Haifa, Israel
Gonzalo Navarro
  • Department of Computer Science, University of Chile, Santiago, Chile
  • Center for Biotechnology and Bioengineering (CeBiB), Santiago, Chile
Nicola Prezza
  • DAIS, Ca' Foscari University of Venice, Venice, Italy
Teresa Anna Steiner
  • University of Southern Denmark, Odense, Denmark
Simon Rumle Tarnow
  • Technical University of Denmark, Lyngby, Denmark

Funding

  • Bannai, Hideo: JSPS KAKENHI Grant Number JP24K02899.
  • Bille, Philip: Danish Research Council grant DFF-8021-002498.
  • Gørtz, Inge Li: Danish Research Council grant DFF-8021-002498.
  • Navarro, Gonzalo: Basal Funds FB0001 and AFB240001, ANID, Chile.
  • Prezza, Nicola: Funded by the European Union (ERC, REGINDEX, 101039208). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them.
  • Steiner, Teresa Anna: Supported by a research grant (VIL51463) from VILLUM FONDEN.

Acknowledgements

Work initiated at Dagstuhl Seminar 24472 "Regular Expressions: Matching and Indexing."

References

  1. Arne Andersson, Torben Hagerup, Stefan Nilsson, and Rajeev Raman. Sorting in linear time? In Proc. 27th STOC, pages 427-436, 1995. URL: https://doi.org/10.1145/225058.225173.
  2. Arturs Backurs and Piotr Indyk. Which regular expression patterns are hard to match? In Proc. 57th FOCS, pages 457-466, 2016. URL: https://doi.org/10.1109/FOCS.2016.56.
  3. Hideo Bannai, Tomohiro I, Shunsuke Inenaga, Yuto Nakashima, Masayuki Takeda, and Kazuya Tsuruta. The "runs" theorem. SIAM J. Comput., 46(5):1501-1514, 2017. URL: https://doi.org/10.1137/15M1011032.
  4. Philip Bille. New algorithms for regular expression matching. In Proc. 33rd ICALP, pages 643-654, 2006. URL: https://doi.org/10.1007/11786986_56.
  5. Philip Bille and Martin Farach-Colton. Fast and compact regular expression matching. Theoret. Comput. Sci., 409:486-496, 2008. URL: https://doi.org/10.1016/J.TCS.2008.08.042.
  6. Philip Bille and Inge Li Gørtz. Substring range reporting. Algorithmica, 69:384-396, 2014. URL: https://doi.org/10.1007/S00453-012-9733-4.
  7. Philip Bille and Inge Li Gørtz. Sparse regular expression matching. In Proc. 35th SODA, pages 3354-3375, 2024. URL: https://doi.org/10.1137/1.9781611977912.120.
  8. Philip Bille, Inge Li Gørtz, Max Rishøj Pedersen, and Teresa Anna Steiner. Gapped indexing for consecutive occurrences. Algorithmica, 85(4):879-901, 2023. URL: https://doi.org/10.1007/S00453-022-01051-6.
  9. Philip Bille, Inge Li Gørtz, Max Rishøj Pedersen, and Teresa Anna Steiner. Gapped indexing for consecutive occurrences. Algorithmica, 85(4):879-901, 2023. URL: https://doi.org/10.1007/S00453-022-01051-6.
  10. Philip Bille, Inge Li Gørtz, Hjalte Wedel Vildhøj, and Søren Vind. String indexing for patterns with wildcards. Theory Comput. Syst., 55(1):41-60, 2014. URL: https://doi.org/10.1007/S00224-013-9498-4.
  11. Philip Bille and Mikkel Thorup. Faster regular expression matching. In Proc. 36th ICALP, pages 171-182, 2009. URL: https://doi.org/10.1007/978-3-642-02927-1_16.
  12. Philip Bille and Mikkel Thorup. Regular expression matching with multi-strings and intervals. In Proc. 21st SODA, 2010. Google Scholar
  13. Karl Bringmann, Allan Grønlund, and Kasper Green Larsen. A dichotomy for regular expression membership testing. In Proc. 58th FOCS, pages 307-318, 2017. URL: https://doi.org/10.1109/FOCS.2017.36.
  14. Richard Cole, Lee-Ad Gottlieb, and Moshe Lewenstein. Dictionary matching and indexing with errors and don't cares. In Proc. 36th STOC, pages 91-100, 2004. URL: https://doi.org/10.1145/1007352.1007374.
  15. Martin Dietzfelbinger and Friedhelm Meyer auf der Heide. A new universal class of hash functions and dynamic hashing in real time. In Proc. 17th ICALP, pages 6-19, 1990. URL: https://doi.org/10.1007/BFB0032018.
  16. Bartłomiej Dudek, Paweł Gawrychowski, Garance Gourdel, and Tatiana Starikovskaya. Streaming regular expression membership and pattern matching. In Proc. 33rd SODA, pages 670-694, 2022. Google Scholar
  17. Minos N Garofalakis, Rajeev Rastogi, and Kyuseok Shim. SPIRIT: Sequential pattern mining with regular expression constraints. In Proc. 25th VLDB, pages 223-234, 1999. URL: http://www.vldb.org/conf/1999/P22.pdf.
  18. Daniel Gibney. An efficient elastic-degenerate text index? not likely. In Proc. 27th SPIRE, pages 76-88, 2020. URL: https://doi.org/10.1007/978-3-030-59212-7_6.
  19. Daniel Gibney and Sharma V. Thankachan. Text indexing for regular expression matching. Algorithms, 14(5), 2021. URL: https://doi.org/10.3390/a14050133.
  20. Isaac Goldstein, Tsvi Kopelowitz, Moshe Lewenstein, and Ely Porat. Conditional lower bounds for space/time tradeoffs. In Proc. 15th WADS, pages 421-436, 2017. URL: https://doi.org/10.1007/978-3-319-62127-2_36.
  21. Costas S. Iliopoulos and M. Sohel Rahman. Indexing factors with gaps. Algorithmica, 55(1):60-70, 2009. URL: https://doi.org/10.1007/S00453-007-9141-3.
  22. Theodore Johnson, S. Muthukrishnan, and Irina Rozenbaum. Monitoring regular expressions on out-of-order streams. In Proc. 23nd ICDE, pages 1315-1319, 2007. URL: https://doi.org/10.1109/ICDE.2007.369001.
  23. Kenrick Kin, Björn Hartmann, Tony DeRose, and Maneesh Agrawala. Proton: multitouch gestures as regular expressions. In Proc. SIGCHI, pages 2885-2894, 2012. URL: https://doi.org/10.1145/2207676.2208694.
  24. S. C. Kleene. Representation of events in nerve nets and finite automata. In C. E. Shannon and J. McCarthy, editors, Automata Studies, Ann. Math. Stud. No. 34, pages 3-41. Princeton U. Press, 1956. Google Scholar
  25. Donald E. Knuth, James H. Morris Jr., and Vaughan R. Pratt. Fast pattern matching in strings. SIAM J. Comput., 6(2):323-350, 1977. URL: https://doi.org/10.1137/0206024.
  26. Roman M. Kolpakov and Gregory Kucherov. Finding maximal repetitions in a word in linear time. In Proc. 40th FOCS, pages 596-604, 1999. URL: https://doi.org/10.1109/SFFCS.1999.814634.
  27. Tsvi Kopelowitz and Robert Krauthgamer. Color-distance oracles and snippets. In Proc. 27th CPM, pages 24:1-24:10, 2016. URL: https://doi.org/10.4230/LIPICS.CPM.2016.24.
  28. Sailesh Kumar, Sarang Dharmapurikar, Fang Yu, Patrick Crowley, and Jonathan Turner. Algorithms to accelerate multiple regular expressions matching for deep packet inspection. In Proc. SIGCOMM, pages 339-350, 2006. URL: https://doi.org/10.1145/1159913.1159952.
  29. Moshe Lewenstein. Indexing with gaps. In Proc. 18th SPIRE, pages 135-143, 2011. URL: https://doi.org/10.1007/978-3-642-24583-1_14.
  30. Moshe Lewenstein, J. Ian Munro, Venkatesh Raman, and Sharma V. Thankachan. Less space: Indexing for queries with wildcards. Theor. Comput. Sci., 557:120-127, 2014. URL: https://doi.org/10.1016/J.TCS.2014.09.003.
  31. Moshe Lewenstein, Yakov Nekrich, and Jeffrey Scott Vitter. Space-efficient string indexing for wildcard pattern matching. In Proc. 31st STACS, pages 506-517, 2014. URL: https://doi.org/10.4230/LIPICS.STACS.2014.506.
  32. Quanzhong Li and Bongki Moon. Indexing and querying XML data for regular path expressions. In Proc. 27th VLDB, pages 361-370, 2001. URL: http://www.vldb.org/conf/2001/P361.pdf.
  33. Makoto Murata. Extended path expressions of XML. In Proc. 20th PODS, pages 126-137, 2001. Google Scholar
  34. E. W. Myers. A four-russian algorithm for regular expression pattern matching. J. ACM, 39(2):430-448, 1992. URL: https://doi.org/10.1145/128749.128755.
  35. G. Navarro and Y. Nekrich. Top-k document retrieval in compressed space. In Proc. 36th SODA, pages 4009-4030, 2025. Google Scholar
  36. Gonzalo Navarro and Veli Mäkinen. Compressed full-text indexes. ACM Comput. Surv., 39(1):2-es, 2007. URL: https://doi.org/10.1145/1216370.1216372.
  37. Gonzalo Navarro and Mathieu Raffinot. Fast and simple character classes and bounded gaps pattern matching, with applications to protein searching. J. Comput. Bio., 10(6):903-923, 2003. URL: https://doi.org/10.1089/106652703322756140.
  38. Yakov Nekrich. New data structures for orthogonal range reporting and range minima queries. arXiv preprint arXiv:2007.11094, 2020. URL: https://arxiv.org/abs/2007.11094.
  39. Yakov Nekrich. New data structures for orthogonal range reporting and range minima queries. In Proc. 32nd SODA, pages 1191-1205, 2021. URL: https://doi.org/10.1137/1.9781611976465.73.
  40. Yakov Nekrich and Gonzalo Navarro. Sorted range reporting. In Proc. 13th SWAT, pages 271-282, 2012. URL: https://doi.org/10.1007/978-3-642-31155-0_24.
  41. Pierre Peterlongo, Julien Allali, and Marie-France Sagot. Indexing gapped-factors using a tree. Int. J. Found. Comput. Sci., 19(1):71-87, 2008. URL: https://doi.org/10.1142/S0129054108005541.
  42. Rajeev Raman, Venkatesh Raman, and S. Srinivasa Rao. Succinct indexable dictionaries with applications to encoding k-ary trees and multisets. In Proc. 13th SODA, pages 233-242, 2002. URL: http://dl.acm.org/citation.cfm?id=545381.545411.
  43. Philipp Schepper. Fine-grained complexity of regular expression pattern matching and membership. In Proc. 28th ESA, 2020. Google Scholar
  44. K. Thompson. Regular expression search algorithm. Commun. ACM, 11:419-422, 1968. URL: https://doi.org/10.1145/363347.363387.
  45. Larry Wall. The Perl Programming Language. Prentice Hall Software Series, 1994. Google Scholar
  46. Fang Yu, Zhifeng Chen, Yanlei Diao, T. V. Lakshman, and Randy H. Katz. Fast and memory-efficient regular expression matching for deep packet inspection. In Proc. ANCS, pages 93-102, 2006. URL: https://doi.org/10.1145/1185347.1185360.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact