Matching Regular-Typed Pattern Languages: Quadratic-Time Algorithms
Abstract
Pattern languages (PAT) are a class of languages generated by expressions called patterns that may contain variables. In a pattern, each variable can be instantiated with an arbitrary string. Typed pattern languages extend PAT by associating a type (constraint) with each variable that restricts the domain of allowed substitutions. In this paper, we study regular-typed PAT (PATwRT), where all types are represented either by a regular expression or by an -NFA. We consider the PATwRT matching problem for patterns with a single repeated variable of the form . We present simple algorithms whose running time is linear in and quadratic in the input length , with polynomial dependence on the sizes of the type representations. Our results extend previous quadratic-time work in two directions: (1) the quadratic-time algorithm for untyped PAT of Fernau et al. (STACS 2015), and (2) the quadratic-time algorithm for the restricted PATwRT , i.e., of Nogami and Terauchi (MFCS 2025).
Keywords and phrases:
Pattern languages, Regular expressions, String algorithms2012 ACM Subject Classification:
Theory of computation Formal languages and automata theory ; Theory of computation Pattern matchingAcknowledgements:
The author is deeply grateful to Soh Kumabe for fruitful discussions during the early stages of this research and for providing valuable comments. The author also thanks the anonymous reviewers for their insightful feedback, which significantly improved the presentation of this paper. In particular, one of the reviewers kindly suggested using an -time algorithm for computing the reflexive transitive closure of a Boolean matrix, which improved the running time of our algorithm for dense -NFAs.Funding:
This work was supported by JST, CREST Grant Number JPMJCR21M3.Editors:
Philip Bille and Nicola PrezzaSeries and Publisher:
Leibniz International Proceedings in Informatics, Schloss Dagstuhl – Leibniz-Zentrum für Informatik
1 Introduction
In this paper, we study the following decision problem for the formalism of regular-typed pattern languages (PATwRT).
Matching Problem for Regular-Typed Pattern Languages
Input 1: An input string .
Input 2: A pattern and a type constraint .
Task: Deciding whether where denotes the language generated by under .
To explain PATwRT, we first recall pattern languages (PAT), since PATwRT is an extension of them. Pattern languages were introduced by Angluin in the context of automata learning theory [4, 5]. In PAT, variables can be used to describe repeated substrings. For example, consider the following pattern:
Over an alphabet , the language generated by consists of all strings obtained by substituting strings for and , respectively; that is,
For example, let and consider an assignment such that and . Then, the instantiated string belongs to . For , the language is neither regular nor context-free (cf. [31, Theorem 8]). In general, the language class PAT is contained in NL, the class of languages recognized by nondeterministic log-space Turing machines [6, 23]. On the other hand, PAT does not include all regular languages. For example, the regular language cannot be generated by any pattern.
To restrict the domains used when assigning strings to variables, typed pattern languages (PATwTy) were introduced by Wright [36] and studied by Koshiba [25] in the context of learning theory. In PATwTy, in addition to pattern expressions, we consider a type constraint that assigns a type to each variable . For example, in the pattern above, if we set , then contains , but not .
If all types are regular languages, we obtain regular-typed pattern languages (PATwRT). Let us consider the following example pattern and type constraint :
where is a separator letter. The generated language under is the following:
Using this language , we can perform pattern matching as follows. To test whether a target string occurs in a text , it suffices to check whether . As this example illustrates, and since PATwRT is strictly more expressive than both REG and PAT, developing faster algorithms for the PATwRT matching problem is of both practical and theoretical interest. Moreover, PAT and PATwRT can be viewed as important fragments of regular expressions with backreferences, REWB [33, 24]. Efficient matching algorithms for PATwRT provide efficient procedures for a non-trivial fragment of REWB and play an important role in practical text processing.
Our Contribution
To state our main results clearly, let us formalize our matching problem and its parameters. For an integer , we consider the matching problem for the pattern
where are distinct variables, and only occurs more than once in . Let be an input string of length . For the type constraints associated with the variables, let be the maximum size of the representations, where denotes the expression size if given by a regular expression , or the number of states if given by an -NFA .
Theorem 1 (Complexity with Regular-Expression Representation).
The matching problem with regular-expression types can be solved in time.
Theorem 2 (Complexity with -NFA Representation).
The matching problem with -NFA types can be solved in time.111We recall that the constant is the exponent of fast Boolean matrix multiplication: given two Boolean matrices and , we can compute their matrix product in time. The current value of is [3].
As we will see in Section 2, our result extends previous work in two directions. We generalize the quadratic-time result of Fernau et al. [12, 13] for untyped pattern languages to the regular-typed setting. We also extend the result of Nogami and Terauchi [29] from the restricted form to the general form considered here.
Organization.
In Section 2, we compare our results with previous work by Fernau et al. [12, 13] and by Nogami and Terauchi [29]. We also review existing results on pattern languages. In Section 3, we introduce the necessary preliminaries. In Section 4, we define and compute our key building block . In Section 5, we introduce and compute our data structure . In Section 6, we combine and to obtain a quadratic-time algorithm for the basic case . In Section 7, we extend this algorithm to the general form .
2 Related Work
Theorem ([13, Theorem 4.2]; [12, Lemma 2]).
The matching problem for is solvable in time, where is the length of the input untyped pattern .
In their notation, denotes the class of patterns with at most one repeated variable. Their result addresses the untyped case of this restriction. The present paper studies the regular-typed matching problem for patterns of the form and shows that, for fixed , it is solvable in quadratic time in . In this sense, our result can be viewed as a regular-typed extension of the quadratic-time result of Fernau et al. [13, 12].
We next review the result of Nogami and Terauchi [29].
Theorem ([29, Theorem 5 (Recap.)]).
The matching problem for REWB of the form , where are regular expressions, can be solved in time, where .
They consider regular expressions with backreferences (REWB) , rather than (regular-typed) pattern languages. However, this expression is equivalent to the following simple instance of PATwRT:
It should be noted that they only consider the very restricted form and do not consider the general case . Since holds for the parameter in their statement, our result extends theirs to the general form. Furthermore, our approach is technically simpler and relies only on elementary tools such as suffix tries.
For REWB, Schmid shows that every fixed REWB expression can be simulated by a two-way multihead finite automaton (2WMFA) [33, Lemma 18]. Since the language class of 2WMFA coincides with NL [19], the language class of REWB is contained in NL. Schmid also gives an automata-theoretic characterization of REWB by introducing memory automata (MFA) [34]. Freydenberger and Schmid consider a deterministic subclass of REWB (DRX) and give an efficient linear-time matching algorithm [17].
It is natural to employ regular languages as types. Similar systems have been studied from several directions [10, 30], [6, Definition 4.7]. It should be noted that Câmpeanu and Yu’s formalization [10] permits nesting of pattern variables, and thus is different from standard typed pattern languages (cf. [10, Example 1]).
The general matching problem – deciding, for an input pattern and a word , if – is NP-complete even without types [4, 11, 22, 27]. This NP-hardness does not contradict the above quadratic-time algorithm by Fernau et al. [13, 12], since it concerns the general matching problem, whereas their (and our) algorithm treats the number of repeating variables as a fixed parameter. Fernau and Schmid [14] refine the NP-completeness by analyzing bounded-parameter restrictions (such as the number of variables and the alphabet size ) and determining when they remain NP-complete or not. Fernau et al. [15] show that many natural parameterizations are -hard. Under ETH (Exponential Time Hypothesis, cf. [20]), they also rule out algorithms with subexponential dependence on the number of variables even when is bounded. For a comprehensive survey on matching patterns with variables, refer to [26].
As mentioned in Introduction, the language class of patterns PAT is incomparable with the language class REG (the class of regular languages) and CFL (the class of context-free languages) [4, Theorem 3.10]. Besides, the language class PAT is contained in NL [6, 23]. Jain et al. studied conditions under which a given pattern belongs to REG (and CFL) [21]. By their results, it can be shown that a pattern forms a non-context-free language [21, Lemma 11] (if ). On the other hand, a similar pattern generates a regular language: indeed, . Reidenbach and Schmid studied the setting or [31]. They showed that the language of a (fixed) pattern varies significantly by changing the alphabet size . For a pattern , (1) if , then is regular; however, (2) if , then is not regular [31, Proposition 9].
3 Preliminaries
Strings and Languages
Let be a finite alphabet. We write for its size. In this paper, we regard as fixed. Hence, is treated as a constant in time complexity.
A word (string) over is a finite sequence of symbols from . The special symbol denotes the empty word. For a word , we write for its length; in particular, . We write (or simply ) for the concatenation of words and . We extend concatenation to sets of words (languages) as follows: for two languages and . We write for the set of all words of length over and write for the set of all words over : with . We write for the -th letter of . We use -based indexing; thus, is the first letter. We write to denote the substring and to denote . In particular, with and with . To denote a prefix, we write instead of . To denote a suffix, we write instead of .
Boolean Matrices over the Boolean Semiring
Let be the Boolean semiring, where 0 and 1 represent false and true respectively, addition is the Boolean-OR , and multiplication is the Boolean-AND .
Let and be Boolean matrices. To denote the Boolean matrix multiplication, we write . As mentioned in Introduction, we can compute in time. In particular, it is known that [3].
We write for the elementwise Boolean OR of and . It is clear that we can compute in time.
Regular Expressions
Regular expressions over are inductively defined by the following grammar:
where is a symbol. We write to denote the set of regular expressions over .
The language is defined inductively in the usual way:
For , we write to denote the size (i.e., the number of symbols) of .
-NFAs (Nondeterministic Finite Automata with -transitions)
An -NFA is a tuple where is a finite set of states, is an input alphabet, is a set of transition rules, is the set of transitions, is the initial state, and is a set of final (accepting) states. We write to denote the number of its states .
For each transition , we write . For , we write . A rule means that we can move from to by consuming the letter . We write if is reachable from by a (possibly empty) sequence of -transitions: . We also write if we can move from to while consuming exactly one letter : i.e., . We naturally extend this notation to arbitrary words as . Then, the language of , denoted , is the set of words accepted by , formally defined as:
By well-known Thompson’s construction, every regular expression can be translated in linear time into a sparse -NFA .
Pattern Languages
Let be a finite alphabet, be a set of variables, disjoint from . A pattern is a string of , . For example, is a pattern where are variables and are symbols. To instantiate the variables in a given pattern, we use assignments . We naturally extend to the monoid homomorphism by letting for every . For example, when and , then the above pattern is instantiated as follows:
The language generated by is
Note that we consider the erasing semantics of pattern languages, where variables can be replaced by the empty word .
Typed and Regular-Typed Pattern Languages
Typed pattern languages are an extension of pattern languages with constraints on variables. In plain pattern languages as defined above, we cannot restrict the domains of words assigned to variables. For example, let us consider generating the language where . However, the plain pattern generates , and in particular .
For practical applications, however, it is often useful to restrict the word domains for variables. To this end, in typed pattern languages, we introduce constraints (types) on the variables. Although the original papers introducing typed pattern languages allow a very general class of types [36, 25], in this paper, we use regular languages as types.
We write for the type assigned to , represented either by a regular expression over or by an -NFA: i.e., is a regular expression or an -NFA . Thus, means the size of the assigned expression or the number of states of .
An assignment is proper with respect to if for all variables occurring in . The language generated by a regular-typed pattern under is
For example, let and , and again consider . Using the type , we obtain , and .
4 Our Key Data Structure
Let be the string used throughout this section, and let denote its length. We assume that the alphabet is fixed in advance and is not part of the input.
In this section, we introduce our key data structure, , which stores, for every (distinct) non-empty substring of , all of its occurrences in sorted order. Our construction is very simple and in fact uses only suffix tries (rather than suffix trees) to build in time.
4.1 Sorted Occurrences and Size Bound
Let (or simply, ) be the set of nonempty substrings of . In this paper, a substring is a contiguous factor of . Namely, for , is a substring of whereas is not.
Notation for Substrings
For any substring of (i.e., ), we introduce the following notation:
-
denotes the set of all the occurrences of in . That is, the set of starting positions with such that .
-
denotes the sorted list of in ascending order, so . In other words, is the sorted version of .
If there is no ambiguity, we simply write and instead of and , respectively.
Example.
The following straightforward proposition bounds the total size of all occurrence lists, which corresponds to the total number of all substring occurrences in .
Proposition 4.
We can show this bound by a simple calculation. The proof can be found in Appendix A.
4.2 Building Our Data Structure in Time
While is just the set of nonempty substrings of , our actual data structure is
We now show how to compute this data structure in using the suffix trie of . Before delving into the formal algorithm, we illustrate our approach.
Illustration of Our Approach.
Our strategy is to augment the standard suffix trie of into a decorated version, where each node is labeled with the sorted occurrence list of the substring it represents. The following example illustrates this target structure. The left tree is the (plain) suffix trie of constructed by our procedure, where is the standard extra terminal symbol used in suffix tries. The right tree is its decorated version, where each non-root node is labeled with . Here denotes the substring represented by node .
Suffix Trie
We briefly recall the standard definition of suffix tries [9, 8]. A suffix trie of is a rooted tree of maximum out-degree representing all suffixes of the string , where is a unique terminal symbol. The suffix trie can be constructed by the following procedure:
-
1.
Initialize to be the empty tree whose root is unlabeled.
-
2.
For every suffix of , traverse from the root using the letters in this order. When traversing, if there is no outgoing edge labeled by the current letter from the current node, add such an edge to a fresh child node.
-
3.
Assign an integer to each leaf node , denoting the starting index of the suffix represented by in .
We write to denote this suffix trie built for . We can construct in time.
Proposition 5 (Known Fact).
The suffix trie can be constructed in time.
For every node of , we write for the string spelled on the path from the root to . Namely, if , then . In particular, for every leaf node labeled with , the string is the suffix of starting at .
Decorated Suffix Trie
We define the decorated suffix trie as the suffix trie augmented with an additional label for every non-root node , satisfying .
Lemma 6.
We can transform into the corresponding in time.
Proof.
We traverse by a DFS in postorder.
When visiting a leaf node of , we simply set .
Next, consider an internal node . Suppose that has children where and the pair means that there exists an edge in . Then, holds.
In this postorder DFS, all lists for have already been computed when is visited. Hence, we build as follows:
Here indeed equals , because every occurrence of in is followed by exactly one character, which determines a unique child . In particular, the lists are pairwise disjoint. Since each is a sorted list, we can efficiently merge them using the well-known -way merge (or -pointer) technique. Because , the time needed for this merging is .
Summing over all internal nodes, the total merge cost (i.e., the running time) is as follows:
For the last equation, we apply Proposition 4 to . Since , this yields .
To build our goal , we only need to visit every internal node except the root node of because the sentinel appears only at the leaves, meaning that the internal nodes (excluding the root) exactly correspond to all nonempty substrings of . This leads to the main result of this section.
Lemma 7.
We can build in time.
5 Data Structure for Range Query
Let be the string used throughout this section and let denote its length.
Let be an -NFA. Recall that is the set of labeled transitions and is the set of -transitions .
In this section, we show an efficient way to compute a Boolean table that satisfies the following: for and ,
Preprocessing.
-
For , we build its adjacency list in time and adjacency matrix in time.
-
For , we build its adjacency list in time and adjacency matrix in time for each .
5.1 Computing for Sparse NFAs
Let be a set of states. We represent as a Boolean array indexed by states, so that membership queries can be answered in time.
We first consider the following two functions:
(1) We can compute in time as follows. We consider the directed graph consisting only of -transitions. Then is exactly the set of vertices reachable from the source set in . Hence, using the adjacency list of , we can compute by a standard multi-source graph search: initialize the queue with all states in , mark them as visited, and traverse only -edges. By the standard linear-time analysis of BFS on adjacency-list graphs [7, Chapter 20], this takes time. (2) We can compute in time by scanning the labeled transition set once and collecting all targets such that with .
We now consider the following function that computes the set :
It can be computed in time. This construction and time bound are standard in the context of efficient -NFA simulation; see, e.g., [2, Section 3.7].
Now we can compute the table in time as follows. We assume all entries of are initialized to .
Proposition 8.
We can compute for in time.
5.2 Computing for Dense NFAs
When the -NFA is dense (i.e., ), the above procedure for requires time. Here, for dense -NFAs, we give an alternative -time algorithm.
Computing in time.
We first compute the -reachability relation as a Boolean matrix in time. Note that computing by running the above for every would take time, which becomes cubic when .
Observe that is the reflexive transitive closure of . Namely, holds. Now, we use the following classical result for efficiently computing . On the Boolean semiring , for a given Boolean matrix , we can compute the reflexive transitive closure of in time [28, 18, 16, 1]. This leads to the following property.
Computing .
Using , we can compute in time as follows.
For each , we pre-compute in time. By definition, iff . We then run the following -time DP algorithm. For notational convenience, we identify with the Boolean matrix whose -entry is .
This procedure leads to the following time complexity.
Proposition 10.
We can compute for in time.
6 Quadratic Time Algorithm for
In this section, we consider the matching problem for the pattern . Namely, given an input string , we decide whether there exists a decomposition
In Section 7, we extend the algorithm presented here to the more general patterns .
Notation
We first set up the notation used in this section. Let be an input string and .
We work uniformly with -NFA representations: for , let denote the -NFA for , and let denote the -NFA for . If a type is given by a regular expression, we first replace it by the equivalent -NFA obtained by Thompson’s construction (Proposition 3).
Let and . In the regular expression case, Thompson’s construction yields . In the -NFA case, by definition, .
For any -NFA , once the table has been computed using Proposition 8 or 10, we define a Boolean table for by:
Given , the table can be built in time by checking, for each , whether there exists with .
We also use half-open interval notation for these tables. For , we write for ; in particular, is the -reachability matrix. Similarly, we write for when . We set iff .
Finally, for substring of , we write for and for . For , we write for .
Preprocessing Cost
Building takes time by Lemma 7.
Once the four tables have been computed, building the corresponding four tables takes the following time in total:
If the input types are given by regular expressions, then by Thompson’s construction (Proposition 3), each type is translated into a sparse -NFA with states and transitions. Hence, by Proposition 8, computing the four -tables takes the following time:
If the input types are given by -NFAs, then by Proposition 10, computing the four -tables takes the following time:
Therefore, using in the regular-expression case and in the -NFA case, we obtain the following time bound.
Proposition 11.
The total preprocessing time is when the types are given by regular expressions, and when they are given by -NFAs.
6.1 Our Procedure
Concentrating on .
We first handle the easy case . If , let be the -NFA obtained by concatenating , where for . Then deciding whether is just the standard membership problem for -NFAs, which can be solved in time by standard -NFA simulation [2, Section 3.7]. If this test succeeds, we return MATCH.
Therefore, in the remainder of this section, we focus on the non-empty case.
Our Strategy.
Our overall strategy is as follows:
-
1.
For each pair , we treat as a candidate for .
-
2.
Check whether in time by picking arbitrary and testing .
-
3.
If , we decide whether there exists a valid decomposition in time. This deciding procedure will be given below (Section 6.2).
The total running time over all candidates is
6.2 Procedure for Deciding whether
For a fixed substring of , we give a decision procedure for deciding whether there exists such a decomposition that runs in time.
Before giving our procedure, we introduce an auxiliary procedure NextPhase as follows.
Input: Cur: -length Boolean vector and : -NFA.
Output: Next: -length Boolean vector that satisfies the following condition called (♠): for every ,
We give a complete implementation of this procedure in Section 6.3. In particular, this procedure runs in time.
Using NextPhase, we build our procedure as follows:
Remark.
For any -NFA , if , iff .
Lemma 12.
returns iff there exists a valid decomposition such that , , and .
Time Complexity.
The above procedure runs in for each fixed substring . Hence, the total time is as (✩). Proposition 11 leads to the following results.
Lemma 13.
The matching problem can be solved in time when the types are given by regular expressions, and in time when they are given by -NFAs.
6.3 Implementation and Properties of NextPhase
A naive implementation of NextPhase checks condition (♠) for every pair with . This takes time, which is slow for our target bound.
To avoid this quadratic dependence on , our approach is based on a monotone two-pointer scan. Let be the -length Boolean vector representing the -closure of the initial state of .
For a fixed target occurrence , after Line 6 the vector already contains the contribution of all already-scanned indices with , advanced from the previous endpoint to the current endpoint . The while-loop then scans newly eligible indices satisfying . Whenever , we add the runs of starting at and ending at . Hence, after the while-loop terminates, represents exactly the states reachable from the initial state of after reading for all already-processed indices with . We formalize this argument by the following invariant.
Invariant
For all states , the following conditions are equivalent:
-
-
there is an index such that
-
–
and and and
-
–
is reachable from the initial state of after reading .
-
–
With the invariant formally defined, we now state the key preservation property of this invariant. This property serves as the cornerstone for proving the overall correctness of our algorithm.
The key point is that Line 6 advances all already represented runs from to , while the while-loop incorporates each newly eligible witness exactly once. The full proof is deferred to Appendix B.
We now use this preservation property to derive the correctness of NextPhase.
Lemma 15.
returns a vector Next that satisfies the following condition: for every ,
Using Lemma 14, we observe that, when the while-loop terminates, no unprocessed index can still satisfy . Hence the invariant captures exactly all valid witnesses. The full proof is deferred to Appendix C.
We next analyze the running time of NextPhase.
Lemma 16.
The time complexity of the procedure is bounded by , where is the number of states of .
The key observation is that the pointer moves only forward, so the body of the while-loop is executed at most times in total. The full amortized analysis is deferred to Appendix D.
7 Quadratic Time Algorithm for General Case
We now extend the construction of Section 6 to the general form . As in Section 6, we first handle the easy case separately: if , we test whether is accepted by the -NFA obtained by concatenating . In the remainder of this section, we therefore focus on the case .
Procedure for a Fixed Candidate
As a representative example, consider the pattern . For a fixed nonempty candidate substring , we extend our FindDecomposition as follows.
By the same argument as in Lemma 12, returns iff there exists a valid decomposition such that for .
The same construction extends immediately to the general pattern : for a fixed candidate ,
-
1.
First, define iff ; and then
-
2.
Next, iteratively set for .
-
3.
Finally, accept iff there exists such that and .
Time Complexity
For a fixed candidate , the procedure performs calls to NextPhase. Hence, by Lemma 16, its running time is where . Summing over all candidates and using , we obtain . Recall that, for a type , if is a regular expression , then is its expression size ; if is an -NFA , then is its number of states .
In the regular-expression case, together with the preprocessing cost for the -NFAs and , this yields . In the -NFA case, together with the preprocessing cost, this yields .
Now we have the following main theorems, which are restated from Introduction.
References
- [1] Alfred V. Aho, John E. Hopcroft, and Jeffrey D. Ullman. The Design and Analysis of Computer Algorithms. Addison-Wesley, USA, 1974. URL: https://dl.acm.org/doi/10.5555/578775.
- [2] Alfred V. Aho, Monica S. Lam, Ravi Sethi, and Jeffrey D. Ullman. Compilers: Principles, Techniques, and Tools (2nd Edition). Addison Wesley, 2006.
- [3] Josh Alman, Ran Duan, Virginia Vassilevska Williams, Yinzhan Xu, Zixuan Xu, and Renfei Zhou. More asymmetry yields faster matrix multiplication. In SODA 2025, pages 2005–2039. SIAM, 2025. doi:10.1137/1.9781611978322.63.
- [4] Dana Angluin. Finding patterns common to a set of strings. J. Comput. Syst. Sci., 21(1):46–62, 1980. doi:10.1016/0022-0000(80)90041-0.
- [5] Dana Angluin. Inductive inference of formal languages from positive data. Inf. Cont., 45(2):117–135, 1980. doi:10.1016/S0019-9958(80)90285-5.
- [6] Henning Bordihn, Jürgen Dassow, and Markus Holzer. Extending regular expressions with homomorphic replacement. RAIRO Theor. Inf. Appl., 44(2):229–255, 2010. doi:10.1051/ita/2010013.
- [7] Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms, 4th edition. MIT Press, 2022. URL: https://mitpress.mit.edu/9780262046305/introduction-to-algorithms/.
- [8] Maxime Crochemore, Christophe Hancart, and Thierry Lecroq. Algorithms on Strings. Cambridge University Press, 2007. doi:10.1017/CBO9780511546853.
- [9] Maxime Crochemore and Wojciech Rytter. Jewels of Stringology. World Scientific, 2002. doi:10.1142/4838.
- [10] Cezar Câmpeanu and Sheng Yu. Pattern expressions and pattern automata. Inf. Process. Lett., 92(6):267–274, 2004. doi:10.1016/j.ipl.2004.09.007.
- [11] Andrzej Ehrenfeucht and Grzegorz Rozenberg. Finding a homomorphism between two words is NP-complete. Inf. Process. Lett., 9(2):86–88, 1979. doi:10.1016/0020-0190(79)90135-2.
- [12] Henning Fernau, Florin Manea, Robert Mercaş, and Markus L. Schmid. Pattern matching with variables: Fast algorithms and new hardness results. In STACS 2015, volume 30 of LIPIcs, pages 302–315. Schloss Dagstuhl, 2015. doi:10.4230/LIPIcs.STACS.2015.302.
- [13] Henning Fernau, Florin Manea, Robert Mercaş, and Markus L. Schmid. Pattern matching with variables: Efficient algorithms and complexity results. ACM Trans. Comput. Theory, 12(1):6:1–6:37, 2020. doi:10.1145/3369935.
- [14] Henning Fernau and Markus L. Schmid. Pattern matching with variables: A multivariate complexity analysis. Inf. Comput., 242:287–305, 2015. doi:10.1016/j.ic.2015.03.006.
- [15] Henning Fernau, Markus L. Schmid, and Yngve Villanger. On the parameterised complexity of string morphism problems. Theory Comput. Syst., 59(1):24–51, 2016. doi:10.1007/s00224-015-9635-3.
- [16] Michael J. Fischer and Albert R. Meyer. Boolean matrix multiplication and transitive closure. In SWAT 1971, pages 129–131. IEEE Computer Society, 1971. doi:10.1109/SWAT.1971.4.
- [17] Dominik D. Freydenberger and Markus L. Schmid. Deterministic regular expressions with back-references. J. Comput. Syst. Sci., 105:1–39, 2019. doi:10.1016/j.jcss.2019.04.001.
- [18] M. E. Furman. Application of a method of rapid multiplication of matrices to the problem of finding the transitive closure of a graph. Doklady Akademii Nauk SSSR, 194(3):524, 1970. URL: https://www.mathnet.ru/eng/dan35686.
- [19] Juris Hartmanis. On non-determinancy in simple computing devices. Acta Informatica, 1(4):336–344, 1972. doi:10.1007/BF00289513.
- [20] Russell Impagliazzo, Ramamohan Paturi, and Francis Zane. Which problems have strongly exponential complexity? J. Comput. Syst. Sci., 63(4):512–530, 2001. doi:10.1006/jcss.2001.1774.
- [21] Sanjay Jain, Yuh Shin Ong, and Frank Stephan. Regular patterns, regular languages and context-free languages. Inf. Process. Lett., 110(24):1114–1119, 2010. doi:10.1016/j.ipl.2010.09.010.
- [22] Tao Jiang, Efim Kinber, Arto Salomaa, Kai Salomaa, and Sheng Yu. Pattern languages with and without erasing. Int. J. Comp. Math., 50(3–4):147–163, 1994. doi:10.1080/00207169408804252.
- [23] Neil D. Jones and Sven Skyum. Recognition of deterministic ETOL languages in logarithmic space. Inf. Control., 35(3):177–181, 1977. doi:10.1016/S0019-9958(77)90058-4.
- [24] Vladimir Komendantsky. Matching problem for regular expressions with variables. In TFP 2012, pages 149–166. Springer, 2013. doi:10.1007/978-3-642-40447-4_10.
- [25] Takeshi Koshiba. Typed pattern languages and their learnability. In EuroCOLT 1995, volume 904 of LNCS, pages 367–379. Springer, 1995. doi:10.1007/3-540-59119-2_192.
- [26] Florin Manea and Markus L. Schmid. Matching patterns with variables. In Robert Mercaş and Daniel Reidenbach, editors, Combinatorics on Words, pages 1–27. Springer, 2019. doi:10.1007/978-3-030-28796-2_1.
- [27] Alexandru Mateescu and Arto Salomaa. Aspects of classical language theory. In Grzegorz Rozenberg and Arto Salomaa, editors, Handbook of Formal Languages, Volume 1: Word, Language, Grammar, pages 175–251. Springer, 1997. doi:10.1007/978-3-642-59136-5_4.
- [28] Ian Munro. Efficient determination of the transitive closure of a directed graph. Inf. Process. Lett., 1(2):56–58, 1971. doi:10.1016/0020-0190(71)90006-8.
- [29] Taisei Nogami and Tachio Terauchi. Efficient matching of some fundamental regular expressions with backreferences. In MFCS 2025, volume 345 of LIPIcs, pages 81:1–81:19. Schloss Dagstuhl, 2025. doi:10.4230/LIPIcs.MFCS.2025.81.
- [30] Dirk Nowotka and Max Wiedenhöft. The equivalence problem of E-pattern languages with length constraints is undecidable. In CPM 2025, volume 331 of LIPIcs, pages 4:1–4:23. Schloss Dagstuhl, 2025. doi:10.4230/LIPIcs.CPM.2025.4.
- [31] Daniel Reidenbach and Markus L. Schmid. Regular and context-free pattern languages over small alphabets. Theor. Comput. Sci., 518:80–95, 2014. doi:10.1016/j.tcs.2013.07.035.
- [32] Jacques Sakarovitch. Elements of Automata Theory. Cambridge University Press, 2009. doi:10.1017/CBO9781139195218.
- [33] Markus L. Schmid. Inside the class of regex languages. Int. J. Found. Comput. Sci., 24(7):1117–1134, 2013. doi:10.1142/S0129054113400340.
- [34] Markus L. Schmid. Characterising REGEX languages by regular languages equipped with factor-referencing. Inf. Comput., 249:1–17, 2016. doi:10.1016/j.ic.2016.02.003.
- [35] Ken Thompson. Regular expression search algorithm. Commun. ACM, 11(6):419–422, 1968. doi:10.1145/363347.363387.
- [36] Keith Wright. Inductive identification of pattern languages with restricted substitutions. In COLT 1990, pages 111–121. Morgan Kaufmann, 1990. URL: http://dl.acm.org/citation.cfm?id=92595.
Appendix A Proof of Proposition 4
Proposition 4. [Restated, see original statement.]
Proof.
Let be an integer with , and write for the set of -length substrings: .
For all positions with , there is a unique -length substring . Thus, the multiset of all occurrences of all -length substrings is in one-to-one correspondence with the set of all possible starting positions . Therefore, holds.
Summing over all lengths , we obtain
Since by our definition, the above equation completes the proof.
Appendix B Proof of Lemma 14
Lemma 14. [Restated, see original statement.]
Proof.
Fix an outer-loop index . We show that holds at the following two evaluation points:
For the base case, consider the first outer-loop iteration . At the first evaluation point (Line 7), Line 6 has already been executed. Since was initialized to the all-zero vector, multiplying it by still yields the zero vector. Besides, . As there is no index , both sides of the equivalence are false for every state .
Next, assume the invariant holds after termination of the while-loop in the -st outer iteration. Then Line 6 multiplies by , which advances every currently represented run from endpoint to endpoint . Moreover, since the previous while-loop terminated only when no further eligible index remained, every index already satisfies . Therefore, at Line 7, all runs already represented in are correctly advanced to the new endpoint , and the corresponding witnesses remain valid. Hence the invariant also holds at the first evaluation point (Line 7) in the -th iteration.
Now assume that the invariant holds before some test of the while-condition, and suppose the condition is true. If , then Line 10 is skipped, is unchanged, and incrementing does not add any new witness; hence the invariant continues to hold. If , then Line 10 adds exactly the states reachable from the initial state of after reading , i.e., it adds precisely the missing witness . After the increment of , the invariant is restored in the form with .
Thus the invariant holds at all claimed program points.
Appendix C Proof of Lemma 15
Lemma 15. [Restated, see original statement.]
returns a vector Next that satisfies the following condition: for every ,
Proof.
First, consider the base case . For , in the right-hand side of , any candidate witness satisfies , hence . Since , we have . Therefore the right-hand side of (♠) is false. Since holds, also holds.
Next, fix an outer-loop index . By Lemma 14, the invariant holds at every evaluation point of the while-loop: namely, immediately before each test of the while-condition (Line 7), and, whenever the loop body is executed, immediately after the increment of (Line 12). Therefore, when the while-loop terminates, the invariant still holds: if the loop body has been executed at least once, this is given by the last occurrence of Line 12; otherwise, it is given by the first occurrence of Line 7.
Observe the termination condition of the while-loop. It terminates when either or . Because the sequence of occurrence indices is strictly increasing, this termination condition guarantees that no index can satisfy . In other words, any index satisfying must strictly satisfy .
Therefore, we can safely drop the restriction from the invariant. It follows that for every state , we have if and only if there exists an index such that , , and is reachable from the initial state of after reading .
At Line 14, the procedure sets if and only if . This intersection is non-empty exactly when there exists an accepting state such that . Combining this with the above equivalence, we obtain:
Together with the base case , this establishes (♠) for all .
Appendix D -time Complexity Analysis of NextPhase
We show the following about .
Lemma 16. [Restated, see original statement.]
The time complexity of the procedure is bounded by , where is the number of states of .
Proof.
Recall that every precomputed can be accessed in time.
We break down the cost of each operation as follows.
1. Initialization Phase.
Lines 2–4 perform the necessary initializations. Initializing the index takes time. The Boolean array states, which acts as a state configuration vector, is initialized to all zeros in time. Similarly, the Boolean array Next of length is initialized to all zeros in time. Thus, the total initialization cost is .
2. Main Loop and Vector-Matrix Multiplications.
The (outer) for-loop iterates times. In each iteration, we perform the following key operations:
-
State Extension (Line 6): We compute the multiplication of a Boolean row vector states of size and a Boolean matrix of size . Using standard matrix multiplication over the Boolean semiring, this operation takes time per iteration. Over the entire for-loop, this contributes to the total running time.
3. Amortized Analysis of the While Loop.
The while-loop incrementally adds the contributions of newly eligible with to . Although it is nested within the for loop, the index variable is initialized to 1 and is only incremented at Line 11. Because is never decreased and can reach at most , the body of the while loop is executed at most times in total across all iterations of the outer for loop. The total number of evaluations of the while-condition is also , which does not affect the final bound.
Inside the while loop, the array lookup takes time. The dominant operation is Line 10, where we again multiply the Boolean vector by the precomputed matrix and compute the element-wise Boolean OR () with the current states array. The vector-matrix multiplication takes time, and the Boolean OR takes time. Since this inner body runs at most times globally, the aggregated time spent inside the while loop over the entire procedure is strictly bounded by .
Summing the costs of initialization, the outer loop operations, and the globally bounded inner loop operations, the overall time complexity is:
