Abstract 1 Introduction 2 Related Work 3 Technical Overview of Theorem 7 4 Hardness References

On the Complexity of the Matching Problem of
Regular Expressions with Backreferences

Soh Kumabe ORCID CyberAgent, Inc., Tokyo, Japan    Yuya Uezato ORCID CyberAgent, Inc., Tokyo, Japan
National Institute of Informatics, Tokyo, Japan
Abstract

Regular Expression Denial of Service (ReDoS) is a well-known type of algorithmic complexity attack, where an adversary supplies maliciously crafted strings to a regular expression matching engine, aiming to exhaust computational resources of systems. Even quadratic-time behavior in matching engines has been exploited in successful attacks, as exemplified by major outages at Stack Overflow (2016) and Cloudflare (2019). These incidents motivate a fundamental question: Is it possible to construct matching engines that run in linear or near-linear time in the length of the input string? For classical regular expressions (REGEX), Thompson’s construction yields a linear-time algorithm for fixed expressions. However, practical engines support powerful features such as backreferences, which allow capturing a substring and reusing it later. This feature strictly extends the expressive power of REGEX but unfortunately increases the risk of ReDoS attacks.

This paper investigates the fine-grained complexity of the string matching problem for regular expressions with backreferences (REWBs). Specifically, we consider r-use k-REWBs, i.e., REWBs with k variables such that, in any computation, the total number of backreference executions is at most r. On the hardness side, we show that the string matching problem for k-REWBs cannot be solved in O(n2kϵ) time for any ϵ>0 under the Strong Exponential Time Hypothesis (SETH), where n is the length of the input string. We also prove that this problem is W[2]-hard when parameterized by the length of the REWB expression, strengthening the previous W[1]-hardness result. Moreover, we prove that this problem for 2-use 2-REWBs cannot be solved in n1+o(1) time unless the triangle detection problem can be solved in that time. On the algorithmic side, we present an O(nlog2n)-time algorithm for 1-use REWBs. In particular, we focus on the ABCBD problem, which is the REWB matching problem for the form A(B)xC\xD where A, B, C, and D are fixed REGEXes. We also show that every 1-use REWB can be transformed into this canonical form. Our algorithm significantly improves upon the recent O(n2)-time algorithm for the ABCBD problem by Nogami and Terauchi (MFCS, 2025). Our algorithm is highly nontrivial and employs several techniques, including suffix trees, transition monoids of REGEXes, factorization forest data structures, and periodicity of strings.

Keywords and phrases:
Pattern Matching, Regular Expression, Backreference, Fine-grained Complexity
Category:
Track A: Algorithms, Complexity and Games
Funding:
Yuya Uezato: Yuya Uezato was supported by JST, CREST Grant Number JPMJCR21M3.
Copyright and License:
[Uncaptioned image] © Soh Kumabe and Yuya Uezato; licensed under Creative Commons License CC-BY 4.0
2012 ACM Subject Classification:
Theory of computation Pattern matching
; Theory of computation Formal languages and automata theory
Related Version:
Full Version: https://arxiv.org/abs/2605.07289 [53]
Acknowledgements:
We are grateful to the anonymous reviewers for their careful reading and valuable comments. We are especially indebted to the anonymous reviewers of an earlier version of this work, whose detailed and constructive feedback led to substantial improvements in the technical exposition, the organization, and the overall clarity of the paper.
Editors:
Sayan Bhattacharya, Danupon Nanongkai, Michael Benedikt, and Gabriele Puppis

1 Introduction

Background.

Regular expressions are widely used in web services to validate and filter user-provided text. However, depending on the engine implementation and the specific expressions used, the worst-case regular expression matching time can be prohibitive. In most web-service settings, expressions are fixed by the application, while the input string is provided by potentially untrusted users. In such cases, an adversary can craft inputs that trigger the worst-case behavior, potentially exhausting computational resources (CPU and memory) and causing service outages.

Attacks that exploit such worst-case algorithmic behavior are known as algorithmic complexity attacks [21]. Those specifically involving regular expressions are called ReDoS (Regular Expression Denial of Service) [20]. ReDoS is a major vulnerability and is cataloged in the Common Weakness Enumeration (CWE, the de facto standard for vulnerability classification) as CWE-1333 [58].

We highlight two well-documented service outages caused by ReDoS, together with the real-world regular expression that triggered the worst-case behavior in the deployed engines.111These two examples are written in the syntax of the corresponding real-world regex engines, not in the formal syntax introduced below. For example, \s is a whitespace character class and \u200c is the Unicode escape for U+200C. These examples are used only to demonstrate that backtracking engines can already cause quadratic-time ReDoS without backreferences.

  1. 1.

    The 2016 Stack Overflow outage, caused by the expression
    ˆ[\s\u200c]+|[\s\u200c]+$ [78].

  2. 2.

    The 2019 Cloudflare outage, caused by an expression whose core part is .*.*=.* [43].

According to their reports [78, 43], these outages occurred because the employed regular expression engines relied on backtracking algorithms. Consequently, the above expressions exhibited quadratic-time computation on crafted inputs. It is well known that backtracking-based engines can cause catastrophic exponential backtracking [36, 16, 39], and it is widely believed that avoiding exponential behavior would be sufficient. However, these incidents were surprising in showing that even quadratic-time behavior can lead to a system outage. This is stated explicitly in the post-mortem [78], which we quote verbatim below:

So the Regex engine has to perform a “character belongs to a certain character class” check (plus some additional things) 20,000+19,999+19,998+…+3+2+1 = 199,990,000 [sic] times, and that takes a while. This is not classic catastrophic backtracking (performance is 𝐎(𝐧2), not exponential, in length), but it was enough.

Motivated by this background, we propose the following criterion:

A regular expression engine is ReDoS-safe if, for each fixed expression, it runs in linear or near-linear time in the length of the input string.

In this paper, we address the following fundamental questions: Can we construct a ReDoS-safe regular expression engine in the above sense? If so, which practical extensions can such an engine still support?

Complexity of REGEX matching problems

The term “regular expression” is used for several different formalisms. In this paper, following classical formal language theory, we use the standard (Kleene) regular expressions, denoted by REGEX.

Definition 1 (Syntax of REGEX).

The syntax of REGEX over an alphabet Σ is given by the grammar:

E::=ϵσE+EEEEwhere σΣ.

We give the formal semantics, including the language 𝕃(), in the full version [53]. For REGEX, we consider the following computational problem.

REGEX Matching Problem

  Fixed Object: A REGEX E.

Input: A string w.

Task: Deciding if w𝕃(E) where 𝕃(E) is the language of E.

Is it possible to design a ReDoS-safe engine for the REGEX matching problem? The answer is Yes. Using Thompson’s construction [80] and the standard efficient NFA simulation [16][3, Sec 3.7], we can solve this problem in O(|E||w|) time, which is linear in |w| for each fixed E.

From REGEX to REWB

Does the linear-time algorithm for REGEX matching fully resolve ReDoS in practice? The answer is No. This is because most real-world regular expression engines support extensions beyond REGEX. One of the most prominent extensions is backreferences. We use REWB to denote the class of regular expressions with backreferences. Unlike REGEX, REWB allows variables that capture substrings matched by a subexpression and refer back to that captured string later. Backreferences are widely supported in the standard regular expression libraries of major programming languages (e.g., C++, Java, JavaScript, Python, Ruby, Perl, PHP, and OCaml). The syntax of REWB is given as follows.

Definition 2 (Syntax of REWB).

The syntax of REWB is given by the following grammar:

E::=ϵσE+EEEE(E)x\x,

where σΣ and x is a variable. The expression (E)x captures the substring matched by E and stores it in x, while \x refers to the content stored in x.

As with REGEX, we give the formal semantics of REWB in the full version [53].

We note that REWB is strictly more expressive than REGEX. Consider the expression E:=((0+1))x\x. Intuitively, the subexpression ((0+1))x matches an arbitrary string w{0,1} and binds it to x. Then, \x matches the captured substring stored in x. Hence, E defines the non-regular language {ww:w{0,1}}.

This paper focuses on the following problem.

REWB Matching Problem

  Fixed Object: A REWB E.

Input: A string w.

Task: Deciding if w𝕃(E).

This raises a natural question: Is it possible to design a ReDoS-safe engine for the REWB matching problem? Unfortunately, No. If we view E as part of the input and parameterize the problem by c, the number of variable occurrences in E, then the REWB matching problem is W[1]-hard [79, 33]. In particular, under the standard assumption 𝐅𝐏𝐓W[1], there is no uniform algorithm running in f(c)|w|O(1) for all inputs (w,E). However, this worst-case hardness is too pessimistic for practice. In practice, many real-world REWB expressions use backreferences in restricted ways. To identify which REWB subclasses are ReDoS-safe and which remain vulnerable, we need a fine-grained analysis of the REWB matching problem.

Classification of REWB

We write k-REWB to denote the set of REWBs that have at most k distinct variables. We say that an expression E is r-use if, for every input string w and for all computations of E on w, the total number of executed backreferences (i.e., executed occurrences of \x) is at most r. Intuitively, a computation is one possible run of E on w; different runs may arise from different choices at the union and Kleene-star subexpressions of E, and may therefore execute different numbers of backreferences. If no such finite r exists, we call E ω-use. We illustrate these notions with the following examples:

  • (Σ)x\x is a 1-use 1-REWB because there is only one variable x, which is referenced once.

  • (Σ)x\x\x is a 2-use 1-REWB because there is only one variable x, which is referenced twice.

  • (Σ)x\x(Σ)x\x is also a 2-use 1-REWB that reuses the single variable x.

  • (Σ)x(\x) is an ω-use 1-REWB, as x can be referenced an unbounded number of times.

  • (Σ)x(Σ)y\y\x is a 2-use 2-REWB because it contains two variables x and y, which are referenced a total of two times.

We allow both capture groups and backreferences to appear inside capture groups. That is, the body of a capture group ()x may contain subexpressions of the form \y or ()y, where y may be equal to x. For example, (\x\x)x (“doubling the content of x”) and (0\x 0)x (“wrapping x with 0s”) are well-formed.

Known algorithmic results for REWB matching

The current best-known algorithm for k-REWB matching runs in O(|E||w|1+2k) time [25, §IX. C][70] based on standard dynamic programming. Recently, Nogami and Terauchi studied the ABCBD problem, a subclass of the 1-REWB matching problem, and gave an O(|E|2|w|2)-time algorithm [70, Thm. 5]. Although their result improved upon the standard DP-based O(|E||w|3)-time algorithm, it is not ReDoS-safe due to its quadratic dependence on |w|.

1.1 Our Contribution

We summarize our main hardness and algorithmic results.

1.1.1 Hardness Results for the REWB Matching Problem

We strengthen the previously known W[1]-hardness results [79, 33] by showing W[2]-hardness.

Theorem 3.

When the expression E is part of the input, the REWB matching problem is 𝐖[𝟐]-hard parameterized by the expression size |E|.

Assuming that Triangle Detection on an m-edge graph does not admit an m1+δo(1)-time algorithm [1, Conjecture 3], we derive a conditional lower bound even for the highly restricted class, 2-use 2-REWB.

Theorem 4.

Assuming the no-almost-linear-time hypothesis for Triangle Detection, there exists a fixed 2-use 2-REWB expression Ψ such that the matching problem for Ψ cannot be solved in |w|1+δo(1) time for some constant δ>0.

Under the k-Orthogonal Vectors hypothesis (𝒌-OV) [84], which is a weaker assumption than the strong exponential time hypothesis (SETH), we have the following result.

Theorem 5.

Assuming the 𝐤-OV hypothesis, for each fixed integer k1, there exists a fixed ω-use k-REWB expression Ψk such that the matching problem for Ψk cannot be solved in O(|w|2kϵ) time for any ϵ>0.

Very recently and independently of our work, Nogami and Terauchi [71, Theorem 1] proved the following lower bound: assuming the 2-OV hypothesis, no algorithm solves the 1-REWB matching problem in O(|w|2ϵpoly(|E|)) time. Our result strengthens theirs in the following two ways: (1) our hardness holds for general k; and (2) as our hardness holds for a fixed expression Ψk, it rules out any speedup gained by allowing arbitrary dependence on |E|.

1.1.2 Algorithmic Results

We have shown that the REWB matching problem remains hard even for 2-use 2-REWBs (Theorem 4) and for (ω-use) 1-REWBs (Theorem 5 with k=1) when seeking a near-linear-time algorithm. This naturally raises the question: What is the complexity of 1-use REWB matching? Our main algorithmic result is a near-linear-time algorithm for the following special case:

ABCBD Problem

  Fixed Objects: REGEXes A, B, C, and D.

Input: A string w.

Task: Deciding if we can decompose w into w=wAwBwCwBwD so that wA𝕃(A), wB𝕃(B), wC𝕃(C), and wD𝕃(D). Equivalently, decide whether w𝕃(A(B)xC\xD).

At first glance, this syntax may appear highly restricted. However, the ABCBD problem captures the general case of 1-use REWB in the following sense:

Theorem 6.

For any 1-use REWB E, there exists an equivalent REWB of the following form:

(A1(B1)xC1\xD1)+(A2(B2)xC2\xD2)++(AK(BK)xCK\xDK)+E

where Ai, Bi, Ci, Di, E are REGEXes. We can construct the above representation in O(|E|4) time. Furthermore, the following holds for their size:
K=O(|E|2),|Ai|=O(|E|2),|Bi|=O(|E|),|Ci|=O(|E|2),|Di|=O(|E|),|E|=O(|E|2).

The main result of this paper is as follows.

Theorem 7.

The ABCBD problem admits an O(|w|log2|w|)-time algorithm. The hidden constant is 2O((|A|+|B|+|C|+|D|)2).

This significantly improves the O(|w|2)-time algorithm of Nogami and Terauchi [70]. Our algorithm is highly nontrivial; for a more detailed overview, see Section 3. Note that our algorithm can also be viewed as a near-linear-time fixed-parameter algorithm for the ABCBD matching problem, parameterized by the (combined) size of REGEXes A, B, C, and D. Combining Theorems 6 and 7, we obtain the following.

Theorem 8.

The 1-use REWB matching problem admits an O(|w|log2|w|)-time algorithm. The hidden constant is 2O(|E|4).

Note that the hidden constant 2O(|E|4) is larger than in Theorem 7 because Theorem 6 may yield expressions of quadratic length.

Remark: Uniformity.

The algorithms in Theorems 7 and 8 are uniform. Namely, they take the relevant expressions (A,B,C,D in Theorem 7, and E in Theorem 8) together with the input string w, perform all preprocessing depending on these expressions, and then run the main procedures. The fixed-object formulation only emphasizes the dependence on |w|; no non-uniform advice depending on the fixed expressions is assumed.

As a key subroutine for solving the ABCBD problem, we have also developed an efficient algorithm for the following problem.

XYαZ Problem

  Fixed Objects: An integer α2 and REGEXes X, Y, and Z.

Input: A string w.

Task: Deciding if we can decompose w into w=wXwYwYα timeswZ so that wX𝕃(X), wY𝕃(Y), and wZ𝕃(Z). Equivalently, decide whether w𝕃(X(Y)x(\x)α1Z).

Our theorem for this problem is the following.

Theorem 9.

The XYαZ problem admits an O(|w|log|w|)-time algorithm. The hidden constant is 2O(s2)+α2O(s), where s:=|X|+|Y|+|Z|. Alternatively, at the cost of an additional log|w| factor (i.e., time O(|w|log2|w|)), we can achieve a polynomial dependence on s+α.

1.1.3 Open Problems

Several problems remain to be investigated. The first direction is to specify classes of REWB that admit near-linear-time algorithms. We have shown that 1-use REWBs admit a near-linear-time algorithm, whereas 2-use 2-REWBs and ω-use 1-REWBs are unlikely to admit such algorithms. However, the matching complexity of r-use 1-REWB for any constant r2 remains open.

The second direction is to improve our algorithm for the ABCBD problem. The most direct question is whether we can eliminate the remaining logarithmic factor and obtain a truly linear-time algorithm: reducing O(|w|log2|w|) to O(|w|). Moreover, when REWB E is given as input, our algorithm has an exponential dependency on the expression length |E|. Specifically, it is open whether it is possible to reduce the dependence on |E| to polynomial while preserving near-linear dependence on |w|. Note that, if we allow quadratic dependence on |w|, the algorithm of Nogami and Terauchi [70] already achieves O(|E|2|w|2)-time. Furthermore, as we describe in Section 3, our algorithm consists of many building blocks and is therefore quite complex. Simplifying it is a meaningful task both for advancing theoretical understanding and for improving practical implementability.

The third direction is to broaden the scope. Real-world regular expression engines support many features beyond backreferences, such as lookaround [36, 41], greedy and lazy Kleene stars [36, 42], and atomic grouping [36, 40]. Therefore, investigating the complexity of matching regular expressions augmented with such features is important in practice. We believe that these problems are also theoretically interesting, in the same way that the REWB matching problem admits deep algorithmic study, as demonstrated in this paper.

1.1.4 Organization

The rest of the paper is organized as follows. Section 2 reviews related work on ReDoS and on hardness and algorithmic results for REGEX, REWB, and related matching problems. Section 3 gives a technical overview of our main algorithmic result, Theorem 7. Section 4 presents our hardness results: W[2]-hardness from k-Dominating Set, a conditional lower bound for 2-use 2-REWBs under the Triangle Detection hypothesis, and a lower bound for k-REWBs under the k-OV hypothesis, and hence under SETH. The remaining proofs and complete algorithmic details are deferred to the full version [53].

2 Related Work

2.1 ReDoS

Regular Expression Denial of Service (ReDoS) has been recognized as an important instance of algorithmic complexity attacks [21] since at least 2003 [20]. Outages at Stack Overflow (2016) [78] and Cloudflare (2019) [43] are representative examples, and notably, both incidents were triggered by quadratic behavior of regular expression engines, highlighting that even polynomial complexity can result in severe real-world failures. ReDoS is a common type of DoS attack that is officially cataloged as CWE-1333: Inefficient Regular Expression Complexity [58]. Vulnerabilities continue to be discovered and reported in practice; high-risk examples include [61, 62, 63, 64, 65] from 2021 to 2024. In 2025, notable entries in the CVE (Common Vulnerabilities and Exposures, which maintains a public record of disclosed security issues) include [66, 67, 68].

At its core, ReDoS arises from backtracking-based regex engines, which may exhibit catastrophic (possibly exponential) backtracking [39]. This behavior has long been known to automata theorists, and an influential book [36] helped disseminate the problem more widely to practitioners. For classical REGEX matching, Cox [16, 17] recommended that practitioners prefer Thompson NFA-based implementations over backtracking-based engines, as the former run in O(|E||w|) time. In contrast, it is unlikely that REWB matching admits such a “silver bullet” solution like the Thompson construction, as this problem is NP-hard in general [2, 9]. This has motivated complexity-theoretic research into the REWB matching problem, including the identification of tractable classes [35, 70, 75].

As an example of ReDoS involving expressions with backreferences, we focus on CVE-2017-16114 [59] and CVE-2019-25103 [60]. Both cases are triggered by an expression that can be abstracted as E:=(a)xb((a+b)b)b\x, on which backtracking-based engines take cubic time in the worst case. In CVE-2019-25103, the corresponding patch [13] rewrote E to E1:=(a)x((a+b)b)\x; it modified only the “REGEX-part” without touching the “backreference-part”. Consequently, although the ReDoS vulnerability is mitigated, potential vulnerabilities remain, as even the patched version exhibits quadratic time complexity; this issue seems to be an inevitable overhead of backtracking-based engines, as they try all possible prefixes that the variable x captures. In CVE-2017-16114, the corresponding patch [82] rewrote E to E2:=(a)x(b+(b(a+b)b))\x. Although E2 admits linear-time matching, it alters the accepting language: for example, E and E1 accept aaba but E2 does not. In contrast, the original expression falls within our ABCBD fragment; thus, it can be evaluated in near-linear time by our algorithm without modifying the semantics.

2.2 Hardness of Regular Language Matching Problems

On REGEX

While Thompson’s construction [80] solves REGEX matching in O(|E||w|) time, Backurs and Indyk [7, Thm.3] proved that O((|E||w|)1ϵ) time is unachievable under SETH.

Recently, Bringmann et al. introduced the NFA Acceptance Hypothesis [12]: intuitively, for a dense NFA A, the membership problem w𝕃(A) cannot be solved in O((|A||w|)1ϵ) time for any ϵ>0. Bille and Gørtz [10] introduced a parameter called density Δ (the total number of active states during the simulation on the input w) and proved that matching for sparse NFAs (or REGEXes) also cannot be solved in O(Δ1ϵ) time under SETH. These results suggest that improving upon the textbook O(|E||w|)-time algorithm for REGEX matching is unlikely at present.

On REWB

The REWB matching problem is known to be NP-complete [2, Thm 6.2] when the REWB expression is a part of the input, even over a unary alphabet [9]. Moreover, when parameterized by the number of variables k, the k-REWB matching problem is W[1]-hard [33, Thm. 1]. In fact, W[1]-hardness already holds for a more restricted fragment of REWB called pattern languages [33, Thm. 1][79, Thm. 1]. We briefly recall pattern languages below to relate them to REWB and summarize known algorithmic results.

2.3 Algorithms for Regular Expression Matching Problems

On REWB

The current best-known algorithm for the k-REWB matching problem is based on dynamic programming and runs in O(|E||w|1+2k) time [25, §IX. C]. Applying this algorithm to the ABCBD problem yields an O(|E||w|3)-time algorithm. Nogami and Terauchi gave an O(|E|2|w|2)-time algorithm, and thereby improved the dependence on |w| from cubic to quadratic. As noted earlier, our Theorem 7 further improves the dependence on |w| to near-linear.

Freydenberger and Schmid [35] considered the matching problem for the class of deterministic REWBs, which is a subclass of REWBs that can deterministically match strings when reading from the left. They gave an O(k|w|)-time algorithm for k-REWB. Schmid [75] extended this result to the wider class of memory-deterministic REWBs and gave an O(|E|c|w|)-time algorithm for some constant c. They also refined the time complexity of the REWB matching problem to O(|E||w|O(α)) for the cases of small α, where α(k) denotes the active variable degree, which represents the maximum number of variables that are “active” at the same time.

Pattern Language and Word Equation

A pattern language is a class of languages generated by patterns, that is, strings that may also contain variables that can be matched to strings. For example, the pattern P:=01X10X over Σ={0,1} with a variable X generates the language { 01w10w:wΣ}. This concept was introduced by Angluin in the late 1970s in the context of learning theory and has since been studied extensively [5, 6, 57, 77].

Pattern languages can be seen as a very restricted subclass of REWB. Specifically, a pattern language can be expressed as a REWB that: (1) uses a capture group ()x for the first occurrence of each variable x, (2) uses a backreference \x for all subsequent occurrences, and (3) only allows concatenation as the outermost operator (no union or Kleene star over captures/backreferences). For example, the above pattern P equals the REWB 01(Σ)X10\X.

The matching problem of pattern languages is NP-complete [5, 6, 29, 51] even for the case with alphabet size 2, each variable occurs at most twice, and each variable can be replaced with a string of length in {0,1} or {1,2,3} [32]. Ibarra et al. gave an O((|P|+|w|)|w|k1(k1)!)-time algorithm for the membership problem of k-variable pattern languages, where P is the given pattern expression [47]. In particular, it runs in linear time when k=1. Fernau et al. [30] gave algorithms for several classes of pattern languages with good properties. Particularly, they showed that for the case of non-crossing (i.e., scopes of different variables do not overlap) pattern languages, the matching problem admits an O(m|w|log|w|)-time algorithm, where m is the number of one-variable blocks occurring in a given pattern [30, Thm 8][31, Thm 4.16]. Observe that our XYαZ problem generalizes the case m=1 by allowing arbitrary REGEXes for X, Y, and Z.

Given two pattern languages (possibly sharing variables), the word equation problem asks whether there is a string that matches both of them. The case that one of the languages contains no variables is equal to the pattern language matching problem. In general, the word equation problem is known to be in PSPACE [74]. For the 1-variable case, Dąbrowski and Plandowski [23] gave an O(m+#Xlogm)-time algorithm where m is the total size of given patterns and #X is the total number of variable occurrences. Later, in the RAM-model setting, Jeż [48] gave an O(m)-time algorithm using the recompression technique introduced by Jeż [49]. Dąbrowski and Plandowski [24] gave an O(m5)-time algorithm for the 2-variable case. In contrast, the problem becomes significantly harder for k3 variables; even for the 3-variable case, it is not even known whether the problem is in NP [50]. Schulz [76] considered a variant of the word equation problem that each variable has a REGEX constraint. They proved this variant is decidable; it was later proved to be PSPACE-complete by Diekert et al. [26].

On REGEX with Lookaround

Lookaround is one of the features that real-world regular expression engines support, which checks whether a specified pattern exists around the current position, without consuming characters. It is folklore that the language class of REGEX equipped with lookaround is regular222We can convert it to a two-way alternating finite automaton (2AFA) [14], whose language class is regular [54].; thus, it admits O(|w|) time matching by Thompson’s construction when we only consider the dependence on |w|. Therefore, research concerns the dependence on the expression length |E|, and particularly, focuses on O(|E||w|) time algorithms.

Mamouras and Chattopadhyay [56] provided an O(|E||w|) time membership algorithm. Chattopadhyay et al. [15] mechanized an algorithm of the same complexity based on [56] and verified its correctness using the proof assistant Rocq (formerly known as Coq). Barrière and Pit-Claudel [8] provided a matching algorithm of the same complexity that is faithful to JavaScript REGEX matching semantics. Fujinami and Hasuo [37] provided a matching algorithm of the same complexity, which also supports atomic grouping, which is a practical extension that does not enlarge the language expressiveness [36, 40]. Uezato [81] proved that the membership problem of REWB equipped with lookahead (lookaround that only examines after the current position) is PSPACE-complete.

Gapped Patterns

Another well-studied regular-language formalism, motivated by practical pattern matching applications (rather than REWB), is that of gapped patterns. A typical gapped pattern G has the form

Gp0Σ[α1,β1]p1Σ[α2,β2]Σ[αm,βm]pm,

where each piΣ is a fixed string and Σ[α,β]:={wΣ:α|w|β} is the set of strings whose lengths lie between α and β. Thus, every gapped pattern denotes a regular language and can be expressed as a classical REGEX. Matching such patterns, including dictionary variants in which a finite set of gapped patterns is matched against a text, has been studied extensively; see, e.g., [69, 34, 11, 44, 46] and the survey [55]. Particularly relevant from a fine-grained perspective is the work of Amir et al. [4] on dictionary matching with one gap. In the uniformly bounded case, a dictionary 𝒟={G1,G2,,Gn} consists of patterns of the form Gi=p1iΣ[α,β]p2i with common bounds α,β. The task is to report their occurrences in an input text w. Under the 3SUM conjecture [38, 72, 52], for suitable dictionaries, they proved a lower bound of Ω(|w|(βα)1o(1)+𝑜𝑝) where 𝑜𝑝 is the output size [4, Thm. 5 and Table 2]. Complementing this, they also gave an online reporting algorithm whose running time depends on structural parameters of dictionaries, in particular the degeneracy of the associated bipartite graph [4, Thm. 9].

3 Technical Overview of Theorem 7

We provide an overview of our algorithm for Theorem 7, which finds a decomposition w=wAwBwCwBwD with wξ𝕃(ξ) for ξ{A,B,C,D}. The following figure, Figure 1, summarizes the overall structure.

Figure 1: Overview of the algorithm for Theorem 7.
Remark.

The complete proofs and the full algorithmic details are given in the full version [53]. In this technical overview, we use the subscript F to refer to section numbers in the full version: for instance, §10F means Section 10 of the full version. Thus, unless explicitly stated otherwise, section references with this subscript refer to the full version rather than to the present conference version.

To streamline our algorithm, we first normalize the input w and expressions so that ϵ𝕃(D) and the alphabet is binary Σ={0,1} in Section 4F. Then, we can assume wDϵ. In Section 5F, we reduce the ABCBD problem to two subproblems: the XYYZ problem (Section 6F) and the branching ABCBD problem (Sections 7F–12F). In the former problem, we consider the case wC=ϵ, i.e., w=wAwBwBwD. In the latter problem, we consider the case where two suffixes wCwBwD and wD satisfy a branching condition; that is, their first letters are distinct. To show that this decomposition covers all cases, we rely on the closure properties of REGEX and the equivalence with nondeterministic finite automata (NFAs), as detailed in Section 5.2F.

The key structure of the former (XYYZ) problem (our XYαZ problem with α=2, Section 6F) is that, since wC=ϵ, two occurrences of wB appear consecutively in w as w=wBwB. We crucially exploit the periodicity of w induced by this immediate repetition wBwB. We use Crochemore’s maximal local power enumeration [19, Sec. 9.2][18] (Section 6.1F), in combination with the periodic properties of REGEX (Section 6.3F). This part also yields Theorem 9.

To solve the latter (branching ABCBD) problem (Sections 7F–12F), we work with the suffix tree of w, 𝖲𝖳w, or simply 𝖲𝖳 (see Section 2F for the definition). Each leaf has an index i of w and corresponds to the suffix w[i..|w|] (w=w[1]w[2],w[|w|].) Each inner node v represents the substring given by the path-label from the root. For each v, we maintain the set v of all indices i of leaves in the subtree rooted at v. We note that our suffix tree 𝖲𝖳 is binary because Σ={0,1}.

The key property of the problem is the branching condition. It means that the two paths from the root to wBwCwBwD and to wBwD branch at the node v corresponding to wB (as illustrated in the figure below). Using this property, it suffices to consider the following task:

(): For each node v, find a pair of suffixes branching at v that induces a feasible decomposition.

We can illustrate our task () as follows using a suffix tree:

Let v:=u, where u is a child of v maximizing the size |u|, and define v:=vv. We employ a classical small-to-large (or heavy-light decomposition) strategy: for each iv we determine whether there exists jv such that the pair (i,j) induces a feasible decomposition. Since v:node|v|=O(|w|log|w|), the remaining task of () is to find such an index jv in O(log|w|) time for each iv. We then split the task () into the two cases Right-B and Left-B in Section 7F.

In Right-B, for each iv, we consider the case i<j: i.e., the shorter suffix wBwD starts at j. Namely, we search for jv such that w[j..|w|]=wBwD, illustrated as follows:

(R)w=w[1..i)wAw[i..i+|wB|)wBw[i+|wB|..j)wCw[j..j+|wB|)wBw[j+|wB|..|w|]wD.

In Left-B, we consider the case j<i where j is the starting position of the longer suffix wBwCwBwD. We search for jv such that w[j..|w|]=wBwCwBwD, illustrated as follows:

(L)w=w[1..j)wAw[j..j+|wB|)wBw[j+|wB|..i)wCw[i..i+|wB|)wBw[i+|wB|..|w|]wD.

To handle both cases, we use the same high-level approach: traverse 𝖲𝖳 bottom-up and maintain the sets v using suitable data structures. Since the concrete algorithms and data structures differ between the two cases, we present them separately.

Right-𝑩 Case (Sections 8F and 9F)

Let v be a node of 𝖲𝖳 and wB be the string represented by v. Let iv, where w[i..|w|] corresponds to wBwCwBwD. We assume w[1..i)𝕃(A) and wB𝕃(B) because otherwise there is no feasible solution. Our task (R) is to find an index jv such that w[j..|w|] corresponds to wBwD. To treat the constraint wC𝕃(C), we manage v using a data structure that answers whether there is an index jv with i<j such that w[i+|wB|..j)𝕃(C). We call this structure RightEnumerator (Section 9F).

However, this is not yet sufficient, as we must also enforce the constraint wD𝕃(D). Crucially, whether a fixed index j satisfies this constraint depends on the current node v. This is because the substring corresponding to wD begins at j+|wB|, and this starting position shifts as the length |wB| varies with v. Consequently, we must handle the constraints for both C and D simultaneously. To address this, we partition v into subsets such that all indices in the same subset share the same validity regarding D (Section 8F). Leveraging the fact that REGEX admits a finite transition monoid (see Section 2F for definition), we can ensure that we need to maintain only a constant number of subsets. By maintaining a separate instance of RightEnumerator for each subset and merging them using a disjoint set union data structure, we can resolve the issue and handle this case.

A simple way of constructing RightEnumerator is to use a segment tree, where each node is equipped with the set of elements of transition monoids (Section 9F). This yields an O(|w|log3|w|)-time algorithm. In our algorithm, employing a factorization forest (see Section 2F for definition), we further eliminate one logarithmic factor and achieve O(|w|log2|w|) time.

Left-𝑩 Case (Sections 10F–12F)

Let v be a node of 𝖲𝖳 and wB be the substring represented by v. Let iv, where w[i..|w|] corresponds to wBwD. We assume wB𝕃(B) and wD𝕃(D) because otherwise there is no feasible solution. Our task (L) is to find an index jv with j<i such that w[j..|w|] corresponds to wBwCwBwD. Moreover, we can handle the constraint for A simply by managing only indices jv with w[1..j)𝕃(A) (recall that, unlike the Right-B case, it is independent of v). The remaining task is to find j satisfying j+|wB|<i and w[j+|wB|..i)𝕃(C). We treat it as a data structure problem (Section 10.2F). We note that RightEnumerator is insufficient for our purpose, since the constraint j+|wB|<i depends on |wB|, which varies with the current node v.

To address the problem, we consider the following two subcases separately:

near-𝒋:j+|wB|i<j+2|wB|,far-𝒋:j+2|wB|i.

In the near-j case (Section 12F), we derive a periodicity for wB because, for any two indices j,j in the near-j case, the substrings w[j..j+|wB|) and w[j..j+|wB|) overlap. With the periodicity inherent to REGEX, we can efficiently solve the near-j case in O(log|w|) time. On the other hand, in the far-j case, the index j is “sufficiently far” from i. Technically, the condition j+2|wB|i implies that the distance between j+|wB| and i is at least |wB|; we use this margin for the design of our data structure.

To solve the far-j case, we adopt the following strategy. We first build a data structure AuxiliaryEnumerator (Section 11F), which maintains intervals [j..k) as the item [j,k) while ensuring j+|wB|k. Given i, it answers whether there is an interval [j,k) such that w[j+|wB|..i)𝕃(C) and ki. For the condition j+2|wB|i, it suffices to ensure kj+2|wB| holds at any moment. The issue is that as we traverse 𝖲𝖳, the node v changes (thus, |wB| decreases), so the condition kj+2|wB| will be violated. To resolve this issue, when the violation occurs (the margin k(j+|wB|) becomes at least |wB|), we “re-insert” the new interval [j,k) to the data structure, where k:=j+|wB|. The total overhead cost is O(log|w|) time because “re-insertion” happens only after |wB| is halved; this is ensured by the “far-j” condition.

The remaining task is to construct AuxiliaryEnumerator (Section 11F). This is similar to RightEnumerator, but requires additional mechanisms to quickly answer queries of w[j+|wB|..i)𝕃(C), which depends on the current v (and thus, wB). We adopt a lazy update strategy for this purpose. As in RightEnumerator, using the segment tree leads to the time complexity O(|w|log3|w|), which can be accelerated to O(|w|log2|w|) using the factorization forest.

4 Hardness

We show our three hardness results in this section:

  1. 1.

    The REWB matching problem is W[2]-hard (Theorem 3) in Section 4.1.

  2. 2.

    Under a hypothesis for the triangle detection problem, 2-use 2-REWB matching problem cannot be solved in almost linear time (Theorem 4) in Section 4.2.

  3. 3.

    Under SETH (more technically, k-OV Hypothesis), k-REWB matching problem cannot be solved in O(n2kϵ) time (Theorem 5) in Section 4.3.

Remark.

Due to space limitations, we omit proofs in this conference version. The full version contains the complete proofs of all statements in this section.

In Section 4.1 and 4.2, we use the following notation. Let G=(𝒱,) be an undirected graph without self-loops where 𝒱 is a set of vertices and is a set of edges. For our constructions, we assume that each vertex v is uniquely encoded as a binary string of length O(logn); that is, v(0+1)+, where, for a REWB R, we write R+:=RR for one or more repetitions of R.

For readability, in this Section 4, we sometimes write [E]x instead of (E)x.

4.1 W[2]-hardness for REWB: Reduction from 𝒌-Dominating Set Problem

We prove Theorem 3 by reduction from the following k-dominating set problem (k-DSP).

k-DSP: k-Dominating Set Problem

  Input: An undirected graph G=(𝒱,) and an integer k.

Task: Deciding if there is a k-dominating set: i.e., a subset 𝒟V of size exactly k such that:

v𝒱.((v𝒟)(u𝒟.(u,v))).

We note that, since G is undirected, (u,v)E(v,u)E. It is known that this problem is 𝐖[𝟐]-complete [27] when parameterized by k. Furthermore, the following is known.

Lemma 10 ([73][22, Thm 14.42]).

Assuming SETH, there is no algorithm running in O(nkϵ) time solving k-DSP for any ϵ>0.

Our main task is to provide the following reduction from k-DSP to k-REWB matching problem.

Lemma 11 (Reduction from k-DSP to k-REWB Matching Problem).

Let k1 be an integer. In time O(k), we can construct a k-REWB Ψk of length O(k) satisfying the following.

Let G=(𝒱,) be an undirected graph. There exists a string G~ such that the following holds:

G~𝕃(Ψk) there is a k-Dominating Set 𝒟 of G.

In particular, we can construct the string G~ in O((n+m)logn) time with |G~|=O((n+m)logn) where n:=|𝒱| is the number of vertices and m:=|| is the number of edges.

Remark.

We refer the reader to the full version for the proof of this reduction lemma.

From this lemma, we obtain the following W[2]-hardness result. See 3

Proof.

We give an FPT-reduction from k-DSP.

Given an instance (G,k), we output the matching instance (Ψk,G~), where Ψk is constructed in O(k) time and G~ is constructed in O((n+m)logn) time as in Lemma 11, such that G~𝕃(Ψk) iff G has a k-dominating set of size exactly k.

Moreover, the parameter of the output instance satisfies |Ψk|=O(k). Hence, this is an FPT-reduction, and the claim follows since k-DSP is W[2]-hard. We note that this result is stronger than W[2]-hardness parameterized by the number of variables k, since the number of variables k is at most the expression length |Ψk|.

4.2 Hardness for 𝟐-use 𝟐-REWB: Reduction from Triangle Detection Problem

We prove Theorem 4 by reduction from the following triangle detection problem.

Triangle Detection Problem

  Input: An undirected graph G=(𝒱,) with n:=|𝒱| vertices and m:=|| edges.

Task: Deciding whether G contains a triangle, that is, whether there is a set of three distinct vertices {a,b,c} with (a,b),(b,c),(c,a).

The following is conjectured about the triangle detection problem.

Hypothesis (No-Almost-Linear-Time Hypothesis [1, Conjecture 3]).

There is a constant δ>0, such that in the Word RAM model with words of O(logn) bits, any algorithm requires m1+δo(1) time in expectation to detect whether a graph with m edges contains a triangle.

In this section, our main task is to give the following reduction from the triangle detection problem to the 2-use 2-REWB matching problem.

Lemma 12 (Reduction from Triangle Detection Problem to 2-use 2-REWB Matching Problem).

There exists a fixed 2-use 2-REWB expression Ψ that solves the triangle detection problem in the following sense:

Let G=(𝒱,) be an undirected graph with n vertices and m edges. There exists a string G^ such that the following holds:

G^𝕃(Ψ) there is a triangle in G.

In particular, we can construct the string G^ in O(mlogn) time, and the size of G^ is |G^|=O(mlogn).

Remark.

We refer the reader to the full version for the proof of this reduction lemma.

We can show our hardness result from Lemma 12. While the proof follows a standard reduction argument, we provide the details below for completeness.

Theorem 4. [Restated, see original statement.]

Assuming the no-almost-linear-time hypothesis for Triangle Detection, there exists a fixed 2-use 2-REWB expression Ψ such that the matching problem for Ψ cannot be solved in |w|1+δo(1) time for some constant δ>0.

Proof.

Assume the No-Almost-Linear-Time Hypothesis holds. Let δΔ>0 be the constant specified in the hypothesis; that is, any algorithm requires time m1+δΔo(1).

Let Ψ be the fixed 2-use 2-REWB expression from Lemma 12. For every graph G with n vertices and m edges, we can construct G^ such that G^𝕃(Ψ)G has a triangle.

Set γ:=δΔ/2. It suffices to rule out algorithms running in time O(|w|1+γ), since any |w|1+γo(1)-time algorithm also runs in time O(|w|1+γ). Suppose, for contradiction, that there is an algorithm 𝒜 for Ψ running in time O(|w|1+γ).

We construct a triangle detection algorithm 𝒯 as follows: Given G, remove isolated vertices so that n2m, construct G^, and run the matching algorithm 𝒜 on G^. The construction of G^ takes O(mlogn) time and yields |G^|=O(mlogn). Since n2m, we have |G^|=O(mlogm). Running 𝒜 on G^ takes the following time T𝒜(G^):

T𝒜(G^)=O((mlogm)1+γ)=O(m1+γ(logm)1+γ).

Since (logm)1+γ=mo(1), the total running time of 𝒯 is:

T𝒯(G)=O(mlogm)+m1+γ+o(1)=m1+γ+o(1).

Substituting γ=δΔ/2, we get:

T𝒯(G)=m1+δΔ2+o(1).

For sufficiently large m, 1+δΔ2+o(1)<1+δΔo(1). Thus, 𝒯 solves the triangle detection problem strictly faster than the hypothesized lower bound m1+δΔo(1), which is a contradiction. Thus, no O(|w|1+γ)-time algorithm for Ψ exists. In particular, no |w|1+γo(1)-time algorithm exists. Taking δ:=γ proves the theorem.

4.3 Hardness from 𝒌-Orthogonal Vectors Problem Hypothesis

We now prove our O(n2qϵ)-time hardness for the q-REWB matching problem, where n denotes the length of input strings, under the k-orthogonal vectors (k-OV) hypothesis [28, 84], and hence under SETH. To use k for the k-OV and its hypothesis, we use “q” to denote the number of variables of REWB in this section.

Let v1,v2,,vk{0,1}d be d-dimensional 0-1 vectors over the ring . The k inner-product is defined as follows:

𝐈𝐏k(v1,v2,,vk):=i=1d(j=1kvj[i]).

These vectors are k-orthogonal if 𝐈𝐏k(v1,v2,,vk)=0. In our construction, we use the following equivalent condition: i=1dj=1k(vj[i]=0).

We now consider the k-OV and the k-OV Hypothesis.

𝒌-OVN,d Problem

  Input: k sets A1,,Ak{0,1}d where each set Ai has size |Ai|=N.

Task: Deciding if the following holds:

v1A1.v2A2.vkAk.𝐈𝐏k(v1,v2,,vk)=0.
Hypothesis (𝒌-OV Hypothesis [28, Hypothesis 2],[84, Hypothesis 4]).

Let k2 be an integer. For any ϵ>0 and every dimension function d(N)=ω(logN) (e.g., d(N)=log2N), 𝐤-OVN,d(N) cannot be solved in O(Nkϵ) time.

Remark.

It is known that the k-OV hypothesis is weaker than SETH; indeed, SETH implies the k-OV hypothesis [84, Theorem 3.1] [28, 83].

We provide an efficient reduction from 2k-OV problem to k-REWB matching problem.

Lemma 13.

Let k1 be an integer. There exists a (fixed) k-REWB Ψk that solves 2k-OV in the following sense: Let N1 be an integer and A1,,A2k{0,1}d be an instance of 2k-OV with |A1|==|A2k|=N. There is a string 𝒮(A1,,A2k) such that
𝒮(A1,,A2k)𝕃(Ψk)v1A1.v2A2..v2kA2k.𝐈𝐏2k(v1,v2,,v2k)=0.

In particular, we can construct 𝒮(A1,,A2k) in O(kNd2) time.

Before providing the proof of this lemma (shown in Section 4.3.1), we here prove Theorem 5 using this lemma. While the proof follows a standard reduction argument, we provide the details below for completeness.

Theorem (Reformulation of Theorem 5).

The q-REWB matching problem cannot be solved in O(|w|2qϵ) time for any ϵ>0 under the 𝐤-OV hypothesis.

Proof.

We proceed by contradiction. Suppose there exists an algorithm 𝒜 solving the problem in time O(|w|2qϵ) for some constant ϵ>0. We construct an efficient algorithm to solve 2q-OVN,d as follows:

  1. 1.

    Given an instance of 2q-OVN,d consisting of 2q sets {A1,A2,,A2q} of N vectors in {0,1}d, we first transform it into a string 𝒮:=𝒮(A1,,A2q) by our reduction. By the assumption, this reduction takes T(N)=O(Nd2) time, and n𝒮:=|𝒮|cNd2 for some constant c depending on q.

  2. 2.

    Next, we run the algorithm 𝒜 on 𝒮. The running time is

    T𝒜(n𝒮)=O(n𝒮2qϵ)=O((Nd2)2qϵ).

Fix the dimension function d(N):=(logN)2. Since d(N)=ω(logN) holds, the k-OV Hypothesis (with k=2q) applies also to 2q-OVN,d(N).

By Step 2 we obtain a 2q-OV algorithm with running time:

Ttotal(N)=O(Nd2)+O((Nd2)2qϵ)=O(Nd2)+O(N2qϵd4q2ϵ)whered=d(N).

Since d=(logN)2, we have d4q2ϵ((logN)2+1)4q2ϵ=(logN)O(1)=No(1). Thus for any δ>0 and sufficiently large N,

d4q2εNδ.

Choose δ=ϵ/2. Then

Ttotal(N)=O(N1+o(1))+O(N(2qϵ)+(ϵ/2))=O(N2qϵ/2),

contradicting the k-OV Hypothesis with k=2q.

4.3.1 Proof of Lemma 13

To explain our idea, we first give a reduction from 2-OV to 1-REWB matching problem. The reduction can be easily extended to a reduction from 2k-OV to k-REWB.

Lemma 14 (Reduction for k=1).

There exists a (fixed) 1-REWB Ψ1 that solves 2-OV in the following sense: Let N1 be an integer and A1,A2{0,1}d be an instance of 2-OV with |A1|=|A2|=N. There is a string 𝒮(A1,A2) such that

𝒮(A1,A2)𝕃(Ψ1)v1A1.v2A2.𝐈𝐏(v1,v2)=0.

In particular, we can construct 𝒮(A1,A2) in O(Nd2) time.

Proof.

We note that the proof presented here can be viewed as the REWB-version of a similar result and construction for 1-turn bounded-height (d) pushdown automata [45, Theorem 4].

Let A1={v1,v2,,vN} and A2={u1,u2,,uN}. We set Σ:={0,1,#,@,} and define the following string , which serves as the arena for our construction:

:=#v1#v2##vN#@#u1#u2##uN#

where v means the reversed version of v: i.e., 11001=10011. It should be noted that ||=O(dN).

A key idea of our reduction is to represent a pair of vectors (vi,uj) with 1i,jN by a single substring of . A substring π of represents (vi,uj) if the following holds:

=#vi##@##πuj#.

More formally, let π=π(i,j) be the factor (substring) of that starts right after the occurrence of vi and ends right before the occurrence of uj. It is explicitly given by:

π(i,j):=(#vi+1##vN)#@#(u1##uj1#)

If i=N, the left part is empty; if j=1, the right part is empty.

For example, if A1={v1=011,v2=001} and A2={u1=101,u2=100}, then our string takes the following form:

=# 110# 100#@# 101# 100#.

In this setting, the substring π=#100#@#101# represents (v1,u2) because it starts right after v1 and ends right before u2 in . Below we write ~ to denote for this example.

We define the following building block expression ψ and φv,φu where

  • we use ψ by applying it to a copy of to nondeterministically select and represent vectors v from A1 and u from A2; and

  • we use φv (resp. φu) by applying it to a copy of to check v[]=0 (resp. u[]=0) for each index [1..d].

We first define our expression ψ as follows:

ψ:=Γ(#Γ@Γ#)xΓ

where Γ:=(0+1+#) is the REGEX. We note that Γ cannot consume the symbols @ and .

We apply ψ to to nondeterministically select vectors v from A1 and u from A2. For our example A1={v1=011,v2=001} and A2={u1=101,u2=100}, applying ψ to ~ can generate the following match, in which v=v1 and u=u2 are selected:

#110Γ#100#@# 101#(#Γ@Γ#)x100#Γwhere x has # 100#@# 101#.

Another run can select v=v2 and u=u1 as follows:

# 110# 100Γ#@#(#Γ@Γ#)x101# 100#Γ.where x has #@#.

By the above observation, the following is clear.

Proposition 15.

Applying ψ to , it completely consumes and saves a substring into the variable x, which represents (vi,uj) for some 1i,jN. Moreover, any pair (vi,uj) can be captured and represented via x by applying ψ to .

Next we define our expressions φv and φu as follows:

φv:=Γ#(0\x)x#Γ,φu:=Γ#(\x 0)x#Γ.

where :=(0+1).

The former φv requires v[]=0 and the latter φu requires u[]=0 for some index [1..d]. For example, applying φv to another copy of ~ under x=#100#@#101#, it works as follows:

Γ##110#100#@# 101# 1(0\x)x00##Γ.

This application also updates x as follows:

x=#100#@# 101#0#100#@# 101# 1.

Since one application of φv and φu extends x by one symbol on each side, we can check =1d(v[]=0u[]=0) by repeatedly applying the expression (φv+φu) . For example, applying φu to another copy of ~ under the updated x, it checks if u[2]=0 and updates x as follows:
Γ##11 0#100#@# 101#1 0(\x 0)x0##Γ,x=0#100#@# 101#11 0#100#@# 101#1 0.

By the above observation, the following is clear.

Proposition 16.

Assume the content of x represents vi[] and uj[] for some 1d. Then, applying (φv+φu) to , the following holds: if the application succeeds, then vi[]uj[]=0 and x is updated to represent vi[+1] and uj[+1] (when <d).

In summary, we employ the following gadgets:

𝒮(A1,A2):=(1+d)-copies,Ψ1:=ψ(φv+φu)

It should be noted that the expression Ψ1 does not depend on the instance A1 and A2; this Ψ1 can be used for any 2-OV instances.

By the above argument, it is clear that 𝒮(A1,A2)𝕃(Ψ1) iff there are orthogonal vectors in A1 and A2. We note that |𝒮(A1,A2)|=O(Nd2) because ||=O(Nd), and 𝒮(A1,A2) can be constructed in O(Nd2) time.

We can generalize the above reduction to one from 2k-OV to k-REWB.

Lemma 13. [Restated, see original statement.]

Let k1 be an integer. There exists a (fixed) k-REWB Ψk that solves 2k-OV in the following sense: Let N1 be an integer and A1,,A2k{0,1}d be an instance of 2k-OV with |A1|==|A2k|=N. There is a string 𝒮(A1,,A2k) such that
𝒮(A1,,A2k)𝕃(Ψk)v1A1.v2A2..v2kA2k.𝐈𝐏2k(v1,v2,,v2k)=0.

In particular, we can construct 𝒮(A1,,A2k) in O(kNd2) time.

Proof.

We extend the construction for Lemma 14 to the case k2. We represent 2k vectors using k variables x1,x2,,xk. We assume the following representation:

A1={v11,,vN1},A2={u11,,uN1},A3={v12,,vN2},A4={u12,,uN2},A2k1={v1k,,vNk},A2k={u1k,,uNk}.

We change the definition of as follows:

:=12k,

where

r:=#v1r##vNr#@#u1r##uNr#.

For example, if k=2,

=#v11##vN1#@#u11##uN1##v12##vN2#@#u12##uN2#.

We use the variable xr (1rk) to represent (vir,ujr) with 1i,jN. For example, if k=2, we use x1 and x2 as follows:

#v11#v21##vN1#@#x1u11#u21##uN1##v12#v22##vN2#@#u12#x2u22##uN2#

To accommodate multiple variables, we below modify ψ, φv, and φu in Lemma 14 (the case k=1).

We first define ψ to nondeterministically select vectors from A1,A2,,A2k and bind them into x1,xk as follows:

ψ:=ψ1ψ2ψk, where ψr:=Γ(#Γ@Γ#)xrΓ,Γ:=(0+1+#).

As with ψ in Lemma 14, the following holds for ψ.

Proposition 17.

Applying ψ to , it completely consumes and saves substrings to the variables x1,x2,,xk where each xr represents (vir,ujr) for some 1i,jN. Moreover, for each 1rk, any pair (vir,ujr) can be captured and represented via xr by applying ψ to .

We next define φr to check vir[]=0 or ujr[]=0 (1d) as follows:

φr:=φr1φr2φrk(1rk),

where

φrs:={φs,(rs),φs,v+φs,u(r=s),φs,v:=Γ#(0\xs)xs#Γ,φs,u:=Γ#(\xs 0)xs#Γ,φs,:=Γ#(\xs)xs#Γ,=(0+1).
Proposition 18.

Assume that, for an index [1..d], every xr represents vir[] and ujr[] for 1rk. If the application of φr to succeeds and consumes , then either vir[]=0 or ujr[]=0 holds. Moreover, every xr is updated to represent vir[+1] and ujr[+1] (when <d).

Conversely, if vir[]=0 or ujr[]=0 holds, then applying φr to must succeed.

Using φ1,,φk, we further define φ to check the 2k-orthogonality condition:

φ:=φ1+φ2++φk.

We observe that applying φ to succeeds iff (r=1k(vir[]ujr[]))=0 when each xr represents vir[] and ujr[]. Then, we finally define 𝒮(A1,A2,,A2k1,A2k) and Ψk as follows:

𝒮(A1,A2,,A2k1,A2k):=(1+d)-copies,Ψk:=ψφ.

From the above properties, we observe the following, which completes our proof:

𝒮(A1,A2,,A2k1,A2k)𝕃(Ψk)the 2k-OV instance contains
2k-orthogonal vectors.

References