Identity Testing and Lower Bounds for Read-$k$ Oblivious Algebraic Branching Programs

Read-$k$ oblivious algebraic branching programs are a natural generalization of the well-studied model of read-once oblivious algebraic branching program (ROABPs). In this work, we give an exponential lower bound of $\exp(n/k^{O(k)})$ on the width of any read-$k$ oblivious ABP computing some explicit multilinear polynomial $f$ that is computed by a polynomial size depth-$3$ circuit. We also study the polynomial identity testing (PIT) problem for this model and obtain a white-box subexponential-time PIT algorithm. The algorithm runs in time $2^{\tilde{O}(n^{1-1/2^{k-1}})}$ and needs white box access only to know the order in which the variables appear in the ABP.


Introduction
Algebraic complexity studies the complexity of syntactically computing polynomials using arithmetic operations. The most natural model for computing polynomials is an algebraic circuit, which is a directed, acyclic graph whose leaves are labeled by either variables from {x 1 , . . . , x n } or elements from a field F, and whose internal nodes use the arithmetic operations + and ×. Each node thus computes a polynomial in the natural way. The associated complexity measures are the size (the number of wires) and the depth (the length of a longest path from an input node to the output node) of the circuit. A circuit whose underlying graph is a tree is called a formula.
Another model of computation, whose power lies between that of circuits and formulas, is that of an algebraic branching program (ABP). An ABP is a directed layered acyclic graph with a source node and a sink node, whose edges are labeled by polynomials. An ABP computes a polynomial in the following way. Every directed source-sink path computes the polynomial that is obtained from taking the product of all edge labels along the path. The polynomial computed by the ABP is the sum over all paths of those polynomials. 1 Here, another relevant complexity measure is the width of the program, which is the maximal number of vertices in a layer (see Section 1.1 for the exact definitions of the models that are considered in this work).
Two of the most important problems in algebraic complexity are (i) proving exponential lower bounds for arithmetic circuits (i.e., proving that any circuit computing some explicit polynomial f must be of exponential size), and (ii) giving an efficient deterministic algorithm for the polynomial identity testing (PIT) problem. The latter is the problem of given an arithmetic circuit, formula or ABP, computing a polynomial f , we have to decide whether f is the identically zero polynomial. PIT has a simple randomized algorithm that follows from the Schwartz-Zippel-DeMillo-Lipton lemma [Sch80,Zip79,DL78] that says that over a large enough field, a non-zero polynomial will evaluate to a non-zero value on most points. Hence, in order to decide whether f is zero it is enough to evaluate the circuit/formula/ABP on a random point (which can be done efficiently) . We further note that the randomized algorithm described above only needs to ability to evaluate f at a given point. Such algorithms are called black-box PIT algorithms. It is readily seen that black-box algorithms are equivalent to producing a small hitting set, which is a set H of evaluation points that has the property that H contains a non-zero evaluation point for every non-zero f . Algorithms that are given the computation graph as input are called white-box algorithms. Naturally, white-box access is much less restrictive and one expects it will be easier to obtain better algorithms in this case.
Apart from being a very natural problem about arithmetic computation, PIT is one of the most general problems for which an efficient randomized algorithm is known, but no deterministic one. Indeed, many other randomized algorithms -e.g. parallel algorithms for finding matching in graphs [KUW86,MVV87] or algorithms for polynomial factorization [SV10,KSS15] -reduce to PIT, in the sense that derandomization of PIT would derandomize those as well.
For more background on arithmetic circuits we refer the reader to the survey [SY10].
At first glance, the two problems described above seem rather different, as one is concerned with proving lower bounds and the other with providing efficient algorithms. However, a series of works uncovered an intricate web of connections between the two, both in the white-box [KI04,DSY09] and in the black-box [HS80,Agr05] models. That is, derandomizing PIT implies lower bounds for circuits (which gives a convincing explanation for why this problem is hard), and conversely, an explicit hard polynomial gives a recipe to "fool" small arithmetic circuits with respect to non-zeroness, in a very similar manner to the hardness-versus-randomness paradigm in boolean complexity.
In light of the hardness of proving lower bounds for general circuits, research has focused on trying to understand the effect that structural restrictions, like constant depth and multilinearity, have on the expressive power of the model.
One research direction that has attracted a lot of attention considers very shallow depth arithmetic circuits. Following Valiant et al. [VSBR83], Agrawal and Vinay gave a reduction from general circuits to depth-4 circuits, that maps subexponential size to subexponential size [AV08]. This reduction was later improved and extended in [Koi12,Tav15,GKKS13]. In a breakthrough work Gupta et al. [GKKS14] proved exponential lower bounds for depth-4 homogeneous formulas, which is the kind of circuit one gets from the reduction. In the work that followed [GKKS14], lower bounds for homogeneous depth-4 circuits were proved both for "hard" polynomials such as the permanent but also for easier polynomials such the determinant and the iterated matrix multiplication polynomial [KSS14,FLMS14,KLSS14,KS14b,KS14a].
In parallel, a lot of research effort was also focused on PIT for small-depth circuits with various restrictions such as bounded top fan-in or multilinearity [DS07,KS07,KS09,SS12,KMSV13,SV11,OSV15]. Similar to the situation with lower bounds, a derandomization of PIT for depth-4 circuits (or, depth-3 in certain cases) implies a derandomization of the general case [AV08,GKKS13]. As depth-3 multilinear formulas that have small top fan-in are a special case of sum of read-once arithmetic formulas (here, a read-once formula is an arithmetic formula in which each variable labels at most one node), Shpilka and Volkovich gave polynomial identity tests for this model [SV15]. Later, Anderson, van Melkebeek and Volkovich gave a PIT for multilinear read-k formulas, which extend both models [AvMV11].
Another line of work focused on read-once oblivious ABPs (ROABPs, and we again refer to Section 1.1 for the exact definition). ROABPs were defined by Nisan [Nis91] in the context of proving lower bounds for non-commutative formulas. While this model seems a bit restrictive, it was shown that derandomizing PIT for ROABPs implies derandomization of Noether's normalization lemma for certain important varieties [Mul12,FS13a]. It is also not hard to show that ROABPs are strictly stronger than read-once arithmetic formulas. Another motivation to study this model is that it is the algebraic analog of a boolean read-once branching program, which arises in the context of pseudorandomness for small-space computation [Nis92]. Thus, one could hope for cross-fertilization of ideas between the models that could facilitate progress on both fronts. Exponential lower bounds for ROABPs were known since their inception [Nis91], and a whitebox polynomial-time PIT algorithm was given by Raz and Shpilka [RS05]. In the black-box setting, hitting sets of quasipolynomial size were obtained in [FS13b,FSS14,AGKS15], where the last two papers being applicable even if the order in which the variable are read is unknown. This marks a striking difference between the algebraic model and the boolean model. Indeed, in the boolean domain, pseudorandom generators for read-once branching programs in unknown order are much weaker, in terms of the seed length, than Nisan's generator [Nis92] which works only if the order is known. Recently, Gurjar et al. obtained PIT algorithms for sum of ROABPs [GKST15].
In this work, we consider the natural next step, which are read-k oblivious algebraic branching programs. This model generalizes and extends both the models of ROABPs, of read-k arithmetic formulas and of sum of ROABPs. We are able to prove exponential lower bounds and to give subexponential-time PIT algorithms for this model. A summary of our results appears in Section 1.2.
Prior to our work there were no results known for this model. Some results were known for the more restricted model of a sum of k ROABPs (e.g. [GKST15]), and we give more details on those in Section 1.3.

Computational Models
In this section we define the computational models we consider in this work. We begin with the definition of Algebraic Branching Programs (ABPs).

Our Results
We give various results about the class of read-k oblivious ABPs, including lower bounds, PIT algorithms, and separations.

Lower Bounds:
We show an explicit polynomial f such that any read-k oblivious ABP computing f , for bounded k, must be of exponential width.
Theorem 1.5 (proved in Section 4). There exists an explicit polynomial f , which is computed by a depth-3 polynomial-size multilinear circuit, such that any read-k oblivious ABP computing f must have width exp(n/k O(k) ).
Prior to this work, there were no lower bounds for this model. For the more general class of read-k oblivious ABPs, we provide a white-box PIT algorithm that runs in subexponential time.
Theorem 1.7 (proved in Section 5). There exists a white-box PIT algorithm for the class of n-variate, degree-d, and width-w read-k oblivious ABPs that runs in time (nwd)Õ (n 1−1/2 k−1 )·exp(k 2 ) . Furthermore, white-box access is only needed to know the order in which the variables are read. That is, given this order, we construct an explicit hitting set of the above size for the class of read-k oblivious ABPs that read their variables in that order.

Separations:
Recently, Kayal, Nair and Saha [KNS15] constructed a polynomial f that can be computed by a sum of two ROABPs in different orders, each of constant width, such that any ROABP computing f must be of width 2 Ω(n) . Note that sum of two ROABPs is a special case of a 2-pass varying-order ABP.
In order to exemplify the strength of the multiple-reads model, we show a polynomial that can be computed by a small 2-pass varying-order ABP, but cannot be computed by a small sum of ROABPs of small width. Theorem 1.8 (proved in Section 3). There exists an explicit polynomial f on n 2 variables that is computed by a 2-pass varying-order ABP of constant width, but any sum of c ROABPs computing f must be of width exp(Ω( √ n/2 c )).

Related Work
Algebraic Models As mentioned before, Nisan [Nis91] proved exponential lower bounds for ROABPs, and Raz and Shpilka [RS05] gave a white-box polynomial-time PIT algorithm for this model. Forbes and Shpilka [FS13b] were the first to consider the black-box version of this problem, and obtained a hitting set of size (nwd) O(log n) , for n-variate, degree-d and width-w ROABPs, if the order in which the variables are read is known in advance. Forbes, Shpilka and Saptharishi [FSS14] obtained a hitting set of size (nwd) O(d log(w) log n) for unknown order ROABPs. This was improved later by Agrawal et al. [AGKS15] who obtained a hitting set of size (nwd) O(log n) which matches the parameters of the known-order case.
For higher number of reads, much less was known. Gurjar et al. [GKST15] considered the model of a sum of c ROABPs, and obtained a white-box algorithm that runs in time (ndw 2 c ) O(c) , and a black-box algorithm that runs in time (ndw) O(c2 c log(ndw)) , so that the running time is polynomial in the former case and quasipolynomial in the latter, when c is constant. A sum of c ROABPs can be simulated by read-c oblivious ABPs, and we show (in Section 3) that read-c oblivious ABPs are in fact strictly stronger.
Lower bounds against the model of sums of ROABPs were obtained in a recent work of Arvind and Raja [AR15], who showed that for every constant ε > 0, if the permanent is computed by a sum of n 1/2−ε ROABPs, then at least one of the ROABPs must be of width 2 n Ω(1) .
We also mention an earlier work of Jansen et al. [JQS10], who also gave white-box and blackbox tests for the weaker model of sum of constantly many read-once ABPs, where in their definition every variable is allowed to label only a single edge in the ABP.
Another model which is subsumed by oblivious read-k ABPs is that of bounded-read formulas. Shpilka and Volkovich [SV15] constructed quasipolynomial-size hitting set for read-once formulas, and Anderson, van Melkebeek and Volkovich [AvMV11] extended this result to multilinear read-k formulas and obtained a polynomial-time white-box algorithm and quasipolynomial-time black-box algorithm. The natural simulation of read-k formulas by ABPs produces an ABP in which every variable labels at most k edges, and it can be seen that such programs can be converted to read-k oblivious ABPs with only a polynomial overhead.
To conclude, earlier results apply only to restricted submodels of read-k oblivious ABPs.

Boolean Models
Let us now make a small detour and consider the boolean analogs for our models. A (boolean) branching program is a directed acyclic graph with a source node s and two sink nodes, t 0 and t 1 . Each internal node is labeled by a variable x i with two outgoing edges, labeled 0 and 1. The program computes a boolean function on an input (x 1 , . . . , x n ) ∈ {0, 1} n by following the corresponding path along the program. A read-k-times boolean branching program is allowed to query every variable at most k times along every path from the source a sink. Note that this is more general than our definition of readk oblivious branching program. Further distinction is made in the boolean case between semantic read-k branching programs, in which this restriction is enforced only on paths that are consistent with some input, and between syntactic read-k branching programs, in which this restriction applies for all paths (further note that in the read-once case, there is no distinction between the syntactic and the semantic model).
Exponential lower bounds for read-once branching program for explicit functions are known since the 1980's [Zák84,BHST87,Weg88], even for functions that are computed by a polynomial size read-twice branching program. Okolnishnikova [Oko91], and Borodin, Razborov and Smolensky [BRS93] extended these results and obtained exponential lower bounds for syntactic read-k-times branching programs, by giving an explicit boolean function f such that every syntactic read-k-times branching program for f has size exp(n/2 O(k) ) (in fact, the lower bound in the second work also holds for the stronger class of non-deterministic branching programs).
A strong separation result was obtain by Thathachar [Tha98], who showed a hierarchy theorem for syntactic read-k-times boolean branching program, by giving, for every k, a boolean function f which is computed by a linear-size syntactic read-(k + 1)-times branching program such that every syntactic read-k-times branching program computing f must have size exp(Ω(n 1/k /2 O(k) )).
The semantic model seemed more difficult, but nevertheless Ajtai [Ajt05] was able to prove an exponential lower bound for semantic read-k-times programs (when k is constant), which was extended by Beame at al. [BSSV03] to randomized branching programs.
PIT is the algebraic analog of constructing pseudorandom generators (PRGs) for boolean models. A PRG for a class C of boolean circuits is an easily computable function G : {0, 1} ℓ → {0, 1} n , such that for any circuit C ∈ C, the probability distributions C(U n ) and C(G(U ℓ )) are ε-close (where U m is the uniform distribution over {0, 1} m ).
Nisan [Nis92] constructed a PRG for polynomial size read-once oblivious branching programs with seed length O(log 2 n). This was followed by a different construction with the same seed length by Impagliazzo, Nisan and Wigderson [INW94]. However, for the constructions to work it is crucial that the order in which the variables are read is known in advance.
Beyond that, and despite a large body of work devoted to this topic [BDVY13, BPW11, BRRY14, De11, GMR + 12, IMZ12, KNP11, RSV13, Ste12, SVW14], all the results for the unknown order case or for read-k oblivious branching programs have much larger seed length, unless further structural restrictions are put on the program (such as very small width, regularity, or being a permutation branching programs). Specifically, we highlight that even for read-2 oblivious branching programs, the best result is by Impagliazzo, Meka and Zuckerman [IMZ12] who gave a PRG with seed length s 1/2+o(1) for size s branching program (note that the the dependence here is on s rather than on n). In particular, no non-trivial results are known for general polynomial size read-2 oblivious boolean branching program.

Proof Technique
Before delving into the details of our proof, it is perhaps instructive to think again about readonce branching programs. The main exploitable weakness of these branching programs is that by the read-once property, their computation can be broken into two subcomputations over disjoint variables, that communicate with each other only through a small "window" of width w, the width of the branching program. If w is small it is natural to expect that upon reaching the middle layer, the branching program must "forget" most of the computation of the first half so that both subcomputations are "almost independent" in a way. This property calls for a divide-and-conquer strategy, which was indeed, in very crude terms, the strategy that was applied both in the boolean model [Nis92] and in the algebraic model [FS13b,FSS14,AGKS15] (the details in each case, of course, are much more complicated than this simplistic description).

Evaluation dimension and ROABPs
Unfortunately, the above intuition breaks down when we allow a variable to be read multiple times, and this model requires a different strategy. Our main starting point is the observation that, perhaps surprisingly, multiple "passes" over the input variables, in the same order, do not provide the program with much additional power. That is, a k-pass ABP can be simulated by a ROABP, with a blow-up which is exponential in k (hence, only a polynomial blow-up, if k is constant).
This fact can be directly seen through analysis of the evaluation dimension measure. For a polynomial f (x 1 , . . . , x n ) ∈ F[x 1 , . . . , x n ] and a subset of variables S, we denote by eval S ( f ) the subspace of F[x 1 , . . . , x n ] that consists of all all the possible polynomials obtained from f by fixing the variables in S to arbitrary elements in F. The evaluation dimension of f with respect to a partition S, S, which is denoted evalDim S,S ( f ) is the dimension of eval S ( f ). Over large enough fields, this dimension equals the rank of the partial derivative matrix associated with this partition, as defined by Nisan [Nis91]. In many contexts, however, it is easier to work with the evaluation dimension. We refer to Chapter 4 of [For14] for a detailed discussion on this equivalence, including formal proofs.
The importance of the evaluation dimension measure stems from the fact that f can be computed by a width-w ROABP in the order x 1 , x 2 , . . . , x n , if and only if evalDim {x 1 ,..., Thus, this measure provides a precise characterization for the amount of resources needed to compute a polynomial in this model (see Theorem 2.4).

Evaluation dimension and k-pass oblivious ABPs
We are able to adapt the proof of the "only if" part of the above fact in order to show that if f is computed by a k-pass oblivious ABP (that is, f reads the n variables k times in the same order) then . That is, k passes over the input in the same order cannot create many independent evaluations. Then, using the "if" part of the equivalence, it follows that f can also be computed using a ROABP of width w 2k (see Lemma 2.6).
This discussion immediately implies a hitting set of the class of k-pass oblivious ABPs of size (ndw 2k ) O(log n) (Theorem 1.6), as well as exponential lower bounds for this model, simply by applying the results for ROABPs. It is still not clear, however, how to handle the general case, since even read-2 oblivious ABPs are exponentially stronger than ROABPs (recall that [KNS15] give an exponential separation between a sum of two ROABPs and ROABPs, and we separate 2-pass varying-order ABPs from sums of ROABPs).

PIT for read-k oblivious ABPs
Let us focus, for the time being, on the simplest instance of the more general problem, by considering a 2-pass varying-order ABP computing a non-zero polynomial f . That is, an ABP of width w that, without loss of generality, reads the variables in the order x 1 , x 2 , . . . , x n , x π(1) , x π(2) , . . . , x π(n) , for some permutation π. As we mentioned, we cannot possibly hope to simulate any such branching program by a small ROABP. We do, however, find a large subset of the variables S, such that if we fix all the other variables arbitrarily (or, equivalently, think of f as a polynomial in the variables of S over the field of rational functions F(S)), the resulting polynomial has a small ROABP.
By the well-known Erdős-Szekers Theorem [ES35], any sequence of distinct integers of length n contains either a monotonically increasing subsequence of length √ n, or a monotonically de-creasing subsequence of the same length. Applied to the sequence x π(1) , x π(2) , . . . , x π(n) (with the natural order x 1 < x 2 < · · · < x n ) we get a monotone subsequence of variables, which we might as well -for the sake of this exposition -assume to be monotonically increasing (the case of a decreasing sequence is, somewhat counter-intuitively, even simpler). Let S = y 1 , . . . , y √ n be the set of √ n elements that appear in this monotone subsequence. Having fixed all the variables in S, we are left, by the monotonicity property, with a branching program that reads the variables in the order y 1 , y 2 , . . . , y √ n , y 1 , y 2 , . . . , y √ n . Observe that this is exactly a 2-pass branching program! Hence, the previous arguments apply here, and if f is non-zero, we can efficiently find an assignment to the variables in S from F that keeps the polynomial non-zero. Having reached this point, we can "resurrect" the variables in S, but note that we are left with only n − √ n variables. These are again computed by a 2-pass varying-order ABP, so me may apply the same argument repeatedly. After O( √ n) iterations we are guaranteed to find an assignment to all the variables on which f evaluates to a non-zero output.
At each stage, we construct a hitting set for width-poly(w) ROABPs, of size (nwd) O(log n) . Since we take a cartesian product over O( √ n) sets, the total size of the hitting set will eventually be (nwd)Õ ( √ n) , as promised by Theorem 1.7. Generalizing the argument above for k-pass varying-order ABPs is fairly straightforward, and is done using repeated applications of the Erdős-Szekers Theorem to each of the k sequences in order to obtain a subsequence of a subset of the variables S which is monotone in every pass and has size only n 1/2 k−1 , which accounts for most of the loss in the parameters. 2 The polynomial, restricted to variables in S, will be computed by a k-pass ABP.
In order to handle general read-k oblivious ABPs, we need more ideas. We observe that after repeatedly applying the Erdős-Szekers Theorem to the subsequence of every "read", we do not get a k-pass ABP as before, but rather k monotone sequences that are intertwined together. We next show that by discarding more variables, but not too many, we get a structure that we call a "k-regularly interleaving sequence". This is a technical notion which is presented in full details in Section 5, but the main point is that this definition allows us to argue that the obtained read-k oblivious ABP has a (small) evaluation dimension and therefore it can be simulated by a not-toolarge ROABP. Obtaining this k-regularly interleaving property is the main technical difficulty of the proof.

Lower bounds for read-k oblivious ABPs
The arguments above that give PIT algorithms already give lower bounds for read-k oblivious ABPs. We have shown that if f is computed by a 2-pass varying-order ABP of width w, then there exist a subset of √ n variables S such that f is computed by an ROABP of width w 4 over F(S). This implies that if we pick f so that every restriction to √ n variables has an exponential (in √ n) lower bound for ROABPs, we would receive a subexponential lower bound for computing f in a 2-pass varying-order ABP. (These arguments, again, generalize to read-k oblivious ABPs.) In order to get an exponential lower bound (Theorem 1.5), we observe that we do not need to bound the evaluation dimension for every prefix (namely, to show that a subset of the variables is computed by a small ROABP), but only to show that the evaluation dimension is small for some prefix. This is much easier to achieve since we do not need the order of the reads to be "nicelybehaved" with respect to every prefix, but just with respect to a prefix.
In other words, we invoke a simple averaging argument to show that if f is computed by a width-w read-k oblivious ABP, then there exist sets of variables S (of size at least n/k O(k) ) and T (of size at most n/100), so that whenever we fix the variables in T we get that evalDim S,S (g) ≤ w 2k , where g is any restriction of f obtained by fixing the variables in T. We then construct an explicit polynomial whose evaluation dimension with respect to every set remains large, even after arbitrarily fixing a small set of the variables (see Theorem 4.3).

Separating 2-pass ABPs from sums of ROABPs
In order to prove the separation with a 2-pass varying-order ABPs and sum of c ROABPs (Theorem 1.8), we use a structural result proved by Gurjar et al. [GKST15] that gives a way to argue by induction on ROABPs. Given a polynomial f which is computed by a sum h 1 + h 2 + · · · h c of ROABPs of width w, we would like to find a related polynomial f ′ that is computed by a sum of c − 1 ROABPs of perhaps slightly larger width. Here, the evaluation dimension plays a role as well. The way to do this is to pick a non-trivial linear combination of w + 1 partial evaluations of f that make h 1 zero, which is possible since h 1 has a small evaluation dimension with respect to prefixes of variables corresponding to the order in which the variables are read in h 1 . One can then show that, having eliminated h 1 , each of the other summands can still be computed by a ROABP of width w(w + 1).
We provide a simple polynomial computed by a 2-pass varying-order ROABP whose partial evaluations are complex enough in the sense that they contain many linear independent evaluations and also a "scaled-down" version of the original polynomial as a projection. It then follows by induction, using the above arguments, that this polynomial cannot be computed by a small sum of small ROABPs (see Lemma 3.6).

Organization
We start with some preliminaries and useful facts about the evaluation dimension in Section 2 that almost all the results in this paper rely on. In Section 3, we present the separation between the class of 2-pass varying order ABPs and sums of ROABPs. Following that, in Section 4, we present an exponential lower bound for the class of general read-k ABPs. Then in Section 5 we present the white-box PIT for read-k ABPs. Finally, we conclude with some open problems in Section 6.

Notation
For n ∈ N, we denote by [n] the set {1, 2, . . . , n}. We commonly denote by x a set of n indeterminates {x 1 , . . . , x n }, where the number of indeterminates n is understood from the context. As we often deal with prefixes of this set, we denote by x [i] the set {x 1 , . . . , x i }, and more generally, for For a polynomial f ∈ F[x], a set S ⊆ [n] and vector a = (a 1 , . . . , a |S| ) ∈ F |S| , we denote by f | x S =a the restriction of f obtained by fixing the j-th element in S to a j .
For a subset S ⊆ x of variables, we denote its complement by S. For disjoint subsets S, T ⊆ [n] we denote by S ⊔ T their disjoint union.
In our PIT algorithm, we need to combine hitting sets for smaller sets of variables. Hence, for a partition of [n], S 1 ⊔ S 2 ⊔ · · · ⊔ S m = [n], and sets H i ⊆ F |S i | , we denote by H S 1 1 × · · · × H S m m the set of all vectors in F n whose restriction to the S i coordinates is an element of H i , that is We will also use the following theorem that gives a construction of a hitting set for ROABPs.

ABPs and iterated matrix products
The computation of an ABP corresponds to iterated multiplication of matrices of polynomials.
In the case of oblivious branching programs, the ABP computes an iterated matrix product of univariate matrices. We record this fact as a lemma, and refer to [For14] for a proof and a detailed discussion on this subject.

Evaluation dimension and ROABPs
We now define a complexity measure for polynomials that we will use frequently when analyzing read-k oblivious ABPs.
which is the space of polynomials spanned by all partial evaluations of the S variables in f .
, which shall be denoted by evalDim S,T;R ( f ), as the dimension of the space eval S ( f ) when taken over the field of rational functions F(x R ). That is, we first "move" the variables x R into the field and treat them as constants, and then consider the dimension of eval S ( f ) over F(x R ).
In the special case where R = ∅, we shall just use the notation evalDim S, is the rank of the partial derivative matrix with respect to S,T, as defined by Nisan [Nis91]. The rows of the partial derivative matrix are indexed by monomials m S in S and its columns are indexed by monomials m T in T. The (m S , m T ) entry is the coefficient of m S m T in the polynomial f . Although these two perspectives are equivalent, the formulation via evaluations is sometimes easier to work with. The evaluation dimension measure is useful when arguing about ROABPs since it characterizes the width needed to compute a polynomial f using a ROABP.
Theorem 2.4 ( [Nis91], and see also [For14]). Let f be a polynomial on x = {x 1 , . . . , x n } and suppose for then in any ROABP that computes f in the order x 1 , x 2 , . . . , x n , the width of the i-th layer must be at least w.
Let us give an example of a polynomial which has large evaluation dimension with respect to a specific subset. This example will be helpful not only because it is simple to argue about, but also because all of our constructions of hard polynomials later on will ultimately be based on a reduction to this case.
Lemma 2.5. Let f (u, v, w) be a polynomial of the form where: 1. For every a ∈ F |u| , it holds that g| u=a = g(a, w) ≡ 0.

{ℓ i } t i=1 is a set of linearly-independent linear functions , and so is {ℓ ′
Proof. By applying a linear transformation to the variables (which cannot increase the dimension), if necessary, we may assume without loss of generality that In particular, for any a = (a 1 , . . . , a t ) ∈ {0, 1} t we can evaluate u to a so that , 1} t are linearly independent. Further, by the assumption on g, we also have that g(a, w) is non-zero. Hence these polynomials (in v) remain linearly independent even when multiplied by the variable-disjoint polynomial g(a, w) and so evalDim u,v⊔w ( f ) ≥ 2 t .
The following simple lemma is an illustration of using the evaluation dimension of a polynomial to obtain a small ROABP for that polynomial.
Lemma 2.6. Let f ∈ F[x 1 , . . . , x n ] be a polynomial computed by a k-pass ABP of width w, according to the order π. Then f can be computed by a width-w 2k read-once ABP in the order π.
Proof. Let A be the k-pass ABP computing f . We may assume without loss of generality that the k passes of A read the variables in the order x 1 , . . . , x n . Recall that for any i ∈ [n], we denote x [i] = {x 1 , . . . , x i }. By Theorem 2.4, it is enough to show that for any i ∈ [n], By the assumption on f and by Lemma 2.2, for every i ∈ [n] and j ∈ [k] there exists a matrix M i,j ∈ F w×w such the entries of M i,j are univariate polynomials in x i and , and consider any assignment of the form x [i] = a for a = (a 1 , . . . , a i ) ∈ F i . Having fixed x [i] , we get that for some k matrices N 1 (a), . . . , N k (a), that depend on a, (2.7) It follows that any polynomial g(x i+1 , . . . , By Theorem 2.4, the claim follows.
In fact, the proof of Lemma 2.6 permits a slight generalization of the lemma, by requiring weaker assumptions on the ABP, which is captured by the following definition.
Definition 2.8. Let A be an ABP that computes a polynomial f ∈ F[x 1 , . . . , x n ]. We say that A has the k-gap property with respect to {x 1 , . . . , x i }, if there exist k matrices M 1 , . . . , M k ∈ F w×w [x i+1 , . . . , x n ] such that for every a ∈ F i , there exists k matrices N 1 (a), . . . , N k (a) ∈ F w×w such that (

2.9)
A is said to simply have the k-gap property if it has this property with respect to x [i] , for every i ∈ [n]. ♦ Figure 1 provides a pictorial explanation for the choice of this terminology. Using the exact same arguments as in the proof of Lemma 2.6, we obtain the following lemma. x 1 x 2 x 3 x 4 x 1 x 2 x 1 x 2 x 3 x 4 x 3 x 4 Figure 1: An ABP that reads the variables in this (left-to-right) order is a read-3 ABP that has the 2-gap property with respect to {x 1 , x 2 }.

Separating 2-pass ABPs from sums of ROABPs
Recall that every sum of c ROABPs can be realized by an oblivious read-c ABP. In order to motivate our study of read-k oblivious ABP, we begin by showing a polynomial that can be computed by a constant-width, 2-pass varying-order ABP, and yet cannot be computed by a small sum of polynomial-size ROABPs. Thus, even a weak, but non-trivial, form of read-k oblivious ABPs, for k = 2, is already stronger than sums of ROABPs. Suppose x = {x 1,1 , . . . , x n,n } is a set of n 2 variables. It is useful to think of x as an n × n matrix X such that x i,j appears in the (i, j)-th entry. For every m ∈ [n], define (3.1) Observe that for all i, j, rowSum i and colSum j can be computed by width-2 ROABPs. Moreover, both ∏ n i=1 rowSum i and ∏ n j=1 colSum j can be as well. Indeed, their product P n is computed by a 2-pass varying-order ABP.
The proof exploits the structure of a sum of few ROABPs that Gurjar, Korwar, Saxena and Thierauf [GKST15] used for constructing hitting sets. The following lemma is essentially present implicitly in their result. For completeness, we provide a proof.

Lemma 3.3 ([GKST15]
). Let f = h 1 + · · · + h c where each h i is computed by a width-w ROABP in possibly different orders. Then, for every 0 < t < n, there exists a subset S of t variables such that for every set of w + 1 partial assignments a 1 , . . . , a w+1 ∈ F t , there is some non-trivial linear combination of that is computable by a sum of c − 1 ROABPs of width w(w + 1) in possibly different orders. That is, there exists α 1 , . . . , α w+1 ∈ F, not all zero, such that where each f ′ i is a ROABP of width at most w(w + 1). Proof. Let S be the first t variables that are read in the ROABP that computes h 1 . Since h 1 is computed by a width-w ROABP, evalDim S,S (h 1 ) ≤ w. Hence, every w + 1 partial evaluations are linearly dependent, that is, there exist α 1 , . . . , α w+1 ∈ F, not all zero, such that where the last equality follows from (3.4). Hence, to prove the statement of the lemma it remains to be shown that for every 2 ≤ j ≤ c, ∑ w+1 i=1 α i h j | x S =a i is computed by a ROABP of width w(w + 1). Fix such j. Observe that since h j is computed by a ROABP of width w, for every i ∈ [w + 1] we have that h j | x S =a i is computed by a ROABP of width w (by replacing the variables with the appropriate constants in the ABP that computes h j ), and furthermore all the ROABPs of the form h j | x S =a i for i ∈ [w + 1] are in the same order (inherited from the order of the ROABP computing h j ).
Therefore, we can connect those (w + 1) ROABPs in parallel to obtain a single ROABP, of width The following lemma shows that the polynomial P n defined in (3.1) has many linearly independent partial evaluations.
Furthermore, for any g ∈ span{P n | x S =a i | i ∈ [r]}, there is a set y ⊆ x \ S of (n − t − 1) 2 variables, such that P n−t−1 (y) can be obtained as a projection of g: namely, for z = x \ (y ∪ S) we can find a ∈ F n−|S|−|y| such that g| z=a = P n−t−1 (y).
Proof. Recall that we think of the n 2 variables as an n × n matrix X. By rearranging the rows and columns, assume that all variables in S are present in the first a rows and first b columns. Observe that a, b ≤ t and say a ≤ b so that we also have b ≥ √ t. Also, any linear combination of evaluations of S variables would always be divisible by Q = ∏ i>a rowSum i · ∏ j>b colSum j and hence we shall just work with In the [a] × [b] sub-matrix, set all variables not in S to zero, and let P ′′ be the resulting polynomial. Clearly, it suffices to establish linear independence of partial evaluations of P ′′ . We label the remaining variables in the first b columns by u if they belong to S and by v otherwise. The variables which are not in the first b columns are labeled by w (see Figure 2). Then, we can write so that we have the properties: 1. For i = j, ℓ i and ℓ j are supported on disjoint sets of variables, because they correspond to different column sums, and similarly for ℓ ′ i and ℓ ′ j . In particular, each of the sets {ℓ i } t i=1 and {ℓ ′ i } t i=1 is linearly independent. 2. g is the product the first a row sums, so it is a product of variable-disjoint linear functions in u and w, and in particular for any a ∈ F |u| , g(a, w) ≡ 0.
Hence, there exists r ≥ 2 √ t evaluations a 1 , . . . , a r for the variables in S for which the set of polynomials {P ′ n | x S =a 1 , . . . , P ′ n | x S =a r } are linearly independent. Hence, it also follows that the set of polynomials {P n | x S =a 1 , . . . , P n | x S =a r } are linearly independent as well. This completes the first claim of the lemma. Now also observe that since Q divides each P n | x S =a i it follows that any non-trivial linear combination of {P n | x S =a 1 , . . . , P n | x S =a r } is a non-zero multiple of Q. Let us fix one such linear combination g = h · Q for a non-zero polynomial h. Note that h depends on just the variables in the first a rows and first b columns. By the Schwartz-Zippel-DeMillo-Lipton lemma [Sch80,Zip79,DL78] there exists an assignment to the variables in the first t rows and the first t columns that keeps h · Q non-zero. If y ′ = y ij : i ∈ [n − t], j ∈ [n − t] is a relabeling of the variables in the last (n − t) rows and columns, such an evaluation to the first t rows and columns would result in a polynomial of the form for some field elements α i , β j : i, j ∈ [n − t] . By further setting y i,n = (−α i ) and y n,j = (−β j ) for i, j ∈ [n − t − 1], and fixing y n−t,n−t to a value that preserves non-zeroness, we obtain the projection (up to a constant factor) where y = y ′ \ y ij : i = n or j = n , which equals P n−t−1 (y).
With the above two lemmas, Theorem 3.2 is straightforward.

Proof of Theorem 3.2. The proof is a simple induction on c.
We shall show that if P n is computable by a sum of c ROABPs of width at most w, then n ≤ log 2 (w + 1) + log 2 (w + 1) 2 + · · · log 2 (w + 1) 2 c−1 + c.
Let us assume the hypothesis is true for c − 1 and we now prove it for c. Suppose P n is computable by a sum of c ROABPs of width w. Assume that t = log 2 (w + 1) < n, for otherwise the lower bound follows immediately. By Lemma 3.3, there is some set S of t variables such that for any r = (w + 1) partial evaluations a 1 , . . . , a r on S, some linear combination is computable by a sum of c − 1 ROABPs of width w(w + 1).
On the other hand, Lemma 3.6 states that we can find r ≥ 2 √ t = w + 1 partial evaluations on S that are linearly independent and any linear combination of them can be written has P n−t−1 as a projection.

Lower bounds for read-k oblivious ABPs
In this section we show an explicit polynomial that has a polynomial-size depth-3 multilinear circuit and yet cannot be computed efficiently by a read-k oblivious ABP.

An explicit polynomial with large evaluation dimension
Raz and Yehudayoff [RY09] constructed an explicit multilinear polynomial f (x) with evaluation dimension as high as possible with respect to any partition S, S. Our requirements are slightly different, as we would need some "robustness" property, namely, we would want to argue that the evaluation dimension of the polynomial remains high even when we fix a small constant fraction (say, n/10) of the variables. Later, in Theorem 4.4, we show why this property implies hardness for read-k oblivious ABPs. Our construction is inspired by a recent similar construction of Kayal, Nair and Saha [KNS15]. Consider the complete bipartite graph K n,n with n vertices on each side. We shall label the left vertices as x 1 , . . . , x n and the right vertices as y 1 , · · · , y n . We can write K n,n as a union of n edge-disjoint perfect matchings M 1 ∪ · · · ∪ M n , where for every i ∈ [n], M j contains all edges of the form (x j , y j+i mod n ) for j ∈ [n]. Define the polynomial Q n as Q n (x 1 , . . . , x n , y 1 , . . . , y n , z 1 , . . . , z n (4.1) By its definition, it is clear that Q n is computed by a depth-3 polynomial-size circuit. We now show that even if we fix a small fraction of the variables in x ∪ y, Q n retains a large evaluation dimension with respect to any partition of the variables we have not fixed.
Proof. Assume without loss of generality that |S| ≤ |T|, and that S L := S ∩ x satisfies |S L | ≥ |S|/2. Since (S ∪ T) ∩ y ≥ 0.8n, |T ∩ y| ≥ (0.8n − |S|/2) ≥ 0.3n. Thus, there are Ω(n · |S|) edges between S and T in K n,n . By averaging, some matching M i must include at least Ω(|S|) of these edges. Consider the polynomial f i = ∏ (j,k)∈M i (x j + y k ). As Ω(|S|) of the edges in M i go between S and T, we can write where for every m ∈ [t] we have that u m ∈ S, v m ∈ T, and t = Ω(|S|) (we have "pushed" to g all the factors that correspond to edges in the matching which do not go between S and T).
By Lemma 2.5, evalDim S,T ( f i ) ≥ 2 Ω(|S|) . Since f i is a projection of Q n (under the setting z i = 1 and z j = 0 for all j = i) it follows that evalDim S,T (Q n ) ≥ evalDim S,T ( f i ) ≥ exp (Ω(|S|)).
As an aside, we note that the difference between the above polynomial and the one constructed by Kayal, Nair and Saha ( [KNS15]) is that they use a 3-regular bipartite expander instead of K n,n (which is important for their application). The degree of the graph corresponds to the number of matchings, and hence to the top fan-in of the depth-3 circuit computing the polynomial. In fact, in order to show hardness for read-k oblivious ABPs it is also possible to use a good enough bipartite expander (with a constant degree d that depends only on k) whose expansion property guarantees that a proof strategy along the lines of Lemma 4.2 and Theorem 4.4 would work. This would have allowed us to present a hard polynomial which is computed by a depth-3 circuit with bounded top fan-in, however, this very small gain would have come at the cost of increasing the complexity of the construction and making it depend on k.

Upper bound on evaluation dimension for read-k oblivious ABPs
In this section we show that if f is computed by a read-k oblivious ABP of width w, then we can fix a "small" subset of variables such that the remaining variables can be partitioned into two carefully chosen "large" subsets, under which the evaluation dimension is at most w 2k . We then apply this result to the polynomial Q n (from (4.1)) to show that if Q n is computed by a width-w read-k oblivious ABP, then w ≥ exp(n/k O(k) ).
Proof. Consider an ABP A that computes f . Divide the kn layers into r equal-sized contiguous blocks of kn/r layers (where r shall be set shortly). For each variable, consider the k blocks that its k reads fall in. By a simple averaging, there must exist k blocks B 1 , . . . , B k that contain all k reads of a set U of at least n/( r k ) variables. Let W be the set of variables in B 1 ∪ B 2 ∪ · · · ∪ B k that are not in U, and V be the set of all remaining variables. As each block is of size kn/r, we have that |W| ≤ k 2 n/r, which is at most n/10 if we set r = 10k 2 . Observe that |V| ≥ n − k 2 n/r ≥ 9n/10. Let us ignore the variables in W by considering the ABP over the field F(x W ).
We now claim that evalDim U,V;W ( f ) ≤ w 2k . Having moved the variables in W to the field, each of the r blocks is either entirely contained in U or entirely contained in V. Therefore, since the reads comprise of at most k alternating blocks of variables in U and V, the resulting branching program has the k-gap property with respect to U. It follows immediately from Lemma 2.10 that evalDim U,V;W ( f ) is at most w 2k .
We now show that Q n (defined in (4.1)) is hard to compute for read-k oblivious ABPs. (4.1)). Then w ≥ exp(n/k O(k) ).

Theorem 4.4. Let A be a width-w, read-k oblivious ABP computing the polynomial Q n (defined in
Proof. First observe that we can eliminate the z variables by considering the ABP over the field F(z) so that is now computes a polynomial in the variables x ∪ y.
By Theorem 4.3, there exists a partition U ⊔ V ⊔ W of x ∪ y with the prescribed sizes as in the statement of the theorem, such that evalDim U,V;W (Q n ) ≤ w 2k .
Using the fact that min(|U|, |V|) ≥ n/k O(k) , we get that w 2k ≥ exp(n/k O(k) ), which implies w ≥ exp(n/k O(k) ) as well.

Identity tests for k-pass ABPs
In this section we give PIT algorithms for the class of read-k oblivious ABPs. First, observe that Lemma 2.6 immediately implies a black-box algorithm for the subclass of k-pass ABPs, as those can be simulated efficiently by a ROABP.
Corollary 5.1. There is a hitting set of size (ndw) O(k log n) for the class of n-variate k-pass ABPs of width w and degree d.
Proof. Follows directly from Lemma 2.6 and the (ndw ′ ) O(log n) -sized hitting set for width w ′ readonce ABPs from Theorem 2.1.
We now turn to general read-k oblivious ABPs.

From read-k to per-read-monotone and regularly-interleaving sequences
In this section we show that given any read-k oblivious ABP over x = {x 1 , . . . , x n } computing a polynomial f , we can find a "large" subset of variables y ⊆ x such that f has a "small" ROABP when we think of f as a polynomial in the y variables over the field F(y). This process, in fact, involves only finding the correct subset y (without rewiring any part of the ABP). Therefore, in order to avoid technical overhead it is useful to think in terms sequences over abstract sets of elements, which correspond to the order in which the ABP reads the variables, and not in terms of variables in branching programs.
Let X be a set, and let n = |X|. Let S ∈ X m be an sequence of elements from X. We say S is read-k if each element x ∈ X occurs k times in S (in this case we also have m = nk). As mentioned in Remark 1.3, we will restrict ourselves to considering sequences that are read-k for some k. For i ∈ [k], we denote by S (i) the subsequence of S which consists of the i-th occurrences of elements in X. That is, S (i) is a permutation of the elements of X, according to the order in which they appear in S for the i-th time. Similarly, for i = j ∈ [k], we use the notation S (i,j) for the subsequence of S which consists of the i-th and j-th occurrences of elements in X.
For x ∈ X and i ∈ [k] let Occur i S (x) denote the index of the i-th occurrence of x in S. For an index ℓ ∈ [kn] let Var S (ℓ) denote the pair (x i , c) such that the c-th occurrence of x i appears at index ℓ in S. For a subset X ′ ⊆ X, let S| X ′ denote the restriction of S to the set X ′ that is the result of dropping all elements of X \ X ′ from S. Thus, S| X ′ ∈ X m ′ for m ′ = |X ′ |k.
In order to save on excessive notation and multiple indexing, we will assume without loss of generality that S (1) = (x 1 , . . . , x n ), that is, that the variables in x are already labeled according to the order of their first occurrence. This can be ensured by renaming variables, if necessary.
Next we define a special subclass of read-k sequences which we work with throughout this section.
Definition 5.2. Let S ∈ X nk be a read-k sequence. We say S is per-read-monotone if for every i ∈ [k], S (i) is monotone (that is, the variables all appear in either increasing or decreasing order). ♦ The following well-known theorem asserts that any long enough sequence contains a large monotone subsequence: Theorem 5.3 (Erdős-Szekers Theorem, [ES35,AZ04]). Let S be a sequence of distinct integers of length at least m 2 + 1. Then, there exists a monotonically increasing subsequence of S of length m + 1, or a monotonically decreasing subsequence of S of length m + 1.
As an immediate corollary of Theorem 5.3, we get the following lemma: Lemma 5.4. Let S be a read-2 sequence over X = {x 1 , . . . , x n }. Then, there exists a subset X ′ ⊆ X with Proof. First, observe that the subsequence of first occurrences is monotone by our definition of the order on X according to the first occurrence in S (and thus every subsequence of it is monotone as well). Consider the subsequence S (2) of the n second occurrences. By Theorem 5.3, there exists a monotonic subsequence of length at least √ n. Let X ′ ⊆ X be the set of elements that appear in this monotonic subsequence, and let S ′ = S| X ′ . Then by the choice of X ′ , it follows that S ′ is per-read-monotone.
We can generalize Lemma 5.4 to read-k sequences, at the cost of settling for a weaker lower bound of only n 1/2 k−1 on the length of the subsequence: Lemma 5.5. Let S be a read-k sequence over X = {x 1 , . . . , x n }. Then, there exists a subset X ′ ⊆ X with |X ′ | ≥ n 1/2 k−1 such that the subsequence S ′ = S| X ′ is per-read-monotone.
Proof. As in the proof of Lemma 5.4, the set X ′ can be constructed by repeatedly pruning X using k − 1 repeated applications of Theorem 5.3.
That is, we first apply Theorem 5.3 on the subsequence S (2) of second occurrences and obtain a monotonic subsequence of length n 2 := √ n. We discard all elements of X which do not appear in this monotonic subsequence. We move on to the subsequence S (3) of third occurrences, and find a monotonic subsequence of length n 3 := √ n 2 , again discarding all elements that do not appear in this subsequence. After finding the k-th monotonic subsequence, we are left we a subset X ′ of size n 1/2 k−1 that satisfies the conditions of the lemma.
We now show how to prune per-read-monotone read-2 sequences even further, trading a constant fraction of their size for stronger structural properties. We begin by stating the property we look for.
Definition 5.6. Let S be a read-2 sequence over a set of elements X. We say S is 2-regularly-interleaving if there exists a partition of X to blocks {X i } i∈ [t] such that for every i ∈ [t]: • For every c ∈ {1, 2}, all the c-th occurrences of the block X i appear consecutively in S.
• The interval containing the second occurrences of the block X i immediately follows the interval containing the first occurrences of X i .
A read-k sequence S is said to be k-regularly-interleaving if for any i = j ∈ [k], the subsequence S (i,j) is 2-regularly-interleaving. That is, S is k-regularly-interleaving if restricted to any two reads it is 2regularly-interleaving. ♦ To get a better intuitive sense of the definition, the reader may consult Figure 3. 1st occurrences of X 1 2nd occurrences of X 1 1st occurrences of X 2 2nd occurrences of X 2 · · · 1st occurrences of X t 2nd occurrences of X t Figure 3: A 2-regularly-interleaving sequence.
The following lemma is used to simplify some of the later arguments. It shows that in a read-k per-read-monotone sequence, the monotonically increasing subsequences cannot intersect with monotonically decreasing subsequences.
Lemma 5.7. Let S be a read-k, per-read-monotone sequence over X = {x 1 , . . . , x n }. Suppose S (1) is monotonically increasing. Then we can write S as a concatenation S = (T 1 , T 2 , . . . , T t ), such that: 1. for every j ∈ [t], T j is a read-k j sequence for k j ≤ k.

for every i ∈ [k] there exists j ∈ [t] so that S (i) is contained in T j .
3. for every odd j ∈ [t], all the subsequences S (i) that appear in T j are monotonically increasing, and for any even j, all are monotonically decreasing.
4. for every j ∈ [t − 1], the last element that appears in T j equals the first element appearing in T j+1 , and this element can be either x n (if T j contains monotonically increasing subsequences and T j+1 contains monotonically decreasing subsequences) or x 1 (in the opposite case).
In other words, we can partition S into t disjoint contiguous subsequences, such that every S (i) is completely contained in exactly one subsequence, and in every subsequence, either all reads are increasing or all reads are decreasing, with the pattern alternating.
Proof. The proof is by induction on k. For k = 1, S = S (1) and this is a trivial statement.
For larger values of k, we would like to show first that no decreasing sequence can intersect an increasing one. Suppose without loss of generality that S (2) is decreasing (the other case, where the first sequence is decreasing and the second increasing, in handled analogously). Then, where the first inequality is obvious and the second follows from S being per-read-monotone. Furthermore, since S (2) is decreasing, have that which implies that S (2) cannot intersect S (1) , but rather it must begin after S (1) ends. Let ℓ denote the first index in which a decreasing subsequence S (j) begins (if no such ℓ exists, the lemma is clearly satisfied by picking T 1 = S). By the above argument, all the elements before the ℓ-th index must belong to increasing subsequences which are read entirely. We can define T 1 to be the subsequence of S from index 1 up to index ℓ − 1, and continue inductively on the subsequence S ′ of S from index ℓ to the end, which has k ′ < k reads.
As for item 4, it follows from the fact that a sequence of monotonically increasing subsequences must end in x n (as to maintain monotonicity), and a sequence of monotonically decreasing subsequences must begin with x n , for the same reason. The opposite case is handled analogously.
The following lemma shows that given a 2-read per-read-monotone sequence, we can find a large subsequence which is also 2-regularly interleaving.
Lemma 5.8. Let S be a read-2 per-read-monotone sequence over X = {x 1 , . . . , x s }. Then there is a subset X ′ ⊆ X with |X ′ | ≥ s/3 such that the sequence S ′ = S| X ′ is per-read-monotone and 2-regularlyinterleaving.
Proof. We show how to erase the occurrences of (not too many) elements from S, such that the remaining sequence is 2-regularly-interleaving and maintains its per-read-monotonicity property.
First observe that if the subsequence S (2) of second occurrences is monotonically decreasing, then, by Lemma 5.7, S is already also 2-regularly-interleaving. In this case we have S = (S (1) , S (2) ) and we can pick just one block, X, and satisfy the definition.
From now on we assume then that S (2) is monotonically increasing. For every z ∈ X, denote by d z = Occur 2 S (z) − Occur 1 S (z) the distance between the first and the second occurrence of z in S.
Pick x ∈ X such that d x is maximal and let r := d x . Among the r occurrences between Occur 1 S (x) and Occur 2 S (x), there exist either r/2 first occurrences or r/2 second occurrences. Assume there are at least r/2 first occurrences (the other case is handled in an analogous way), and let A be the set of variables (including x) whose first occurrence appears between the Occur 1 S (x) and Occur 2 S (x), so that |A| ≥ r/2.
Since S (2) is monotonically increasing, for every z ∈ A it holds that Occur 2 S (z) > Occur 2 S (x). Let y ∈ A be the element such that Occur 2 S (y) is maximal. Observe that where the first inequality follows from the fact that S is per-read-monotone and the second follows from the choice of x. Hence, it follows that We now erase from S all the elements that appear in the interval [Occur 1 S (x), Occur 2 S (y)] but do not appear in A. Having done that, the subsequence in this interval satisfies the requirements of the lemma (one can relabel the elements if necessary in order to ensure contiguous indexing in this subsequence, and see also Figure 4). Furthermore, we have kept at least |A| ≥ r/2 elements alive and erased, by (5.9), at most r elements. We continue recursively on the subsequences in both of the intervals [1, Occur 1 S (x) − 1] and [Occur 2 S (y) + 1, 2s]. Observe that these intervals cannot share any element, as that would mean that the two occurrences of this element are of distance more than r apart, which contradicts the choice of x. Hence, we may continue independently on both subintervals. By induction, the statement of the lemma follows.
Viewed as an algorithmic process, the proof of Lemma 5.8 is a procedure that, given a per-readmonotone sequence S over X, decides which elements of X should be erased in order to be left with a 2-regular-interleaving sequence S ′ = S| X ′ . It can also be noted that both properties of being per-read-monotone and being 2-regularly interleaving are downward-closed, in the sense that if we now take a subset X ′′ ⊆ X ′ and look at S ′′ = S ′ | X ′′ , it will maintain both properties. Hence, if we are given a read-k per-read-monotone sequence S, by repeatedly applying the algorithmic process of Lemma 5.8 separately on each subsequence S (i,j) for i = j ∈ [k] (maintaining a constant fraction of the elements on each application), we get the following corollary: Corollary 5.10. Let S be a read-k per-read-monotone sequence over X = {x 1 , . . . , x s }. Then there is a subset X ′ ⊆ X with |X ′ | ≥ s/3 k 2 such that the sequence S ′ = S| X ′ is per-read-monotone and k-regularlyinterleaving.

ROABPs for regularly interleaving sequences
In this section we show that if a polynomial f is computed by a small-width read-k oblivious ABP A such that the sequence S of the reads in A is per-read-monotone and k-regularly-interleaving, then f can in fact also be computed by a small-width ROABP A ′ (in the same order as S (1) ). We show this by proving that A has the k-gap property with respect to that order, and then applying Lemma 2.10.
Lemma 5.11. Let f ∈ F[x 1 , . . . , x n ] be computed by a read-k oblivious ABP A of width w, and let S be the sequence of variables read by A. Suppose further that S is per-read-monotone (with respect to the order x 1 < x 2 < · · · < x n ) and k-regularly-interleaving. Then A has the k-gap property.
In the proof of Lemma 5.11 we will use the following lemma in order to bound the number of "gaps" one obtains for any prefix.
Lemma 5.12. Let S be a read-k, per-read-monotone, k-regularly-interleaving sequence over {x 1 , . . . , x n }. Suppose that for every i ∈ [k], S (i) is monotonically increasing. Let ℓ be an integer and suppose that Var S (ℓ) = (x i , c) 3 and Var S (ℓ + 1) = (x j , d) with j > i. Then, it must be the case that j = i + 1.
Proof. First observe that if c = d the claim is true by monotonicity of S (c) . If c = d, consider the subsequence S (c,d) . Since S is k-regularly interleaving, S (c,d) is 2-regularly interleaving, and in this sequence it also holds that x j immediately follows x i . Furthermore, we have that d < c, as otherwise Occur c S (x j ) < Occur c S (x i ), which contradicts the monotonicity of S (c) , as we assumed that j > i. Hence, in S (c,d) d plays the role of the first read, and c plays the role if the second read. To avoid extraneous terminology let us assume for now that d = 1 and c = 2.
Suppose the blocks are indexed by {X 1 , . . . , X t }, such that x i belongs to X b i and x j belongs to X b j .
Recall that our assumption is that the first read of x j immediately follows the second read of x i . Hence, after the block X b i is read for the second time, we need to read the next block X b i +1 for the first time. This blocks contains x j , so we get that b j = b i + 1 and also that x j is the smallest element in that block, so j = i + 1.
We are now ready to prove Lemma 5.11.
Proof of Lemma 5.11. Let S be the sequence of reads in A. Let i ∈ [n]. Consider any fixing x [i] = a for a ∈ F i . By plugging in the values to A, we can always write (5.13) for some integer t, where for each σ ∈ [t], N σ is a product of univariate matrices of layers that read {x 1 , . . . , x i }, and M σ is a product of univariate matrices of layers that read {x i+1 , . . . , x n }. We wish to show that t can be at most k. For each pair (N σ , M σ ) define their interface to be the pair (Var S (ℓ σ ), Var S (ℓ σ + 1)), where ℓ σ is the last index of a layer that participates in the product that defines N σ , and thus ℓ σ + 1 is the first index of a layer that participates in the product that defines M σ (see Figure 5 for an illustration).
f | x 1 =a 1 ,x 2 =a 2 = N 1 (a 1 , a 2 )M 1 (x 3 , x 4 )N 2 (a 1 , a 2 )M 2 (x 3 , x 4 ) (1,1) We first argue that we can assume, without loss of generality, that for every j ∈ [k], S (j) is increasing. This is because Lemma 5.7 implies that the sequence S can be thought of as a concatenation of subsequences (T 1 , . . . , T t ) that each have this property, and then handle each subsequence separately (one would of course need to prove an analog to Lemma 5.12 and to some of the arguments we give further down the proof for the case where the subsequences in T m are decreasing. The analogs are rather straightforward and can be obtained by "reversing" the decreasing sequences. We leave the details to the reader). Furthermore, by item 4 in Lemma 5.7, the "border" between T m and T m+1 (that is, the last element of T m and the first in T m+1 ) is always marked by either two occurrences of x 1 or of x n . Since x 1 is read at most k times, it is immediate that any setting x 1 = a 1 for a 1 ∈ F admits a representation of the form (5.13) for t ≤ k. The case where all variables x 1 , x 2 , . . . , x n are fixed is trivial to verify. And for any other prefix x [i] these "borders" cannot increase the number of gaps in (5.13).
From now on we assume then that for every j ∈ [k], S (j) is increasing, and consider once again the representation (5.13). We wish to show there cannot be too many interfaces. For every interface σ ∈ [t], let us denote Var S (ℓ σ ) = (x σ 1 , c) and Var S (ℓ σ + 1) = (x σ 2 , d).
Since this is an interface, it must be the case that σ 1 ≤ i < σ 2 . By Lemma 5.12, we must have σ 2 = σ 1 + 1, and thus σ 1 = i and σ 2 = i + 1. Hence, we can map each interface to a unique occurrence of x i . This immediately implies there can be at most k interfaces, and hence at most k gaps.
It now immediately follows that any read-k oblivious ABP the reads the variables in a per-readmonotone and k-regularly-interleaving fashion can be simulated by a small ROABP. We record this fact in the following corollary.
Corollary 5.14. Let f ∈ F[x 1 , . . . , x n ] be computed by a read-k oblivious ABP A of width w, and let S be the sequence of variables read by A. Suppose further that S is per-read-monotone (with respect to the order x 1 < x 2 < · · · < x n ) and k-regularly-interleaving. Then for any i ∈ [n], evalDim In particular, f is computed by a ROABP of width at most w 2k in the variable order x 1 , x 2 , . . . , x n .
Proof. Immediate from Lemma 5.11 and Lemma 2.10.

Identity testing for read-k oblivious ABPs
In this section we give our white-box identity testing algorithm for read-k oblivious ABPs. Before giving the proof, let us first give an overview of the algorithm for the slightly simpler read-2 case.
Given a read-2 oblivious ABP A with read sequence S which computes a polynomial f ∈ F[x 1 , . . . , x n ], Lemma 5.8 shows how to find a read-2 subsequence on a set y = {y 1 , . . . , y √ n } of roughly √ n variables, such that when we think of f as a polynomial in the y variables over the field F(y), it has a small ROABP. We can then use a hitting set for ROABPs in order to find an assignment (from F) to the y variables that keeps the polynomial non-zero. Having done that, we are left with a non-zero polynomial over a smaller set of n − √ n variables, which is again computed by a read-2 oblivious ABP, so we may repeat this process. After at most O( √ n) iterations we find an assignment for all the variables that keeps the polynomial non-zero. We note that a very similar "hybrid argument" that uses a hitting set for ROABPs appears both in [AGKS15] and [OSV15].
The argument for read-k is identical, apart from the loss in the parameters incurred by Corollary 5.10.
Theorem 5.15. There is a white-box polynomial identity test for read-k oblivious ABPs of width w and degree d on n variables that runs in time poly(n, w, d) n 1−1/2 k−1 exp(k 2 ) polylog(n) . Furthermore, given only the order in which the variables are read, we can construct a hitting set for such ABPs that read their variables in this order, of size poly(n, w, d) n 1−1/2 k−1 exp(k 2 ) polylog(n) .
We begin with a technical lemma which we use to bound the number of iterations of the above process.
Lemma 5.16. Let p be a real number such that 0 < p < 1 and r be a positive integer. For any n ∈ N, n 1−p − (n − n p /r) 1−p ≥ (1 − p)/r.
We defer the elementary proof of Lemma 5.16 to Appendix A.
Our PIT algorithm is presented in Algorithm 1.

Algorithm 1 : PIT for read-k oblivious ABPs
Input: a read-k oblivious ABP A computing a polynomial f ∈ F[x 1 , . . . , x n ].
1: x = {x 1 , . . . , x n }, i = 1 2: while x = ∅ do 3: Pick a subset y i ⊆ x of size at least |x| 1−1/2 k−1 /3 k 2 , such that the subsequence that reads only the y i variables is per-read-monotone and k-regularly-interleaving (such a subset exists by Lemma 5.5 and Corollary 5.10).

4:
Construct a set H i ⊆ F |y i | of size (nw 2k d) O(log n) that hits ROABPs of width w 2k in the y i variables, using Theorem 2.1. 5: x ← x \ y i , i ← i + 1 6: end while 7: return the set H = H y 1 1 × · · · × H y t t (where t is the number of iterations of the loop).
We now prove Theorem 5.15.
Proof of Theorem 5.15. Consider Algorithm 1. We first show that the set H it returns hits A, and then we bound the size of H.
By Corollary 5.14, the polynomial f in the y 1 variables is computed by a width-w 2k ROABP over the field F(y 1 ). Hence, by Theorem 2.1, there exists a 1 ∈ H 1 such that f (a 1 ) is non-zero over F(y 1 ). Similarly, we can now find a 2 ∈ H 2 and assign it to the y 2 variables and keep the polynomial non-zero, etc. all the way up to a t . It follows that (a 1 , . . . , a t ) is an assignment from F to all the variables such that f (a 1 , . . . , a t ) is non-zero, as required.
Furthermore, |H i | = (nw 2k d) O(log n) , so, |H| = ((nw 2k d) O(log n) ) t . We now bound the number of iterations t. Let T(n) denote the number of iterations needed for n variables. We show, by induction on n, that T(n) = cn 1−1/2 k−1 , (for some c = c(k) which we will set in a moment). This will imply the desired bound on H. Set p = 1/2 k−1 . After the first iteration, the number of variables we are left with is n ′ := n − n p /3 k 2 variables. By the induction hypothesis, we may assume that T(n ′ ) ≤ c · (n ′ ) 1−p . Hence T(n) ≤ 1 + T(n ′ ) ≤ 1 + c(n ′ ) 1−p = 1 + c · n − n p 3 k 2 1−p and we wish to show that 1 + c n − n p 3 k 2 1−p ≤ cn 1−p .
This is equivalent to 1 c ≤ n 1−p − n − n p 3 k 2 1−p , which is satisfied, by Lemma 5.16, as long as which we can ensure by picking c = 2 · 3 k 2 (recall that p ≤ 1/2). Finally, since finding the set y i on each iteration can be done in polynomial time, the running time of the algorithm is dominated by the time required to construct H, which is poly(|H|).

Conclusions and Open Problems
In this work, we have obtained the first non-trivial lower bounds and identity testing algorithms for read-k oblivious ABPs. We briefly mention some directions that we find worth pursuing for future research.
The most natural open problem we pose is designing an identity testing algorithm for read-k oblivious ABPs with better running time than the algorithm we presented in this paper. Since for ROABPs (the k = 1 case) there exist a white-box polynomial time and black-box quasipolynomialtime algorithms, it seems reasonable to hope that the deterioration in the parameters would not be as sharp when k > 1 (the flip side of this argument, however, is the relative lack of progress in the analogous question in the boolean domain).
Another open problem is obtaining a complete black-box test for read-k oblivious ABPs, in any variable order (that is, without knowing the order in which the variable appear). As we mentioned, for ROABPs there exist a black-box hitting set that works for any variable order [AGKS15], whose size is essentially the same as that of the hitting set that was obtained earlier for the known order case [FS13b]. In our construction, we need to know the order so that we can pick the per-read-monotone and k-regularly-interleaving sequences to which we assign the hitting sets for ROABPs, and simply "guessing" those sets would require exponential time. Still, given the progress in obtaining hitting sets in any order for ROABPs, it might be the case that such a construction could follow from our strategy, even using known techniques.
Finally, we turn back to boolean complexity, and ask whether our ideas and techniques can be adapted to attack the problem of constructing pseudorandom generators for read-k oblivious boolean branching program with sublinear seed length.