Abstract 1 Introduction 2 Proof Sketch 3 Model and Preliminaries 4 Reduction to a Communication Game 5 The Communication Lower Bound References

Optimal White-Box Adversarial Streaming
Lower Bounds for Approximating LIS Length

Anna Gal Department of Computer Science, University of Texas at Austin, TX, USA Gillat Kol ORCID Department of Computer Science, Princeton University, NJ, USA Raghuvansh R. Saxena ORCID School of Technology and Computer Science, Tata Institute of Fundamental Research, Mumbai, India Huacheng Yu ORCID Department of Computer Science, Princeton University, NJ, USA
Abstract

The space complexity of deterministic streaming algorithms for approximating the length of the longest increasing subsequence (LIS) in a string of length n has been known to be Θ~(n) for almost two decades. In contrast, the space complexity of this problem for randomized streaming algorithms remains one of the few longstanding open problems in one-pass streaming. In fact, no better than Ω(logn) lower bounds are known, and the best upper bounds are no better than their deterministic counterparts.

In this paper, we push the limits of our understanding of the streaming space complexity of the approximate LIS length problem by studying it in the white-box adversarial streaming model. This model is an intermediate model between deterministic and randomized streaming algorithms that has recently attracted attention. In the white-box model, the streaming algorithm can draw fresh randomness when processing each incoming element, but an adversary generating the stream observes all previously used randomness and adaptively chooses the subsequent elements of the stream.

We prove a tight (up to logarithmic factors) Ω(n) space lower bound for any white-box streaming algorithm that approximates the length of the LIS of a stream of length n to within a factor better than 1.1. Thus, for this problem, white-box algorithms offer no improvement over deterministic ones.

Keywords and phrases:
White-bos streaming, Longest increasing subsequence
Funding:
Raghuvansh R. Saxena: We acknowledge support of the Department of Atomic Energy, Government of India, under project no. RTI4001.
Copyright and License:
[Uncaptioned image] © Anna Gal, Gillat Kol, Raghuvansh R. Saxena, and Huacheng Yu; licensed under Creative Commons License CC-BY 4.0
2012 ACM Subject Classification:
Theory of computation Streaming, sublinear and near linear time algorithms
Editor:
Shubhangi Saraf

1 Introduction

The LIS problem.

The length of the Longest Increasing Subsequence (LIS) is a fundamental measure of sequences that has been extensively studied, with applications in various areas, including bioinformatics, string-matching, text processing and more. See the books by Gusfield [15] and Pevzner [21] and the survey by Aldous and Diaconis [2].

Formally, given a sequence σ of length n over an ordered alphabet [m], an increasing subsequence in σ is a subsequence i1<<ik such that σ(i1)<<σ(ik). For c1, if the longest LIS in σ is of length , a c-approximation of the length of the LIS in σ is any value between c and .

The streaming model.

In this paper, we study the space complexity of approximating the length of the LIS in the data stream model of computation. In this model, the input arrives as a stream of elements, the algorithm is allowed only one or a few passes over the stream, and it should use limited memory. The streaming model has been extensively studied in recent years, driven by the need to process massive data sets arising in domains such as network traffic monitoring, database management, machine learning, social networks, and large-scale graph analysis (see the surveys by Muthukrishnan [20] and Babcock et al. [4]). In these applications, the input is either stored on an external device, or obtained on the fly from some source, and the algorithm does not have sufficient working memory to store the entire data set.

LIS in the streaming model.

The LIS length approximation problem is particularly motivated in the streaming model. For example, since efficient sorting is infeasible in this setting, some applications wish to estimate how close their input is to being sorted. The length of the LIS is a natural measure of this closeness.

Streaming algorithms for computing or approximating LIS length have received considerable attention. Computing the length of the LIS exactly via a streaming algorithm is known to require Ω(n) space, even for randomized algorithms [14, 25]. This shows that to get sublinear space, one needs to settle for approximation. Indeed, [14] gave a deterministic one-pass streaming algorithm that approximates LIS length to within any multiplicative constant and uses 𝒪(nlogm) space. A matching (up to logm factors) lower bound of Ω(n) was shown by [12, 8] for any alphabet size mn.

As discussed above, the space complexity of (deterministic or randomized) streaming algorithms for computing LIS length exactly, as well as the space complexity of deterministic streaming algorithms for approximating the length of LIS, are well understood.

In contrast, as far as we know, for randomized streaming algorithms that (1+ε)-approximate the length of the LIS, the Ω(logn) lower bound of Sun and Woodruff [25] is still the best known for any (arbitrarily small) constant ε>0. Thus, the optimal space complexity of randomized streaming algorithms for approximating the length of LIS remains wide open.

The white-box adversarial streaming model.

In this paper, we push the limits of our understanding of the LIS length approximation problem by considering it in the white-box adversarial streaming model, an intermediate model between randomized and deterministic streaming algorithms that received a lot of attention lately [1].

In the standard data stream model, it is assumed that the entire input is fixed in advance, and a randomized streaming algorithm has to be correct on any input stream with high probability (over the algorithm’s randomness). This can also be interpreted as the input is chosen by an adversary, where the adversary knows the algorithm, but does not have any information about the algorithm’s internal states (memory and random coins) during the execution of the algorithm.

Recently, streaming algorithms in stronger adversarial settings received a lot of attention, where instead of fixing the entire input in advance, the adversary is allowed to choose the next element of the stream after observing parts of the algorithm’s behavior up to the current step.

The white-box adversarial streaming model, introduced by [1], allows the adversary to have full access to the internal states of the algorithm. More precisely, upon receiving each element, the algorithm can use fresh randomness to update its memory. However, the white-box adversary that generates the stream can observe all previously used random coins during the execution of the algorithm up to the current step (and therefore also knows the algorithm’s memory content), before fixing the next element of the stream. The algorithm has to be correct with high probability over both the algorithm’s and the adversary’s random choices, for every white-box adversary.

The white-box model captures scenarios where algorithms lack access to a securely private source of randomness, and, as pointed out in [1, 10, 9], also some non-adversarial settings. For instance, a server may publish its initial state and users may subsequently generate their data based on this state and send it back to the server. Analogous notions of the white-box streaming model are studied in areas such as dynamic data structures, machine learning, and persistent data structures. For a broader survey of white-box models across different domains, see [1, 10, 9].

1.1 Our Result

We prove the following lower bound on the space complexity of any white-box streaming algorithm for approximating LIS length.

Theorem 1.

Any (randomized) white-box streaming algorithm that, with probability 23, outputs a 1.1-approximation to the length of the LIS of a stream of length n with alphabet size m3n requires Ω(n) space.

The space lower bound in Theorem 1 is tight (up to logm factors) as a deterministic (and therefore also white-box) streaming algorithm with space complexity O(nlogm) is known [14]. Additionally, for alphabet size m, there are deterministic one-pass streaming algorithms that compute the length of LIS exactly using O(mlogm) space (see Section 1.2). Therefore, the space lower bound in Theorem 1 can only hold for large enough alphabet size m.

Our proof of Theorem 1 differs from the proofs of previous white-box streaming lower bounds. Known white-box lower bounds were obtained by proving one-way, two-party deterministic communication lower bounds for the corresponding communication problem, as [1] show that such bounds imply white-box streaming lower bounds. This approach, however, cannot be used to prove Theorem 1 since, for every constant ε>0, there is a dynamic-programming-based two-party one-way deterministic communication protocol with only polylogn communication that gives a (1+ε)-approximation of LIS length. More generally, for k2, there is a k-party one-way deterministic communication protocol with kpolylogn communication [14, 25]. This suggests that to prove Theorem 1 we should consider communication protocols with many parties. Unfortunately, [1] shows that their result cannot be extended to the case of multi-party communication protocols.

Another intermediate model between deterministic and randomized streaming that attracted attention recently is the adversarially robust streaming model [5]. In this model the adversary sees only part of the algorithm’s internal state (see Section 1.2), and therefore any white-box algorithm is also adversarially robust. While our Theorem 1 shows that LIS length approximation is hard in the white-box model, the work of [5] implies that the space complexity of LIS length approximation in the adversarially robust model is essentially the same as that of randomized algorithms. See Section 1.2 for more details. In this sense, our result proves a lower bound in the strongest studied model that does not imply a randomized lower bound.

1.2 Prior Work

LIS in other models.

A classical dynamic-programming-based algorithm, called patience sorting [2], finds a longest increasing subsequence, and thus also computes the length of LIS exactly, in nearly linear 𝒪(nlogn) time using 𝒪(nlogm) space. Fredman [11] proved that the Θ(nlogn) running time of patience sorting is optimal for computing the length of LIS exactly. Sublinear-time and sublinear-space algorithms for approximating the LIS length have also been extensively studied in various settings; see, e.g., [3, 22, 24, 13, 17, 7].

LIS in the streaming model.

Patience Sorting can naturally be viewed as a one-pass 𝒪(nlogm) space streaming algorithm for computing the length of LIS exactly. More precisely, the space bound is 𝒪(Llogm) where L is the length of the LIS. Thus, the one-pass deterministic streaming complexity is 𝒪(min(n,m)logm).

Gopalan, Jayram, Kumar and Krauthgamer [14] and Sun and Woodruff [25] proved an Ω(n) space lower bound for computing the length of LIS exactly, even for randomized algorithms.

[14] gave a deterministic one-pass streaming algorithm to compute a (1+ε)-approximation of the length of LIS using space 𝒪(nεlogm) for any ε>0. Gál and Gopalan [12] and Ergun and Jowhari [8] proved matching lower bounds showing that Ω(nε) space is necessary for deterministic constant-pass streaming algorithms for any ε1/n and large enough alphabet size mn. More precisely, the bound of [12] is of the form Ω(1Rnεlog(2mn)) for R-pass streaming algorithms. Li and Zheng [18] extended the deterministic lower bounds of [12, 8] to smaller alphabet sizes, proving lower bounds of the form Ω(min(n,m)) for constant-pass streaming algorithms. In addition, Li and Zheng [19] extended the above space lower bounds to the streaming model where the stream may be provided in other than the “standard” order of the input sequence σ. Saks and Seshadhri [23] gave a polylogn space randomized streaming algorithm that approximates LIS length within additive error δn for any constant δ>0.

Chakrabarty [6] showed that the multiparty communication problems used in the deterministic space lower bounds for approximating the length of LIS in [12, 8] allow very efficient randomized protocols.

The adversarially robust streaming model.

Another adversarial streaming model, closely related to the white-box model, is the adversarially robust streaming model [5], where the adversary is allowed to observe outputs of the algorithm up to the current step before fixing the next element of the stream, but cannot see the random coins of the algorithm. It is known that space complexity in the “standard” randomized streaming model may be strictly smaller than in adversarially robust streaming [16], which in turn can be strictly smaller than white-box adversarial streaming space complexity [1].

On the other hand, it was shown in [5] that for a certain class of problems “standard” randomized streaming algorithms can be simulated by adversarially robust streaming algorithms without using much more space. In particular, for approximating LIS length this means that the space complexity of adversarially robust streaming is within 𝒪(logn) factor of the space complexity of standard randomized streaming. Thus, proving space lower bounds in the adversarially robust streaming model for approximating the length of LIS would be equivalent to proving lower bounds for standard randomized streaming, up to 𝒪(logn) factors.

In light of this, we focus our attention on the white-box adversarial model, as an intermediate model between randomized and deterministic streaming.

2 Proof Sketch

We overview our proof of Theorem 1 in this section. The proof has two main parts. First, we define a communication game 𝖫𝖳𝖬𝖺𝗍 (Less Than Matrix) and show a reduction saying that a streaming algorithm for approximating the length of the LIS in the input stream implies a protocol that solves the communication game. With this reduction, it suffices to prove a lower bound on the length of the longest message in any protocol solving the communication game 𝖫𝖳𝖬𝖺𝗍, which forms the second part of the proof.

2.1 The Communication Game 𝗟𝗧𝗠𝗮𝘁

The communication game 𝖫𝖳𝖬𝖺𝗍 is parameterized by two parameters, an integer N and a real number η. In the communication game 𝖫𝖳𝖬𝖺𝗍, there are 2N parties and the input of each party is an N bit vector. We interpret the input vectors of the odd-numbered parties as the rows of a matrix A and therefore call these parties 𝖠1,,𝖠N, and interpret the rows of the even-numbered parties as the rows of a matrix B and call these parties 𝖡1,,𝖡N. The goal of the parties is to distinguish between the case when A=B, which we call the no case, and the case where at least an η fraction of the entries in any row of A are strictly smaller than the corresponding entries in B, which we call the yes case.

We study one-way protocols for the game 𝖫𝖳𝖬𝖺𝗍 when the parties’ inputs are determined by a white-box adversary. That is, first an all-powerful adversary selects an input for the party 𝖠1, who then uses its randomness and the selected input to send a message to the party 𝖡1. The adversary can see the sampled randomness (and therefore, the message) and then selects an input for the party 𝖡1, who then uses this input together with fresh randomness to send a message to the party 𝖠2. This continues till the last party 𝖡N computes the output, which is either yes or no. The protocol succeeds in solving the game if whenever the selected input is in the no (respectively, yes) case, the output is no (resp. yes) with high probability.

The game 𝖫𝖳𝖬𝖺𝗍 can also be viewed in the white-box adversarial streaming setting, where the input vectors of the parties 𝖠1,𝖡1,,𝖠N,𝖡N are streamed coordinate by coordinate in that order. Note that, when viewed this way, the length of the resulting stream is n=N2 and that the white-box adversary has more power as it can change the subsequent entries in the same vector after seeing how a streaming algorithm behaved in a prefix. In contrast, the white-box adversary in the communication version commits to the entire vector at once. This means that a lower bound on the length of the longest message in a white-box communication protocol implies a lower bound on the space needed by a white-box streaming algorithm. We use these two views interchangeably when we talk about our reduction.

2.2 Reducing 𝗟𝗧𝗠𝗮𝘁 to Approximating the Length of the LIS

We now show that a white-box streaming algorithm for estimating the length of the LIS implies a white-box streaming algorithm that solves the game 𝖫𝖳𝖬𝖺𝗍. This means that a lower bound on the latter suffices to show Theorem 1.

For this, for any row i and any column j, we define a fixed offset Δi,j and consider the sequence formed by adding the offset Δi,j to the corresponding entry in both the matrices A and B. As the original entries in A and B are just bits, we can define the offsets so that regardless of the original entries, the offsetted entries in any column j are decreasing in both A and B, and any entry in any column exceeds any entry in any preceding column in both A and B.

Observe that because of our choice of offsets, any increasing subsequence (when the input is viewed as a stream) has length at most 2N. Indeed, as the columns are decreasing in both A and B, at most one entry can be selected from each column in each matrix, giving a total of at most 2N entries. Moreover, the only way to have a column j such that an entry from column j of A is in the LIS and an entry from column j of B is in the LIS is if it is the same entry (i,j) in both the matrices and Ai,j<Bi,j. Indeed, it must be the same entry as if not, the fact that the columns are decreasing would violate the LIS property. Moreover, given that it is the same entry, the entry for A comes before the entry for B in the stream, and therefore we must have Ai,j<Bi,j. Combining, we conclude that when the inputs come from the no case of 𝖫𝖳𝖬𝖺𝗍, the length of the resulting LIS is N.

Consider now the case when the inputs come from the yes case of 𝖫𝖳𝖬𝖺𝗍. By the definition of the yes case, we have that for any row i, it holds that Ai,j<Bi,j for at least an η fraction of the columns j. This makes for a total of ηN2 such pairs (i,j) which by the pigeonhole principle means that at least ηN/2 such pairs would be on the same (parallel to the top-left to bottom-right) diagonal. These pairs would each contribute 2 entries to an increasing subsequence, which when combined with one entry from all the columns not participating in these pairs would give an increasing subsequence of length at least N(1+η/2).

As we have shown that inputs from the no case of 𝖫𝖳𝖬𝖺𝗍 give rise to sequences where the length of the LIS is N and inputs from the yes case of 𝖫𝖳𝖬𝖺𝗍 give rise to sequences where the length of the LIS is at least N(1+η/2), we get that an algorithm that gives a (1+η/2)-approximation to the length of the LIS with high probability would solve the game 𝖫𝖳𝖬𝖺𝗍 with the same probability, finishing the proof of our reduction.

2.3 A Lower Bound for 𝗟𝗧𝗠𝗮𝘁

We now show that any white-box communication protocol that solves the game 𝖫𝖳𝖬𝖺𝗍 will have at least one message with length Ω(N). To this end, note that while it may seem that being able to decide the input of the players after seeing the randomness used by the players so far gives the adversary a lot of power, this is hardly the case, as the definition of the game 𝖫𝖳𝖬𝖺𝗍 itself imposes a lot of restrictions on the inputs the adversary can give the parties.

For example, in the no case of 𝖫𝖳𝖬𝖺𝗍, the adversary needs to ensure that A=B which is possible only if the input of the party 𝖠i is the same as the input of the party 𝖡i for all i[N]. This means that, despite being able to see the randomness used by 𝖠i, once the adversary has selected the input for the party 𝖠i, he cannot use that ability to his advantage, as the input for the party 𝖡i must be the same as 𝖠i.

Moreover, the fact that the definition of 𝖫𝖳𝖬𝖺𝗍 treats all the rows i[N] “separately” with nothing connecting the inputs for different rows (for ii) means that selecting the input for the party 𝖠i and then looking at the player’s randomness may not even help the adversary in selecting the inputs for the parties 𝖠i+1 and onwards. Thus, despite the apparent power a white-box adversary has in selecting the input after seeing the parties’ randomness, for the no case in our lower bound, we have to consider very simple adversaries. Specifically, the adversary we consider takes a (balanced) binary error correcting code 𝖢 and samples an independent and uniformly random codeword as the inputs for all the odd parties and gives the same input to the corresponding even parties. Note that the reason we sample a uniformly random codeword instead of a uniformly random bit string is that any two distinct (balanced) codewords would have many coordinates where one of them is larger than the other, a property that we need in the yes case.

We now discuss how our adversary selects the inputs for the parties in the yes case. As the goal of the adversary is to make this case indistinguishable from the no case, the adversary must select the inputs from 𝖢 in this case as well. This means that all the adversary has to do in order to satisfy the promise in the yes case is to ensure that for all i, the parties 𝖠i and 𝖡i always receive different codewords in 𝖢. Then, the fact that they are balanced would imply that the codeword for 𝖠i is strictly smaller than the codeword for 𝖡i in a large fraction of the coordinates. Additionally, all a protocol has to do in order to solve the game 𝖫𝖳𝖬𝖺𝗍 is to check whether the codewords for 𝖠i and 𝖡i are the same or not.

While the requirements of the yes case force the adversary to select different codewords for 𝖠i and 𝖡i, the adversary wants to ensure that the party 𝖡i cannot detect that the codewords are different by looking at its input and the message it gets from 𝖠i. These two requirements are incompatible for general protocols: The party 𝖠i can simply send its input to the party 𝖡i who can then detect that their inputs are different by comparing this input with the received message. However, for short protocols where the messages between the parties have length o(N), it is not possible for 𝖠i to send its entire input, and this protocol does not work. Still the question remains whether the adversary can select a different codeword for 𝖡i without it being able to detect that its different.

Selecting the input of 𝗕𝒊.

In order to show that the adversary can always select an input for 𝖡i, we show a general lemma (see Lemma 7) saying that for any random variable 𝖷 such that no value in the support is taken with probability larger than 12, there exists a way to couple it with an identically distributed random variable 𝖷 such that in the joint distribution, we have 𝖷𝖷 with probability 1. The assumption that no value is taken with probability larger than 12 is tight, as if it is false, there exists x such that Pr(𝖷x)=Pr(𝖷x)<1/2. By a union bound, we get that Pr(𝖷=𝖷=x)>0 contradicting the existence of a coupling for which 𝖷𝖷 with probability 1.

To prove this lemma, we interpret all the values the random variable 𝖷 can take as the vertices in a weighted graph and treat the value Pr(𝖷=x) as the desired degree of this vertex. Then our goal is assign weights to all the edges in the graph (no self-loops) such that the sum of the weights on the edges incident to any given vertex equals its desired degree. If we can achieve this, then, we can treat the weight on any edge (x,x) as the probability that (𝖷,𝖷)=(x,x) and also the probability that (𝖷,𝖷)=(x,x), there by defining a coupling between 𝖷 and 𝖷. As the sum of the weights on the edges incident to any vertex equals its desired degree, we get that 𝖷 and 𝖷 are identically distributed with the desired distribution, and as there are no self-loops, we get that 𝖷𝖷 with probability 1.

We use an iterative “greedy” procedure that adds weight to the edge between two vertices with the highest desired degree, till either the weight adds up to the desired degree for some vertex, or conditioned on the weights already assigned, the probability of the random variable taking some value x reaches 12. In the former case, we can use our procedure again with that vertex removed while in the latter case, we can connect the value whose probability is 12 with all other vertices in a star configuration and the desired weights, finishing the proof.

With this lemma in hand, the adversary looks at the message sent by 𝖠i to 𝖡i and applies the lemma on the distribution of the input of 𝖠i conditioned on this message. We show that, if this distribution assigns probability more than 12 to some input, then it is as if 𝖠i sent that input in its entirety which means the protocol must be long. Otherwise, we can use the lemma, to generate an input for 𝖡i that is guaranteed to be different from the input of 𝖠i and also guaranteed to have the right conditional probability given the message, as desired.

A hybrid argument.

The above analysis shows that one can generate the input of any one party 𝖡i in a way so that it is hard for it to distinguish between the yes case and the no case. To generate the input of all the parties, we use this argument N times, once for each i[N]. For this, we define N+1 adversaries Λ0,,ΛN, where for all i, the adversary Λi is such that the first i rows of the input matrices look like the no case and the remaining look like the yes case. Thus, any two consecutive adversaries only differ in one of the input rows meaning we can apply the argument above and show that any two consecutive adversaries are indistinguishable. Then, by a triangle inequality, we conclude that the adversaries Λ0 and ΛN, corresponding to the yes and no case are also indistinguishable as desired.

For this hybrid argument to work, the simplicity of our adversary above is helpful. Recall that our adversary in the no case treats each row independently, and thus, we can “stitch” together distributions for different copies without worrying about undesired correlations. Finally, for the triangle inequality to be useful, we need that the probability of distinguishing between any two consecutive adversaries is o(1N), which we show holds. In fact, we show that the probability is exponentially small in N.

3 Model and Preliminaries

For a set S, we use Δ(S) to denote the set of distributions supported on S.

3.1 The White-Box Streaming/One-Way Communication Model

In the white-box streaming model, an algorithm is defined by a set of states Σ, a designated initial state σ0Σ, a stream length111As our main result is a lower bound, letting the algorithm know the stream length only makes the lower bound stronger. n>0, an alphabet Γ and a transition function 𝗆:Σ×Γ×{0,1}Σ to modify its state. The algorithm has access to randomness which is included in the transition function as a bit-string {0,1}. Such an algorithm runs in the presence of an adversary Λ whose task is to generate an input stream Γn for the algorithm.

The execution of an algorithm in the presence of the adversary takes place as follows: The algorithm starts in the initial state σ0 proceeds in n steps, where at the beginning of step i[n], the algorithm is in state σi1 and has sampled random strings R1,,Ri1. In step i, the adversary, that knows the algorithm and the random strings R1,,Ri1, generates an input γiΓ based on its knowledge. The algorithm then samples a random string Ri{0,1} and γi to update its state from σi1 to σi=𝗆(σi1,γi,Ri) and the random string Ri is added to the knowledge of the adversary. The state σn obtained after n steps is the output of the algorithm and number of bits log|Σ| needed to represent a state is the memory of the algorithm.

The white-box, one-way, communication model is similar, except that Σ is now the set of messages in the protocol, σ0 is a dummy message, n is the number of parties, Γ is the input space for (all the) parties, and 𝗆 is the function the parties use to compute their messages. Note that the identity of the party can be included in σ and does not need to be given explicitly to the message function. Also, note that when we reduce streaming problems to communication problems, we combine many symbols in the input stream to obtain the input of a single party in the communication model. This will decrease the number of parties n and blowup Γ and also limit the power of the adversary who now has to commit to more symbols at once. In particular, lower bounds proved for the communication model will translate to the streaming model as well.

3.2 Technical Lemmas

Lemma 2.

Let 𝒳 and 𝒴 be finite sets. Let 𝖷, 𝖷 and 𝖸, 𝖸 be random variables supported on 𝒳 and 𝒴 respectively. Suppose there is a function Λ:𝒳Δ(𝒴) such that for all x𝒳, we have 𝖽𝗂𝗌𝗍(𝖸𝖷=x)=𝖽𝗂𝗌𝗍(𝖸𝖷=x)=Λ(x). Let 𝖹 be a random variable independent of (𝖷,𝖷,𝖸,𝖸). For all functions f with a finite range, we have that:

𝖽𝗂𝗌𝗍(𝖸,f(𝖹,𝖷,𝖸))𝖽𝗂𝗌𝗍(𝖸,f(𝖹,𝖷,𝖸))TV𝖽𝗂𝗌𝗍(𝖷)𝖽𝗂𝗌𝗍(𝖷)TV.

Proof.

Let be the range of f. For all sets S𝒴×, we have:
(y,m)SPr((𝖸,f(𝖹,𝖷,𝖸))=(y,m))(y,m)SPr((𝖸,f(𝖹,𝖷,𝖸))=(y,m)) =x𝒳(y,m)SPr((𝖷,𝖸,f(𝖹,𝖷,𝖸))=(x,y,m)) x𝒳(y,m)SPr((𝖷,𝖸,f(𝖹,𝖷,𝖸))=(x,y,m)) =x𝒳(y,m)SPr((𝖷,𝖸)=(x,y))Pr(f(𝖹,𝖷,𝖸)=m(𝖷,𝖸)=(x,y)) x𝒳(y,m)SPr((𝖷,𝖸)=(x,y))Pr(f(𝖹,𝖷,𝖸)=m(𝖷,𝖸)=(x,y)) =x𝒳(y,m)SPr(𝖷=x)Λ(x)(y)Pr(f(𝖹,x,y)=m(𝖷,𝖸)=(x,y)) x𝒳(y,m)SPr(𝖷=x)Λ(x)(y)Pr(f(𝖹,x,y)=m(𝖷,𝖸)=(x,y)) (Definition of Λ) =x𝒳(y,m)SPr(𝖷=x)Λ(x)(y)Pr(f(𝖹,x,y)=m) x𝒳(y,m)SPr(𝖷=x)Λ(x)(y)Pr(f(𝖹,x,y)=m) (Independence of 𝖹 and (𝖷,𝖷,𝖸,𝖸)) =x𝒳(Pr(𝖷=x)Pr(𝖷=x))(y,m)SΛ(x)(y)Pr(f(𝖹,x,y)=m) x𝒳(Pr(𝖷=x)Pr(𝖷=x))𝟙((Pr(𝖷=x)Pr(𝖷=x))) (As for all x, we have 0(y,m)SΛ(x)(y)Pr(f(𝖹,x,y)=m)1) =𝖽𝗂𝗌𝗍(𝖷)𝖽𝗂𝗌𝗍(𝖷)TV.

3.3 Error Correcting Codes

Lemma 3.

Let n>0 be a large enough even integer. There exist a subset C{0,1}n of size 2n/500 such that all cC have Hamming weight222For convenience, we ignore divisibility issues and omit floor/ceiling signs. n/2 and for all ccC, the Hamming distance between c and c is at least 2n/5.

Proof.

Let c1,c2,,c2n/500 be independent and uniformly random elements of {0,1}n/2. By a Chernoff bound, we have for all ii[2n/500] that:

Pr(ci and ci have Hamming distance at most n/5)en/200.

By a union bound over all i,i, there exists c1,c2,,c2n/500 that satisfy the (fractional) Hamming distance property. For the Hamming weight property, simply concatenate all the ci with their complement (which preserves the fractional Hamming distance).

4 Reduction to a Communication Game

We now present a communication problem 𝖫𝖳𝖬𝖺𝗍 (Less Than Matrix) and show that a low-space white-box streaming algorithm that solves LIS implies a low communication white-box protocol for 𝖫𝖳𝖬𝖺𝗍. The problem 𝖫𝖳𝖬𝖺𝗍 is parameterized by an integer N and a parameter η>0. We write 𝖫𝖳𝖬𝖺𝗍N,η when we want to make these parameters explicit. The input to 𝖫𝖳𝖬𝖺𝗍N,η consists of two matrices A,B{0,1}N×N. For all i[N], we shall use Ai and Bi to denote the i-th row of A and B respectively. The goal of 𝖫𝖳𝖬𝖺𝗍N,η is to distinguish between the “no” case A=B and the “yes” case when for all i[N], we have that Ai,j<Bi,j for at least ηN different values of j[N]. If neither case holds, the protocol can output anything.

We consider one-way, 2N-party communication protocols for the problem 𝖫𝖳𝖬𝖺𝗍N,η, where there are 2N parties 𝖠1,,𝖠N and 𝖡1,,𝖡N and for all i[N], the inputs for party 𝖠i is Ai and the input for party 𝖡i is Bi. The messages go one-way where the first message is sent from 𝖠1 to 𝖡1, then second message is from 𝖡1 to 𝖠2, and so on till the party 𝖡N computes the output, which we treat as the last message. We now present our reduction.

Lemma 4.

Let N>0 and δ,η(0,1). For all S>0, if there exists a white-box streaming algorithm that computes a (1+η/2)-approximation to the length of the LIS in S-space with probability δ, then there exists a one-way, 2N-party, white-box communication protocol that solves the problem 𝖫𝖳𝖬𝖺𝗍N,η with probability δ where the length of every message is at most S bits.

Proof.

Fix a streaming algorithm 𝖠𝗅𝗀 as in the lemma statement and consider the following one-way communication protocol Π. As above, the parties send messages in the order 𝖠1,𝖡1,,𝖠N,𝖡N where for all i[N], the message sent by parties 𝖠i and 𝖡i is computed as follows: The party interprets the message received by it as the current memory state of the streaming algorithm 𝖠𝗅𝗀 (or the initial state, if the party is 𝖠1) and changes the entries of whatever input she gets from the white-box adversary by adding 2N(j1)+2(Ni) to the j-th entry, for all j[N]. It then simulates the algorithm 𝖠𝗅𝗀 on this changed input and sends the resulting memory state to the next party. In case this party is the last party 𝖡N, then the party outputs no if 𝖠𝗅𝗀 outputs a number at most N and outputs yes otherwise.

We claim that the protocol Π solves the problem 𝖫𝖳𝖬𝖺𝗍N,η. For this, we show that whenever the inputs to the parties belong to the no case of 𝖫𝖳𝖬𝖺𝗍N,η, then they simulate 𝖠𝗅𝗀 on a sequence with LIS at most N and whenever the inputs belong to the yes case of 𝖫𝖳𝖬𝖺𝗍N,η, then they simulate 𝖠𝗅𝗀 on a sequence with LIS at least N(1+η/2). Thus, a (1+η/2)-approximation to the length of the LIS is enough to determine the output of 𝖫𝖳𝖬𝖺𝗍N,η.

For this, consider any execution of the protocol Π and let A and B be the matrices formed by the inputs of the 2N parties. If these inputs belong to the no case, i.e., if A=B, then for all i[N], the inputs for the parties 𝖠i and 𝖡i are the same. Moreover, for all 1<iN and all j[N], we have that:

Ai,j+2N(j1)+2(Ni) =Ai,j+2N(j1)+2(N(i1))2
<Ai1,j+2N(j1)+2(N(i1)). (As A,B{0,1}N×N)

It follows that, for all j[N], the j-th coordinate of the parties’ inputs forms a non-increasing sequence which implies that the length of the LIS of the input stream to the algorithm 𝖠𝗅𝗀 is at most N. Next, consider the case when these inputs belong to the yes case. That is, for all i[N], we have that Ai,j<Bi,j for at least ηN different values of j[N]. Thus, the number of pairs (i,j) such that Ai,j<Bi,j is at least ηN2. By the pigeonhole principle, at least k=ηN/2 of these pairs have the same value of ji.

Let Δ be this value of ji and let (i1,j1),(ik,jk) be these values sorted in increasing order of the first coordinate. Define j0=0 for notational convenience and consider the N+k=N(1+η/2) values:

Ai1,j0+1,,Ai1,j1,Bi1,j1,,Aik,jk1+1,,Aik,jk,Bik,jk,Bik,jk+1,,Bik,N.

Observe that after adding our offsets, these values form a subsequence of the sequence 𝖠𝗅𝗀 is simulated on. We finish the proof by showing that these values are increasing. Because of our choice of these pairs, this follows if we show that, for all ii[N] and all 1<jN, we have:

max(Ai,j1,Bi,j1)+2N(j2)+2(Ni)
=max(Ai,j1,Bi,j1)+2N(j1)+2(Ni)+2(iiN)
<max(Ai,j,Bi,j)+2N(j1)+2(Ni). (As A,B{0,1}N×N)

5 The Communication Lower Bound

The goal of this section is to show the following theorem.

Theorem 5.

Let N>0 be large enough. Any one-way, 2N-party, white-box communication protocol that solves the problem 𝖫𝖳𝖬𝖺𝗍N,1/5 with probability at least 12+1N must have at least one message of length at least N/1000.

We prove Theorem 5 by contradiction. Fix an N>0 and suppose that there is a protocol such that every message in Π has length at most N/1000. Let =i[N/1000]{0,1}i be the space of all such messages and 𝖢{0,1}N be an error-correcting code with 2N/500 codewords where each codeword c𝖢 has Hamming weight N/2 and the pairwise distance between any two codewords cc𝖢 is at least 2N/5. Such codes are well known to exist and we also include an easy proof of their existence in Lemma 3.

All the white-box adversaries for the protocol Π that we consider will be such that, for all i[N], the party 𝖠i will get as input an independent, uniformly random element Ai𝖢. We will use 𝖠i to denote the random variable corresponding to333This means that we use 𝖠i to denote both the player and the random variable corresponding to the player’s input. Ai. Moreover, for all i[N], the input for the party 𝖡i will be sampled from a distribution that is determined by the message m that it receives from the party 𝖠i and the input Ai sampled for the party 𝖠i. In other words, for our purposes, a white-box adversary for the protocol Π can be defined by a vector of n functions Λ=(Λi)i[N] where for all i[N], the function Λi:×𝖢Δ(𝖢) is a function mapping a message and an input for the party 𝖠i to a distribution of inputs for the party 𝖡i.

Execution in the presence of an adversary.

An execution of the protocol Π in the presence of an adversary Λ=(Λi)i[N] as above proceeds in N steps, where for all i[N], step i involves the party 𝖠i sending a message to the party 𝖡i and the party 𝖡i sending a message to the party 𝖠i+1 (or computing the output, if i=N). In more details, for any i[N], let m𝖡i1 be the message sent by the party 𝖡i1 to the party 𝖠i in the previous step (or the empty message, if i=0). Then, in step i, the party 𝖠i gets an independent uniformly random element Ai𝖢. This party then uses its private randomness R𝖠i to compute a message m𝖠i=m𝖠i(R𝖠i,m𝖡i1,Ai), which is sent to the party 𝖡i. Then, the party 𝖡i gets an input Bi𝖢 sampled from the distribution Λi(m𝖠i,Ai) and uses its private randomness R𝖡i to compute a message m𝖡i=m𝖡i(R𝖡i,m𝖠i,Bi), which is sent to the party 𝖠i+1, and the protocol moves to the next step.

This means that, for the protocol Π, every adversary Λ=(Λi)i[N] induces a (joint) distribution on the tuple (Ai,Bi,m𝖠i,m𝖡i)i[N] consisting of the messages and the inputs of all the parties. We define (𝖠i(Λ),𝖡i(Λ),𝗆𝖠i(Λ),𝗆𝖡i(Λ))i[N] to be the random variables corresponding to this distribution. We extend this definition to the case i=0 by defining those random variables to be some dummy variables that are the same for all Λ. For any random variable 𝖷, we use the notation 𝖽𝗂𝗌𝗍(𝖷) to denote its distribution. From the discussion above, we have that, for all adversaries Λ=(Λi)i[N], all i[N], and all pairs (m𝖠i,Ai),(m𝖡i1,Bi1)×𝖢, it holds that (letting 𝔘(𝖢) denote the uniform distribution on 𝖢):

𝖽𝗂𝗌𝗍(𝖠i(Λ)(𝗆𝖡i1(Λ),𝖡i1(Λ))=(m𝖡i1,Bi1))=𝔘(𝖢),𝖽𝗂𝗌𝗍(𝖡i(Λ)(𝗆𝖠i(Λ),𝖠i(Λ))=(m𝖠i,Ai))=Λi(m𝖠i,Ai). (1)

The adversary 𝚲.

For all i[N], let Λi:×𝖢Δ(𝖢) be the function that ignores its first argument and outputs a distribution that is a point mass on its second argument (equivalently, outputs the second argument with probability 1). We define the adversary Λ=(Λi)i[N]. Observe that any input generated by the adversary Λ is supported only on inputs that belong to the no case of 𝖫𝖳𝖬𝖺𝗍N,1/5. Throughout this proof, we will often abbreviate the random variables corresponding to Λ by writing 𝖠i instead of 𝖠i(Λ), 𝗆𝖠i instead of 𝗆𝖠i(Λ), etc.. We will also abbreviate events of the form 𝗆𝖠i=m𝖠i to m𝖠i.

Bad messages.

Recall that in all adversaries we consider, for all i[N], the party 𝖠i gets as input an independent and uniformly random element Ai𝖢. Thus, each individual input Ai has probability 1|𝖢|=2N/500. We define a message m𝖠i to be “bad” if conditioning on i increases the probability of some input Ai to a value more than 1/2. Formally, the set of all such messages is:

𝖡𝖺𝖽i={m𝖠iAi𝖢:Pr(Aim𝖠i)>12}. (2)

The following lemma shows that messages are unlikely to be bad.

Lemma 6.

For all i[N], it holds that:

Pr(𝗆𝖠i𝖡𝖺𝖽i)<2N/2000.

Proof.

For all m𝖠i𝖡𝖺𝖽i, we have from Equation 2 that there exists Ai𝖢 such that Pr(Aim𝖠i)>12. As Pr(Aim𝖠i)Pr(Ai)Pr(m𝖠i)=2N/500Pr(m𝖠i), this means that Pr(m𝖠i)<21N/500 for all m𝖠i𝖡𝖺𝖽i. As 𝖡𝖺𝖽i, we get:

Pr(𝗆𝖠i𝖡𝖺𝖽i)|𝖡𝖺𝖽i|21N/500<2N/2000.

5.1 Auxiliary Distributions

Intuitively, Equation 2 says that a message m𝖠i is not in 𝖡𝖺𝖽i only if the distribution 𝖽𝗂𝗌𝗍(𝖠im𝖠i) is such that the probabilities of all Ai are most 1/2. For such distributions, we show the following lemma (stated for general functions).

Lemma 7.

Let Ω be a finite, non-empty set and f:Ω be a non-negative function such that 2maxωf(ω)ωf(ω). There exists a symmetric, non-negative function g:Ω×Ω such that for all ωΩ, we have g(ω,ω)=0 and ωg(ω,ω)=f(ω).

Proof.

Define f(Ω)=ωf(ω) for convenience. Proof by induction on |Ω|. The base case |Ω|=1 is trivial. For the case |Ω|=2, note that 2maxωf(ω)ωf(ω) implies that f takes a fixed value, say c, on all of Ω. In this case, defining g(ω,ω)=c𝟙(ωω) for all ω,ωΩ satisfies the conditions of the lemma.

We show the result for |Ω|3 assuming it holds for smaller values. As |Ω|3, we can sort all the elements ωΩ in increasing order of f(ω) and define f(ω3)f(ω2)f(ω1) to be the three largest values in this order (breaking ties arbitrarily). Assume for now that f(ω2)+f(ω3)f(Ω)2. In this case, define the function g according to the following matrix:

ω1ω2ω3ωω1( 0f(Ω)2f(ω3)f(ω1)+f(ω3)f(Ω)20) ω2f(Ω)2f(ω3)0f(ω2)+f(ω3)f(Ω)20ω3f(ω1)+f(ω3)f(Ω)2f(ω2)+f(ω3)f(Ω)20f(ω)ω00f(ω)0

It can be verified that g satisfies the conditions of the lemma. Now, assume that f(ω2)+f(ω3)<f(Ω)2. In this case, we apply the induction hypothesis on the set Ω=Ω{ω2} and the function f:Ω that matches f everywhere except that it is not defined on ω2 and that f(ω1)=f(ω1)f(ω2). As f(ω2)+f(ω3)<f(Ω)2, we have that f and Ω satisfy the condition in the induction hypothesis and we get a non-negative function g:Ω×Ω such that for all ωΩ, we have g(ω,ω)=0 and ω′′g(ω,ω′′)=f(ω). Now, define the function g to match g when neither of its arguments equals ω2. Otherwise, we define g(ω,ω2)=g(ω2,ω)=f(ω2)𝟙(ω=ω1) for all ωΩ. Observe that this satisfies the conditions of the lemma.

Observe that when Lemma 7 is applied to a function f that is a probability distribution over Ω, then the resulting function g represents a probability distribution on Ω×Ω.

Definition 8 (Auxiliary distribution).

Let Ω be a finite, non-empty set and f be a probability distribution on Ω such that maxωf(ω)1/2. Let g be the probability distribution (on the set Ω×Ω) obtained by applying Lemma 7 on f. For all ωΩ, we will use f𝖺𝗎𝗑(ω) to denote the distribution g conditioned on the first element being ω.

5.2 White-Box Adversaries

We now define the other white-box adversaries we consider. For this, we define functions (Λi,i)0i<iN where each such function is a map from ×𝖢 to Δ(𝖢). Let 0i<iN. We define the function Λi,i to be such that for all m𝖠i and all Ai𝖢, it outputs the distribution 𝖽𝗂𝗌𝗍(𝖠im𝖠i)𝖺𝗎𝗑(Ai) if m𝖠i𝖡𝖺𝖽i and otherwise, it outputs a uniformly random element in 𝖢{Ai}. Observe that the fact that m𝖠i𝖡𝖺𝖽i means that the former is well-defined. Using these, for all 0iN, we define the adversary Λi=((Λi)i[i],(Λi,i)i<iN)i[N]. Observe that ΛN=Λ and that the following holds:

Observation 9.

Let 0i1,i2<iN. Then, it holds that Λi1,i=Λi2,i.

Lemma 10.

For all i<i[N], we have that:

𝖽𝗂𝗌𝗍(𝗆𝖡i(Λi),𝖡i(Λi))𝖽𝗂𝗌𝗍(𝗆𝖡i(Λi1),𝖡i(Λi1))TV𝖽𝗂𝗌𝗍(𝗆𝖡i1(Λi),𝖡i1(Λi))𝖽𝗂𝗌𝗍(𝗆𝖡i1(Λi1),𝖡i1(Λi1))TV.

Proof.

To show the lemma, we show that the term

𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi),𝖠i(Λi))𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi1),𝖠i(Λi1))TV,

upper bounds the expression on the left and lower bounds the expression on the right. For the former, we apply Lemma 2 with 𝖷=(𝗆𝖠i(Λi),𝖠i(Λi)), 𝖷=(𝗆𝖠i(Λi1),𝖠i(Λi1)) and 𝖸=𝖡i(Λi), 𝖸=𝖡i(Λi1) and 𝖹=𝖱𝖡i, the private randomness of the party 𝖡i. Observe that the condition in Lemma 2 is satisfied as for all (m𝖠i,Ai)×𝖢, we have 𝖽𝗂𝗌𝗍(𝖡i(Λi)(𝗆𝖠i(Λi),𝖠i(Λi))=(m𝖠i,Ai))=Λi,i(m𝖠i,Ai) and we also have 𝖽𝗂𝗌𝗍(𝖡i(Λi1)(𝗆𝖠i(Λi1),𝖠i(Λi1))=(m𝖠i,Ai))=Λi1,i(m𝖠i,Ai) from Equation 1. These are the same by Observation 9.

For the latter, we use 𝖷=(𝗆𝖡i1(Λi),𝖡i1(Λi)), 𝖷=(𝗆𝖡i1(Λi1),𝖡i1(Λi1)) and 𝖸=𝖠i(Λi), 𝖸=𝖠i(Λi1) and 𝖹=𝖱𝖠i, the private randomness of the party 𝖠i when applying Lemma 2. Observe that the condition in Lemma 2 is satisfied as for all (m𝖠i,Ai)×𝖢, we have that both distributions 𝖽𝗂𝗌𝗍(𝖠i(Λi)(𝗆𝖡i1(Λi),𝖡i1(Λi))=(m𝖡i1,Bi1)) and 𝖽𝗂𝗌𝗍(𝖠i(Λi1)(𝗆𝖡i1(Λi1),𝖡i1(Λi1))=(m𝖡i1,Bi1)) are just the uniform distribution over 𝖢, and hence identical.

Lemma 11.

For all i,i[N], we have that:

𝖽𝗂𝗌𝗍(𝗆𝖡i(Λi),𝖡i(Λi))𝖽𝗂𝗌𝗍(𝗆𝖡i(Λi1),𝖡i(Λi1))TV{0, if i<i2N/2000, otherwise.

Proof.

Fix an arbitrary i[N] and observe that the case i<i is trivial. Moreover, due to Lemma 10, it suffices to consider the case i=i. For this, we will show that:

𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi),𝖡i(Λi))𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi1),𝖡i(Λi1))TV2N/2000.

This suffices as then we use Lemma 2 with 𝖷=(𝗆𝖠i(Λi),𝖡i(Λi)), 𝖷=(𝗆𝖠i(Λi1),𝖡i(Λi1)) and 𝖸=𝖡i(Λi), 𝖸=𝖡i(Λi1) and 𝖹=𝖱𝖡i, the private randomness of the party 𝖡i. The condition in Lemma 2 is satisfied as 𝖸 and 𝖸 are just the second coordinates of 𝖷 and 𝖷 respectively and Lemma 2 then finishes the proof. To see why the foregoing equation holds, note from the definition of Λi and Equation 1 that 𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi),𝖡i(Λi))=𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi),𝖠i(Λi)). This means that it suffices to show that:

𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi),𝖠i(Λi))𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi1),𝖡i(Λi1))TV2N/2000.

Next, note that the behavior of the adversaries Λi, Λi1, and Λ is the same for parties up to and including 𝖠i. This means that 𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi),𝖠i(Λi))=𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi1),𝖠i(Λi1))=𝖽𝗂𝗌𝗍(𝗆𝖠i,𝖠i) and it suffices to show that:

𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi1),𝖠i(Λi1))𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi1),𝖡i(Λi1))TV2N/2000. (3)

To show Equation 3, note that for all (m𝖠i,Bi)×𝖢 such that m𝖠i𝖡𝖺𝖽i, we have that:

Pr((𝗆𝖠i(Λi1),𝖡i(Λi1))=(m𝖠i,Bi))
=Ai𝖢Pr((𝗆𝖠i(Λi1),𝖠i(Λi1),𝖡i(Λi1))=(m𝖠i,Ai,Bi))
=Ai𝖢Pr((𝗆𝖠i(Λi1),𝖠i(Λi1))=(m𝖠i,Ai))Λi1,i(m𝖠i,Ai)(Bi) (Equation 1)
=Ai𝖢Pr((𝗆𝖠i(Λi1),𝖠i(Λi1))=(m𝖠i,Ai))𝖽𝗂𝗌𝗍(𝖠im𝖠i)𝖺𝗎𝗑(Ai)(Bi) (Definition of Λi1,i and as m𝖠i𝖡𝖺𝖽i)
=Ai𝖢Pr(m𝖠i,Ai)𝖽𝗂𝗌𝗍(𝖠im𝖠i)𝖺𝗎𝗑(Ai)(Bi) (As 𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi1),𝖠i(Λi1))=𝖽𝗂𝗌𝗍(𝗆𝖠i,𝖠i))
=Pr(m𝖠i)Ai𝖢Pr(Aim𝖠i)𝖽𝗂𝗌𝗍(𝖠im𝖠i)𝖺𝗎𝗑(Ai)(Bi)
=Pr(m𝖠i)Pr(𝖠i=Bim𝖠i) (Lemmas 7 and 8)
=Pr((𝗆𝖠i,𝖠i)=(m𝖠i,Bi))
=Pr((𝗆𝖠i(Λi1),𝖠i(Λi1))=(m𝖠i,Bi)) (As 𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi1),𝖠i(Λi1))=𝖽𝗂𝗌𝗍(𝗆𝖠i,𝖠i)).

With this equation, Equation 3 now follows as we get:

𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi1),𝖠i(Λi1))𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi1),𝖡i(Λi1))TV
Pr(𝗆𝖠i(Λi1)𝖡𝖺𝖽i)
=Pr(𝗆𝖠i𝖡𝖺𝖽i) (As 𝖽𝗂𝗌𝗍(𝗆𝖠i(Λi1),𝖠i(Λi1))=𝖽𝗂𝗌𝗍(𝗆𝖠i,𝖠i))
=2N/2000 (Lemma 6).

5.3 Finishing the Proof

Observe that the adversary ΛN=Λ always satisfies Ai=Bi for all i[N] and thus, always generates inputs in the no case of 𝖫𝖳𝖬𝖺𝗍N,1/5. We now show that the adversary Λ0 always generates inputs in the yes case of 𝖫𝖳𝖬𝖺𝗍N,1/5. As Ai,Bi𝖢 and 𝖢 only has codewords with Hamming weight N/2 and the pairwise distance between any two codewords 𝖢 is at least 2N/5, it suffices to show that the adversary Λ0 always satisfies AiBi. This is because of the definition of Λ0 and Lemma 7. However, applying Lemma 11 with i=N and using the triangle inequality, we get that:
𝖽𝗂𝗌𝗍(𝗆𝖡N(Λ0))𝖽𝗂𝗌𝗍(𝗆𝖡N(ΛN))TV 𝖽𝗂𝗌𝗍(𝗆𝖡N(Λ0),𝖡N(Λ0))𝖽𝗂𝗌𝗍(𝗆𝖡N(ΛN),𝖡N(ΛN))TV 2N/10000.

As N is large enough, it follows that either the probability that the last message (which is the output) of the protocol Π is yes when the adversary is Λ0 is at most 12+1N or the probability that the last message of the protocol Π is yes when the adversary is ΛN is at least 12. Theorem 5 follows straightforwardly in either case.

Finishing the proof of Theorem 1.

Theorem 1 immediately follows from Lemma 4 and Theorem 5.

References

  • [1] Miklós Ajtai, Vladimir Braverman, T. S. Jayram, Sandeep Silwal, Alec Sun, David P. Woodruff, and Samson Zhou. The white-box adversarial data stream model. In International Conference on Management of Data (PODS), pages 15–27, 2022. doi:10.1145/3517804.3526228.
  • [2] David Aldous and Persi Diakonis. Longest increasing subsequences: From patience sorting to the baik–deift–johansson theorem. Bull. Amer. Math. Soc. (N. S.), 36:413–432, 1999.
  • [3] Alexandr Andoni, Negev Shekel Nosatzki, Sandip Sinha, and Clifford Stein. Estimating the longest increasing subsequence in nearly optimal time. In 63rd IEEE Annual Symposium on Foundations of Computer Science, FOCS, pages 708–719, 2022. doi:10.1109/FOCS54457.2022.00073.
  • [4] Brian Babcock, Shivnath Babu, Mayur Datar, Rajeev Motwani, and Jennifer Widom. Models and issues in data stream systems. In Proceedings of the Twenty-first ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, pages 1–16. ACM, 2002. doi:10.1145/543613.543615.
  • [5] Omri Ben-Eliezer, Rajesh Jayaram, David P. Woodruff, , and Eylon Yogev. A framework for adversarially robust streaming algorithms. Journal of the ACM, 69(2), 2022. doi:10.1145/3498334.
  • [6] Amit Chakrabarti. A note on randomized streaming space bounds for the longest increasing subsequence problem. Information Processing Letters, 2012.
  • [7] Kuan Cheng, Alireza Farhadi, MohammadTaghi Hajiaghayi, Zhengzhong Jin, Xin Li, Aviad Rubinstein, Saeed Seddighin, and Yu Zheng. Streaming and small space approximation algorithms for edit distance and longest common subsequence. In 48th International Colloquium on Automata, Languages, and Programming, ICALP, volume 198 of LIPIcs, pages 54:1–54:20, 2021. doi:10.4230/LIPIcs.ICALP.2021.54.
  • [8] Funda Ergün and Hossein Jowhari. On the monotonicity of a data stream. Comb., 35(6):641–653, 2015. doi:10.1007/S00493-014-3035-1.
  • [9] Ying Feng, Aayush Jain, and David P. Woodruff. Fast white-box adversarial streaming without a random oracle. In Forty-first International Conference on Machine Learning, ICML, 2024.
  • [10] Ying Feng and David P. Woodruff. Improved algorithms for white-box adversarial streams. In International Conference on Machine Learning (ICML), volume 202, pages 9962–9975, 2023. URL: https://proceedings.mlr.press/v202/feng23d.html.
  • [11] Michael L. Fredman. On computing the length of longest increasing subsequences. Discrete Math., 11(1):29–35, January 1975. doi:10.1016/0012-365X(75)90103-X.
  • [12] Anna Gál and Parikshit Gopalan. Lower bounds on streaming algorithms for approximating the length of the longest increasing subsequence. SIAM Journal on Computing, 2010.
  • [13] Paweł Gawrychowski and Wojciech Janczewski. Fully dynamic approximation of lis in polylogarithmic time. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing (STOC), pages 654–667, 2021.
  • [14] Parikshit Gopalan, T. S. Jayram, Robert Krauthgamer, and Ravi Kumar. Estimating the sortedness of a data stream. In Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA, pages 318–327. SIAM, 2007. URL: http://dl.acm.org/citation.cfm?id=1283383.1283417.
  • [15] Dan Gusfield. Algorithms on Strings, Trees, and Sequences: Computer Science and Computational Biology. Cambridge University Press, Cambridge, MA, 1997. doi:10.1017/CBO9780511574931.
  • [16] Haim Kaplan, Yishay Mansour, Kobbi Nissim, and Uri Stemmer. Separating adaptive streaming from oblivious streaming using the bounded storage model. In Advances in Cryptology - 41st Annual International Cryptology Conference, CRYPTO Proceedings, Part III, volume 12827 of Lecture Notes in Computer Science, pages 94–121, 2021. doi:10.1007/978-3-030-84252-9_4.
  • [17] Tomasz Kociumaka and Saeed Seddighin. Improved dynamic algorithms for longest increasing subsequence. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing (STOC), pages 640–653, 2021. doi:10.1145/3406325.3451026.
  • [18] Xin Li and Yu Zheng. Lower bounds and improved algorithms for asymmetric streaming edit distance and longest common subsequence. In 41st IARCS Annual Conference on Foundations of Software Technology and Theoretical Computer Science, FSTTCS, volume 213 of LIPIcs, pages 27:1–27:23, 2021. doi:10.4230/LIPIcs.FSTTCS.2021.27.
  • [19] Xin Li and Yu Zheng. Streaming and query once space complexity of longest increasing subsequence. In Computing and Combinatorics: 29th International Conference, COCOON, pages 29–60, 2023. doi:10.1007/978-3-031-49190-0_3.
  • [20] S. Muthukrishnan. Data Streams: Algorithms and Applications. Found. Trends Theor. Comput. Sci., Now Publisher, Hanover, MA, 2005.
  • [21] Pavel Pevzner. Computational Molecular Biology: An Algorithmic Approach. MIT Press, Cambridge, MA, 2003.
  • [22] Aviad Rubinstein, Saeed Seddighin, Zhao Song, and Xiaorui Sun. Approximation algorithms for lcs and lis with truly improved running times. In Symposium on Foundations of Computer Science (FOCS), 2019.
  • [23] Michael E. Saks and C. Seshadhri. Space efficient streaming algorithms for the distance to monotonicity and asymmetric edit distance. In Proceedings of the Twenty-Fourth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA, pages 1698–1709, 2013. doi:10.1137/1.9781611973105.122.
  • [24] Michael E. Saks and C. Seshadhri. Estimating the longest increasing sequence in polylogarithmic time. SIAM J. Comput., 46(2):774–823, 2017. doi:10.1137/130942152.
  • [25] Xiaoming Sun and David P. Woodruff. The communication and streaming complexity of computing the longest common and increasing subsequences. In Symposium on Discrete Algorithms (SODA), 2007.