Spanoids - an abstraction of spanning structures, and a barrier for LCCs

We introduce a simple logical inference structure we call a $\textsf{spanoid}$ (generalizing the notion of a matroid), which captures well-studied problems in several areas. These include combinatorial geometry, algebra (arrangements of hypersurfaces and ideals), statistical physics (bootstrap percolation) and coding theory. We initiate a thorough investigation of spanoids, from computational and structural viewpoints, focusing on parameters relevant to the applications areas above and, in particular, to questions regarding Locally Correctable Codes (LCCs). One central parameter we study is the $\textsf{rank}$ of a spanoid, extending the rank of a matroid and related to the dimension of codes. This leads to one main application of our work, establishing the first known barrier to improving the nearly 20-year old bound of Katz-Trevisan (KT) on the dimension of LCCs. On the one hand, we prove that the KT bound (and its more recent refinements) holds for the much more general setting of spanoid rank. On the other hand we show that there exist (random) spanoids whose rank matches these bounds. Thus, to significantly improve the known bounds one must step out of the spanoid framework. Another parameter we explore is the $\textsf{functional rank}$ of a spanoid, which captures the possibility of turning a given spanoid into an actual code. The question of the relationship between rank and functional rank is one of the main questions we raise as it may reveal new avenues for constructing new LCCs (perhaps even matching the KT bound). As a first step, we develop an entropy relaxation of functional rank to create a small constant gap and amplify it by tensoring to construct a spanoid whose functional rank is smaller than rank by a polynomial factor. This is evidence that the entropy method we develop can prove polynomially better bounds than KT-type methods on the dimension of LCCs.


Introduction
This (somewhat long) introduction will be organized as follows. We begin by discussing Locally Correctable Codes (LCCs) and the main challenges they present as this was the primary motivation for this work. We proceed to define spanoids as an abstraction of LCCs, and state some results about their rank which hopefully illuminate the difficulties with LCCs in a new light. We continue by describing other natural settings in which the spanoid structure arises in the hope of motivating the questions raised in the context of LCCs and demonstrating their potential to contribute to research in other areas. We then turn to the investigation of functional rank of spanoids, which aims to convert them to actual LCCs. We conclude with describing some of the structural results about spanoids obtained here.

Locally Correctable Codes
The introduction of locality to coding theory has created a large body of research with wideranging applications and connections, from probabilistically checkable proofs, private information retrieval, program testing, fault-tolerant storage systems, and many others in computer science and mathematics. We will not survey these, and the reader may consult the surveys [Yek12,Dvi12]. Despite much progress, many basic questions regarding local testing, decoding and correcting of codes remain open. Here we focus on the efficiency of locally correctable codes, that we now define. Note that the related, locally decodable codes (LDCs), will not be discussed in this paper, as our framework is not relevant to them (LCCs can be converted to LDCs with a small loss in parameters).

Definition 1.1 (q-LCCs).
A code C ⊆ Σ n is called a q-query locally correctable code with errortolerance δ > 0, if for every i ∈ [n] there is a family (called a q-matching) M i , of at least δn disjoint q-subsets of [n], with the following decodability property. For every codeword c ∈ C, and for every i ∈ [n], the value of c i is determined * by the values of c in coordinates S, for every q-subset S in M i . † Intuitively, given a vector c ′ ∈ Σ n which results from corrupting less than (say) ǫδn coordinates of a codeword c ∈ C, recovering c i for any given i ∈ [n] is simple. Picking a random q-subset from M i and decoding c i according to it will succeed with probability at least 1 − ǫ, as only an ǫ-fraction of these q-subsets can be corrupted . We focus in this paper on the most well-studied and well-motivated regime where both the "query-complexity" q and the error-tolerance δ are constants. It is not hard to see that there are no LCCs with q = 1 (unless the dimension is constant) and so we will start with the first interesting case of q = 2. A canonical example of a 2-query LCC, which will serve us several times below, is the Hadamard code. Here Σ = F 2 . Let k be any integer and set n = 2 k − 1. Let A be the k × n matrix whose columns are all non-zero k-bit vectors. The Hadamard code C H ∈ F n 2 is generated by A, namely H consists of all linear combinations of rows of A. Since every column of A can be written as a sum of (namely, spanned by) pairs of other columns in (n − 1)/2 different ways, the matching M i suggest themselves, and so is the linear correcting procedure: add the values in coordinates of the random pair S from M i to determine the ith coordinate.
A central parameter of codes is their rate, capturing the redundancy between the dimension, namely the number of information bits encoded (here k), and the length of the codeword (here n). As in this paper this k will be a tiny function of n, we will focus on the dimension itself. Note that in the example above, as in every linear code, this dimension is also the rank of the generating matrix. In general codes, dimension may be fractional, and is defined as follows. All logarithms are in base 2 unless otherwise noted. Definition 1.2 (Dimension and rate of a code). For a general, possibly non-linear code C ⊆ Σ n , we define the dimension of C to be dim(C) = log |C|/ log |Σ|. Note that this coincides with the linear algebraic definition of dimension when C is a subspace. We refer to the ratio dim(C)/n as the 'rate' of the code.
Note that while the Hadamard code (C H ) has fantastic local correction (only 2 queries), its dimension is only k ∼ log n, which is pathetic from a coding theory perspective. However, no better 2-query LCC can exist, regardless of the alphabet. • There exists a 2-query LCC of dimension Ω(log n) and constant δ (Folklore: Hadamard code).
While we know precisely the optimal dimension for 2 queries, for q ≥ 3 the gap between known upper and lower bounds is huge. The best lower bounds (constructions) are polylogarithmic: they come from Reed-Muller codes (using polynomials over finite fields), and yield dimension Ω((log n) q−1 ).
The best LCC upper bounds are only slightly sub-linear, giving dim(C) ≤ O(n 1− 1 q−1 ) (up to logarithmic factors). This bound, which we will refer to as the Katz-Trevisan (KT) bound, is actually a slight refinement/improvement over the bound originally appearing in [KT00] (which gave n 1−1/q ). This improvement was implicit in several works (e.g. [DK11,Woo07]) and is explicitly stated in [IS18]. We should also note that, over constant-size alphabets, Kerenidis and De-Wolf proved an even stronger bound using quantum information theory [KW04]. This exponential gap between the upper and lower bounds, which we formally state below, has not been narrowed in over two decades. ‡ Explaining this gap (in the hope of finding ways to close it) is one major motivation of this work. Theorem 1.4 (q-LCCs, q ≥ 3). For every fixed q ≥ 3 and all large enough n: • There exists a q-query LCC of dimension Ω((log n) q−1 ) (with constant δ and alphabet of size q + 1) (Reed-Muller codes, see e.g. the survey [Yek12]).
• Every q-query LCC must have dimension at most O(n 1− 1 q−1 ) (for any constant δ and any alphabet) [IS18]. ‡ For LDCs better constructions than Reed-Muller codes are known, through the seminal works of [Yek08,Efr09], but as mentioned we will not discuss them here. Still, the upper bounds for LDCs are the same as for LCCs, and obtained by the same KT-type argument, so the results in this paper may serve to better understand the (smaller, but still quite large) gap between upper and lower bounds in LDCs as well.

Spanoids
We shall now abstract the notion of inference used in LCCs. There, for a collection of pairs (S, i) with S ⊆ [n] and i ∈ [n], the values of codewords in coordinate positions S, determine the value of of some other coordinate i. We shall forget (for now) the underlying code altogether, and abstract this relation by the formal "inference" symbol S → i, to be read "S spans i".
The pair (S, i) will sometimes be written as S → i and read as S spans i in the spanoid S.
One natural way to view a spanoid is as a logical inference system, with the pairs indicating all inference rules. The elements of [n] indicate some n formal statements, and an inference S → i of the spanoid means that if we know the truth of the statements in S, we can infer the truth of the ith statement. With this intuition, we shall adopt the convention that the inferences i → i are implicit in any spanoid, and that monotonicity holds: if S → i then also S ′ → i for every S ′ ⊇ S. These conventions will be formally stated below when we define general derivations, which sequentially combine these implicit rules and the stated rules (pairs) of the spanoid.
A key concept of spanoids is, naturally, the span. Given a subset T ⊂ [n] (which we can think of as "axioms"), we can explore everything they can span by a sequence of applications of the inference rules of the spanoid S.
and there exists S ⊂ T j−1 such that (S, i j ) ∈ S is one of the spanoid rules.
The span (or closure) of T , denoted span S (T ), is the set of all i for which T |= S i. We shall remove the subscript S from these notations when no confusion about the underlying spanoid can arise, and write T |= i and span(T ) for short.
Despite being highly abstract, we will see that spanoids can lead to a rich family of questions and definitions. The first, and perhaps one of the most central definitions is that of the rank of a spanoid. We shall see other notions of spanoid rank later on (and will discuss the relation between them). Definition 1.7 (Rank). The rank of a spanoid S, denoted rank(S), is the size of the smallest subset T ⊆ [n] such that span(T ) = [n]. Note that by the definition of span we always have rank(S) ≤ n.
We note that the "rank" of a logical inference system does appear (under different names) in proof complexity. It is the starting point for expansion-based lower bounds on a variety of proof systems, as introduced for Resolution proofs in [BW01], and used for many others e.g. in [ABRW04] and [AR01]). We shall return to this connection presently.
We can now define the spanoid analog of q-LCCs as spanoids which only specify the correction structure (the matchings M i ) without requiring any codewords or alphabet. Definition 1.8 (q-LCS, Locally correctable spanoid). A spanoid S over [n] is a q-LCS with errortolerance δ if for every i ∈ [n] there exists a family M i of at least δn disjoint q-subsets of [n] such that for each S ∈ M i we have (S, i) ∈ S. Namely, each i ∈ [n] is spanned (in S) by at least δn disjoint subsets of q-elements.
One can now ask about the highest possible rank of a q-LCS. It is not hard to see that the existence of a q-LCC (over any alphabet) C ⊂ Σ n with dimension dim(C) = d automatically implies that there exists a q-LCS (namely, the one given by the same matchings used in C) with rank at least ⌈d⌉. Indeed, otherwise there would be r < d coordinates in [n] that determine any codeword c ∈ C and this would limit the number of codewords to Σ r .
One of our main observations is that, remarkably, in locally correctable spanoids there is no gap between the upper and lower bounds: we know the precise answer up to logarithmic factors, and it matches the upper bounds for LCCs! Observe the analogies to the theorems in the previous subsection, for q = 2 and q ≥ 3. Theorem 1.9 (2-LCSs). For all large enough n: • There exists a 2-LCS over [n] with error-tolerance δ of rank Ω( 1 δ log(δn)). • Every 2-LCS over [n] with error-tolerance δ must have rank at most O 1 δ log(n) . § Here, of course, the inference structure of the Hadamard code proves the first item. To get the required dependence on δ, one can take 1 δ disjoint copies of such spanoids. The second item requires a new proof we discuss below, which generalizes (and implies) the one in Theorem 1. 3. It is quite surprising that, even in this abstract setting, with no need for codewords or alphabet, one cannot do better than the Hadamard code! We now state our results for q ≥ 3.
Theorem 1.10 (q-LCSs with q ≥ 3). For every fixed q ≥ 3 and all large enough n: • There exist a q-LCS of rankΩ(n • Every q-LCS over [n] has rank at mostÕ n 1− 1 q−1 (for any constant δ).
Both parts of this theorem demand discussion. The possibly surprising (and tight) lower bound follows from a simple probabilistic argument (indeed, one which is repeatedly used to prove expansion in the proof complexity references cited above), where the matchings M i are simply chosen uniformly at random. It seems to reveal how significant a relaxation spanoids are of LCCs (where probabilistic arguments fail completely). However, the best known LCC upper bound (Theorem 1.4) does not rule out the possibility that, at least for large alphabets, the two (LCC's dimension and LCS's rank) have the same behavior! From a more pessimistic (and perhaps more realistic) perspective, our lower bound shows the limitations of any (upper bound) proof technique which, in effect, applies also for spanoids. These are proofs in which the LCC structure is used to show that a small subset spans all the others. We note that there are several LCC upper bounds which 'beat' the n 1− 1 q−1 bound for certain very special cases by using additional structure not present in the corresponding abstract spanoid. One example is the bound of [KW04], which uses arguments from quantum information theory to roughly halve the number of queries, over binary (or small) alphabets. Another example is the paper [DSW14], which gives an improved upper bound on the dimension for linear 3-LCCs defined over the real numbers, using specific properties of the Reals such as distance and volume arguments. § The results of [BGT17] can be interpreted as an upper bound of O (poly(1/δ) log(n)) on the rank of 2-LCS with error-tolerance δ.
Our proof of the upper bound, is again more general than for LCCs, and interesting in its own right. We use a simple technique which performs random restrictions and contractions of graphs and hypergraphs (and originates in [DSW14]). It will be described in Section 3, after we have formulated an equivalent, set-theoretic formulation of spanoids in Section 2.2.

Functional rank: bridging the gap between LCCs and LCSs
We conclude this section of the introduction with an attempt to understand (and possibly bridge) the gap between LCCs and their spanoid abstraction. The idea is to start with an LCS of high rank (which we know is possible), and convert it to an LCC without losing too much in the parameters. More generally, for a given spanoid S, we would like to investigate the code C with largest dimension (over any alphabet Σ) which would be consistent with the inferences of S. This is captured in the notion of functional rank which we now define. . A code C ⊂ Σ n is consistent with S if for every inference (S, i) in S, and for every codeword c ∈ C, its values of coordinates S determine its value in coordinate i (by some fixed function, f S,i not depending on c). ¶ Define the functional rank of S, denoted f-rank(S), to be equal to the supremum of the dimension dim(C), over all possible finite alphabets Σ and codes C ⊂ Σ n which are consistent with S.
Of course, the strategy of constructing LCCs in two stages as above can only work if we can bound the gap between rank(S) and f-rank(S). This question, of bounding this gap or proving it can be large, is perhaps the most interesting one we raise (and leave mostly open for now). For now, we are able to show an example in which the two are different. The example providing a gap is depicted in Figure 1, arranging the coordinates as the vertices of a pentagon, the pair of vertices of each edge span the vertex opposite to it. That is,  x 3 x 4 x 5 Theorem 1.12 (Constant gap between rank and functional rank). The pentagon spanoid Π 5 depicted in Figure 1 has rank(Π 5 ) = 3 but f-rank(Π 5 ) = 2.5.
Seeing that rank(Π 5 ) = 3 is easy by inspection. The lower bound of 2.5 on functional rank comes from a set-theoretic construction of consistent codes we will describe (in more generality) in Section 5.2, where we develop an linear programming (LP) relaxation for rank(S) called LP cover (S), ¶ One can think of a code consistent with S also as a 'representation' of S in the spirit of matroid theory.
but surprisingly this LP captures the best set-theoretic construction of consistent codes. But even in this small example, the upper bound on functional rank is nontrivial to determine, as we allow all possible alphabets and consistent codes. Not surprisingly, Shannon entropy is the key to proving such a bound. In Section 5.1, we develop a linear programming relaxation, based on entropy whose optimum LP entropy (S) upper bounds f-rank(S). In this example, it proves 2.5 to be the optimum.
One natural way of amplifying gaps as in the example above, which may also be useful in creating codes of high functional rank, is the idea of tensoring. In Section 6, we develop different notions of tensoring spanoids inspired by tensoring of codes. In particular, we define a product of spanoids called the semi-direct product under which rank is multiplicative and f-rank is sub-multiplicative. By repeatedly applying this product to Π 5 , we get a spanoid with polynomial gap between f-rank and rank. Summarizing, we have the following obvious inequalities between the measures we described so far for every spanoid S. We feel that understanding the exact relationships better is worthy of further study LP cover (S) ≤ f-rank(S) ≤ LP entropy (S) ≤ rank(S). (1)

Other motivations and incarnations of spanoids
We return to discuss other structures, combinatorial, geometric and algebraic, in which the same notions of span and inference naturally occur, leading to a set-theoretic one that elegantly captures spanoids precisely. These raise further issues, some of which we study in this paper and some are left for future work. These serves to illustrate the breadth of the spanoid framework.

Bootstrap percolation and gossip processes
The following general set-up occurs in statistical physics, network theory and probability theory. Fix an undirected graph G([n], E). In a gossip or infection process, or equivalently bootstrap percolation, we are give a set of "rules" specifying, for every vertex v ∈ [n], a family of subsets of its neighbors. The intended meaning of such a rule is that if every member of one such subset is "infected" at a certain time step, then the vertex v becomes infected in the next time step. Given a set of initial infected vertices, this defines a process in which infection spreads, and eventually stabilizes. A well studied special case is the (uniform) r-bond percolation [Bol68], where the family for each vertex is all r-subsets of its neighbors. Many variants exist, e.g. one can have a similar process on the edges, rather than vertices of the graph. An important parameter of such a process is the following: what is the size of the smallest set of vertices which, if infected, will eventually infect all other vertices. A moment's thought will convince the reader that this structure is precisely a spanoid (where inferring sets are restricted by the graph structure). The infection process is precisely the inference process defining span in spanoids. Furthermore, the smallest size of an infecting set is precisely the rank of that spanoid! Much work has been invested to determine that rank even in very special cases, e.g. for the r-bond percolation above, in e.g. Boolean hypercubes, where it is known This turns out to be crucial for understanding, at least for certain structured graphs like lattices studied by physicists, the threshold probability for percolation when initial infections are random.
precisely. Interestingly, the paper [HHQ17] uses the so-called "polynomial method" to reprove that bound, which fits even deeper with our framework. In our language, their method determines the functional rank of this spanoid, and one direction is through constructing an explicit code that is consistent with the spanoid! The reader is encouraged to work out the details.

Independence systems and Matroids
An independence system over [n] is a family F of subsets of [n] which is downwards-closed (if a set is in F, so are all its subsets). The members of F are called independent. While much of what we say below generalizes to all independence systems, we specify them for the important special systems called matroids.
A matroid is an independence system in which the independent sets satisfy the so-called "exchange axiom" (which we will not define here). Matroids abstract linear independence in subsets of a vector space over a field * * , and capture algorithmic problems in which optimization is possible through the greedy algorithm. Matroids thus come with natural notions of span and rank, extending the ones in the linear algebraic setting. The rank of a set is the size of the largest independent set it contains. The span of a set is the maximal superset of it of the same rank. A matroid can thus be naturally viewed as a spanoid, with the inference rules F → i for every independent F ∈ F and every i for which F ∪ {i} is not independent (such minimal dependent sets as F ∪ {i} are called cycles). It is easy to verify that the notions of span and rank of the matroid and the spanoid it defines coincide. This also raises the natural question of bounding the gap between f-rank and rank for the special case of spanoids arising from matroids.
Note that a spanoid resulting from a matroid this way is symmetric: by the exchange property of matroids, if E ⊂ [n] is a cycle of F, then for every i ∈ E it contains the inference E \ {i} → i. Symmetric spanoids are interesting, and we note that the pentagon example witnessing the gap between rank and functional rank is not symmetric, and we do not know such a gap for symmetric spanoids. We also don't know if symmetric spanoids can achieve the lower bound in Theorem 1.10.

Point-line incidences
Sylvester-Gallai theorem is a celebrated result in combinatorial geometry conjectured by Sylvester and proved independently by Melchior and Gallai. It states that for any set of n points in Euclidean space R d , if the line through any two points passes through a third point, then they must all be collinear (namely, they span a 1-dimensional affine space). Over the complex numbers, one can prove a similar theorem but with the conclusion that the points span a 2-dimensional affine space (and there are in fact two dimensional examples known) [Kel86]. Over finite fields the conclusion is even weaker, saying that the span has dimension at most O(log(n)) and this is tight as the example of all points in F k p with n = p k shows. It is not a coincidence that this example reminds one of the Hadamard code described before as an example of a 2-query LCC. It is in fact true that there is a tight connection between configurations of points with many collinear triples and linear 2-query LCCs. This was first noticed in [BDYW11,BDWY12] and was used to prove that 2-LCCs do not exist over the characteristic zero fields (for q ≥ 3 these questions are wide open with even larger gaps than in the finite field case). LCCs with more than 2 queries naturally correspond to point configurations with many (q − 2)-dimensional affine spaces containing at least q points.
Given the connections between Sylvester-Gallai type incidence structures and LCCs, and the insights offered by spanoids for studying LCCs, it is natural that the study of the spanoid structures can help us get new insights on incidence geometry problems. A dual way to view these incidences, which we shall presently generalize, is to consider each point p i : i ∈ [n] as representing a hyperplane F i through the origin (in the appropriate vector space) vanishing on the linear function defined by p i . A point p i is spanned by a collection of points {p j : j ∈ S} iff F i ⊃ ∩ j∈S F j . Therefore the spanning structure of the points p 1 , p 2 , . . . , p n is captured by the spanoid where we would add the inference S → i iff F i contains the common intersection of all F j : j ∈ S.

Systems of polynomial equations
Given the above example, there is no reason to stop at the linear setting. Instead of lines we can consider n (multivariate) polynomials f i over a field, and again consider F i to be the zero set of f i . The spanoid above, having an inference S → i whenever the set F i contains the common intersection of all F j : j ∈ S, is capturing another natural algebraic notion. Namely, it says that the polynomial f i vanishes on the all the common roots of the polynomials {f j : j ∈ S}. By the celebrated Hilbert's Nullstellensatz theorem, over algebraically closed fields, this implies that f i belongs to the radical of the ideal generated by the f j 's. Here the rank function is far from being that of a matroid; the complex spanoid which arises (and in general is far from understood) plays a role in arithmetic complexity (a beautiful example is the recent [Shp18] dealing with degree-2 polynomials).

Intersecting set systems
Let us remove all restrictions from the origin or nature of the n sets F i discussed in the previous discussion. Assume we are given any such family F of sets (from an arbitrary universe, say U ). As above, a natural spanoid S F will have the inference S → i whenever the set F i contains the common intersection of all F j : j ∈ S. Such situations (and hence, spanoids) arise in many questions of extremal set theory, for example the study of (weak) sunflowers, or families with certain forbidden intersection (or union) patterns, e.g. [FLS12,EFF85,F96].
What is interesting in this more general framework, where the initial family of sets F is arbitrary, is that it becomes equivalent to spanoids! In other words, every spanoid S arises as S F of some family of sets F. This possibly surprising fact is not much more than an observation, but it turns out to be an extremely useful formulation for proving some of the results in this paper. Let us state it formally (it will be proved in Section 2.2). Theorem 1.14 (Spanoids and intersecting sets). Let S be any spanoid on [n]. Then, there exists a universe U and a family F of n sets of U , It is convenient to assume that the sets in F have no element in common to all † † . The notions of rank and span are extremely simple in this set-theoretic setting, and do not require the sequential "derivation" and the implicit ordering which we require to define these in spanoids. For a family F of n sets and a subset S ⊂ [n], let us denote by ∩S the subset of U which is the intersection of all {F j : j ∈ S}. Then the rank of S is the size of the smallest subset S ′ ⊆ S for which ∩S ′ = ∩S. Similarly, the span of S is the largest superset S ′′ ⊇ S for which ∩S ′′ = ∩S.
These static definitions of rank and span make many things transparent. For example, the expected fact that testing if the rank of a spanoid (namely the rank of the set [n]) is at most some given integer k is N P − complete (Claim 2.8). Complementing the sets F i in F, and replacing intersection with union, this is precisely the Set Cover problem. This connection also underlies the cover-based linear program discussed earlier, as well as proofs of the main quantitative results Theorem 1.9 and Theorem 1.10. Union-closed families are interesting combinatorial objects with a rich structure. The widely open Frankl's union-closed conjecture states that in every union-closed family of N sets, there exists an element which is contained in at least N/2 sets. Though this was proved for various special classes (see survey [BS15]), the best general bound is Ω(N/ log N ) due to [Kni94,Wój99]. When seen in the framework of spanoids, this follows immediately from Claim 2.5 which says that there is a log N sized hitting set for every union-closed family of N sets. Thus there is an element which should hit at least N/ log 2 (N ) sets. We hope that viewing union-closed families as spanoids could be of use in understanding them.

Organization
In Section 2, we will present two alternative ways to represent spanoids that will be very useful.
In Section 3, we show upper bounds on the rank of q-LCSs for q ≥ 2 thus proving the upper bounds in Theorems 3.5 and 3. 7. In Section 4, we construct q-LCSs thus proving the lower bound in Theorem 3. 7. In Section 5, we explore the relation between rank and f-rank of a spanoid. Specifically, we define the linear programs for LP entropy and LP cover which provide upper and lower bounds on the f-rank respectively. We will also calculate the rank and f-rank of the Pentagon spanoid defined in the introduction ( Figure 1) and thus proving that f-rank can be strictly smaller. In Section 6, we study products of spanoids which we use to amplify the gap between f-rank and rank from constant to polynomial.

Preliminaries on spanoids
We will now describe two equivalent ways to define spanoids which will turn out to be very useful.

Spanoids as union-closed or intersection-closed families
In this subsection, we will define an equivalent and more canonical way of describing spanoids in terms of intersection closed or union closed families. We have defined spanoids in Definition 1.5 by specifying the initial set of derivation rules such as A → i. But two different initial set of rules can lead to the same set of derivations and we should consider two spanoids to be equivalent if they lead to the same set of derivations. We will present an alternative way to describe spanoids which makes them equivalent to union-closed families of sets or alternatively intersection-closed families of sets. Moreover this new representation is a more canonical way to represent spanoids since it will be based only on the set of derivations. For this, the main new notions we need to define are that of a 'closed set' and an 'open set'.  Proof.
(1) Let F = F 1 ∩ F 2 be the intersection of two closed sets. Suppose in contradiction that F spans some element x ∈ [n] \ F then, by monotonicity, both F 1 and F 2 have to span x. Hence, (2) This just follows from (1) by taking complements.
(3) Let F (A) be the intersection of all closed sets containing A. Since span(A) is a closed set we clearly have F (A) ⊂ span(A). To see the other direction, suppose x ∈ span(A) and let F be any closed set containing A. Then, by monotonicity, F must also span x and so we must have x ∈ F . And A |= i in S iff i ∈ span(A). Thus given the set of all closed sets, we can reconstruct all the derivations of the spanoid. ‡ ‡ Closed sets are analogous to 'flats' or 'subspaces' in matroids.
For the converse, suppose we are given an intersection-closed family of subsets of [n], say C. We can define span C (A) = B⊃A,B∈C B and define a spanoid S C where A |= i iff i ∈ span C (A). It is easy to see that the closed sets of this spanoid S C is exactly C.
Thus an equivalent way to define a spanoid is to define all its closed (open) sets which is some intersection (union) closed family. The following claim shows that the rank of a spanoid has a very natural interpretation in terms of the open sets. This interpretation of the rank is used in Section 5.2 to give a linear programming relaxation LP cover which lower bounds the rank. We can also upper bound the rank of a spanoid in terms of the number of closed or open sets as the following claim shows.
Proof. Let r = rank(S) and R ⊂ [n] be a set of size |R| = r spanning [n]. Since the rank of S is r we know that R is independent (not spanned by any proper subset). For each of the 2 r subsets S ∈ 2 R we consider the closed set F S = span(S). We claim that all of these are distinct. Suppose in contradiction that there were two distinct sets S = T ∈ 2 R with span(S) = span(T ). W.l.o.g suppose there is an element x ∈ T \ S. Then x ∈ span(S) and so we get that R \ {x} spans R (by monotonicity) and so spans the entire spanoid in contradiction. Thus |C S | ≥ 2 r .

Spanoids as set systems
In this subsection, we will show yet another way of representing spanoids by families of sets. This representation (which is equivalent to spanoids) will be easier to work with and, in fact, we will later work almost exclusively with it instead of with the definition given in the introduction. Recall the notation introduced at the end of the introduction that, for sets S 1 , . . . , S n and for a subset A ⊂ [n] we let ∩A = ∩ i∈A S i .  Proof. For i ∈ [n] we define S i ⊂ C S to be the subfamily of closed sets of S containing the element i ∈ [n]. For the first direction of the proof suppose that A spans x in the spanoid S. Then, by Claim 2.2, x belongs to any closed set containing A and so ∩ i∈A S i ⊂ S x . For the other direction, suppose ∩ i∈A S i ⊂ S x or that any closed set containing A must also contain x. Hence, x is in the intersection of all closed sets containing A and, by Claim 2.2 we have that x ∈ span(A).
An alternative way to represent spanoids is by unions. (T 1 , T 2 , . . . , T n ) is called a union setrepresentation of the spanoid S when, A |= i in S iff T i ⊂ ∪ j∈A T j . Note that if (S 1 , S 2 , . . . , S n ) is an (intersection) set-representation for S as in Lemma 2.7, then by taking complements, (S 1 ,S 2 , . . . ,S n ) is a union set-representation for S and vice versa. Thus these two notions of representing a spanoid by sets is equivalent.
Claim 2. 8. Given a spanoid S and some positive integer k, deciding if the rank of the spanoid is at most k is NP-complete. Proof. Given the description of a spanoid and a subset of its elements, we can check in polynomial time whether the subset has size at most k and spans all the elements. So the problem is in N P . To prove that it is N P − complete, we reduce Set Cover problem to this.
Given a collection of sets S 1 , S 2 , . . . , S n ⊂ U such that ∪ i S i = U and some positive integer k, the Set Cover problem asks if there are at most k sets in the collection whose union is U . To reduce it to the spanoid rank problem, we can create a spanoid over [n] elements where the inference rules are given by A |= i iff ∪ j∈A S j ⊃ S i . The rank of this spanoid is at most k iff there exists k sets in the collection which cover all of U .

Upper bounds on the rank of q-LCSs
In this section we prove the upper bounds on the rank of q-LCSs stated in Theorems 1.9 and 1.10. The proofs will rely on the set representation described in Section 2.2 and on random restriction and contraction arguments given below.

Graph theoretic lemmas
In this subsection, we will prove a key technical lemma about a random graph process that will be useful for proving upper bounds on the rank of q-LCSs. We denote by D(n) the set of simple directed graphs on n vertices. We always assume w.l.o.g that the set of vertices are the integers between 1 and n.
Definition 3.1 ((α, β)-spread distribution). Let µ be a distribution on D(n). We say that µ is (α, β)-spread if the following conditions are true for a graph G sampled from µ: has an incoming edge with probability at least α i.e.

For every
For example, one can generate an (k/n, 1)-spread distribution µ on D(n) in the following way: Fix arbitrary sets S 1 , . . . , S n ⊂ [n] of size k each. To sample a graph G from µ, pick a uniformly random element j ∈ [n] and let G be the directed graph containing the edges (j, i) for each i such that j ∈ S i . This satisfies the definition since for any fixed i ∈ [n], i has an incoming edge if j ∈ S i which happens with probability |S i |/n = k/n. And for any fixed i ′ , j ′ ∈ [n], the probability that (j ′ , i ′ ) is an edge is at most 1/n since this happens only when j ′ = j and j is chosen uniformly at random from [n]. Note that the sampled edges overall are highly correlated (they all have j as an endpoint). We will need following simple observation about (α, β)-spread distributions.
Proof. This follows from union bound and properties of (α, β)-spread distributions.
Given a distribution µ on graphs we would like to study the random process in which we, at each iteration, sample from µ and 'add' the edges we got to the graph obtained so far. For two graphs G and H on the same set of vertices, we denote by G ∪ H their set theoretic union (as a union of edges). Definition 3.3 (Graph process associated with µ). Let µ be a distribution on D(n). We define a sequence of random variables G µ t , t = 0, 1, 2, . . . as follows. G µ 0 is the empty graph on [n] vertices. At each step t ≥ 1 we sample a graph G according to µ (independently from all previous samples) and set G t = G t−1 ∪ G.
For a graph G ∈ D(n) and a vertex i ∈ [n] we denote by Rea(i) the set of vertices that are reachable from i (via walking on directed edges). By convention, a vertex is always reachable from itself. Similarly, for a set of vertices S ⊂ [n] we denote by Rea(S) = ∪ i∈S Rea(i) the set of vertices reachable from some vertex in S. We denote the set of strongly connected components of G by Γ(G). We denote by C(i) ∈ Γ(G) the strongly connected component of G containing i. We say that C ∈ Γ(G) is a source if C has no incoming edges from any vertex not in C.
Lemma 3. 4. Let µ be an (α, β)-spread distribution on D(n) and let G µ t be its associated graph process. Then, for all t ≥ 0, there is positive probability that the graph Γ(G µ t ) has at most n · (1 − α/4) t + 2β α sources.
Proof. If C ∈ Γ(G) is a source, we define the weight of C to be the number of vertices reachable from C (including vertices of C) that are not reachable from any other source of G. More formally, let Rea ′ (C) = {j ∈ Rea(C) | j ∈ Rea(C ′ ), for all sources C ′ ∈ Γ(G), C ′ = C}.
Then the weight of a source C ∈ Γ(C) is denoted by w(C) = |Rea ′ (C)| (we do not define weight for components that are not sources). Let us call a source C ∈ Γ(G t ) 'light' if its weight w(C) is at most k = αn 2β and 'heavy' otherwise. By the definition of weight, there could be at most n/k = 2β/α heavy sources.
We will argue that, in each step, as we move from G t to G t+1 , the number of light sources must decrease by a factor of (1 − α/4) with positive probability. For that purpose, suppose there are m t sources in G t and among them m ′ t are light. Fix some light source and pick a representative vertex i from it. Since i is contained in a light source, |Rea ′ (C(i))| ≤ αn 2β . When going to G t+1 = G t ∪ G, i gets an incoming edge from outside the set Rea ′ (C(i)) with probability at least α/2 by Lemma 3.2. If this happens then in G t+1 , this source will either stop being a source or merge with another source.
Picking a representative for each light source in G t , we see that the expected number of representatives i which get a new incoming edge from outside Rea ′ (C(i)) is at least (α/2)m ′ t . Hence, this quantity is obtained with positive probability. Now, if at least (α/2)m ′ t light sources 'merge' with another source or stop being a source in G t+1 then the total number of light sources must decrease by at least (α/4)m ′ t (the worst case being that (α/4)m ′ t disjoint pairs of light sources merge with each other). Hence, with positive probability we get that m ′ t+1 ≤ m ′ t ·(1−α/4). Therefore, since the samples in each step t are independent, there is also a positive probability that m ′ t ≤ n · (1 − α/4) t and m t ≤ m ′ t + 2β/α. This completes the proof. Proof. We will work with the (equivalent) set formulation: let F = {S 1 , . . . , S n } be a set system representing the spanoid S as in Lemma 2.7. We start by defining an (α, β)-spread distribution µ on D(n) as follows: To sample a graph G from µ we first pick ℓ ∈ [n] uniformly at random. Then we add a directed edge from j to i for every i, j such that {j, ℓ} ∈ M i . In this case we have S j ∩ S ℓ ⊆ S i and so, after restricting to S ℓ we have

Proof of upper bound from Theorem 1.9
Claim 3. 6. µ is a (2δ, 1)-spread distribution. Proof. For any fixed i ∈ [n], i will get an incoming edge if ℓ, which is randomly chosen from [n], belongs to M i . Since M i has at least δn edges, this will happen with probability at least 2δ. Now fix any i, j ∈ [n], (j, i) will be an edge iff ℓ is equal to the the vertex that matches j in the matching M i , this happens with probability at most 1/n. If j is not matched in M i , the probability is zero.
Consider the graph process G µ t and let S ℓ 1 , . . . , S ℓt be the sets chosen in the t iterations of sampling from µ. If i ∈ Rea(j) in the graph G µ t , this means that, after restricting to the intersection S = S ℓ 1 ∩ . . . ∩ S ℓt , the set S j is contained in S i (i.e., S j ∩ S ⊆ S i ∩ S). By Lemma 3.4, after t = O( 1 δ log 2 n) steps, the graph process G µ t will contain r = O(1/δ) sources. Pick a representative S a 1 , . . . , S ar from each of these sources. Then, the intersection of the t + r = O( 1 δ log 2 n) sets S ℓ 1 , . . . , S ℓt and S a 1 , . . . , S ar is contained in all n sets S 1 , . . . , S n . That is because, when restricted to the intersection of S ℓ 1 , . . . , S ℓt , each set S i contains one of the sets S a j , j ∈ [r].

Proof of upper bound from Theorem 1.10
Theorem 3.7 (Rank of q-query LCSs). Let S be a q-LCS with error-tolerance δ and q ≥ 3. Then Proof. Like the 2-query case, we work with the set representation F = {S 1 , . . . , S n } of S as in Lemma 2. 7. We follow the same strategy as in the proof of the 2-query case. The difference is that, in this case, we will need to pick many sets to restrict to in each step instead of just one. The first observation is that, if {j 1 , . . . , j q } ∈ M i then, restricted to the intersection S = S j 1 ∩ . . . ∩ S j q−1 we have S jq ⊂ S i . The second observation is that, if we choose a subset J ⊂ [n] of size roughly n q−2 q−1 then, in expectation, J will contain q − 1 elements in one of the q-subsets of M i for a constant fraction of the i's. Repeating this a logarithmic number of times and using Lemma 3.4, as in the proof of Theorem 3.5 will then complete the proof.
We start by defining an (α, β)-spread distribution µ on D(n). To sample a graph G from µ we first pick a random set J ⊂ [n] such that each j ∈ [n] is chosen to be in J independently with probability (δn) −1/(q−1) . By Markov's inequality we have that For each i ∈ [n] and each q-subset T ∈ M i we select q − 1 elements of T arbitrarily and refer to them as the distinguished (q − 1)-subset of T . We now argue that, for each i ∈ [n], there is relatively high probability that J will contain the distinguished (q − 1)-subset of at least one q-subset in M i . Proof. J will contain the distinguished q − 1 elements in a specific q-subset with probability (δn) −1 . Since the δn q-subsets in M i are disjoint, the probability that J will not contain any of the distinguished (q − 1)-subsets is at most (1 − (1/δn)) δn ≤ 1/2.
We are now ready to define the edges in the graph G sampled by µ. First we check if |J| ≥ If this is the case then µ outputs the empty graph (by Eq.2 this happens with probability at most 1/4). Otherwise for each i ∈ [n] we check to see if J contains the distinguished (q − 1)-subset from one of the q-subsets of M i . If there is at least one such q-subset, we pick one of them uniformly at random. Suppose the q-subset we chose is {j 1 , . . . , j q } and that the distinguished elements are the first q − 1. Then we add the directed edge j q → i to the graph G. By the above discussion, we know that, restricted to the intersection of all sets indexed by J the set S jq is contained in S i (hence the directed edge representing set inclusion). Claim 3.9. µ is (1/4, 1/δ)-spread.
Proof. By Claim 3.8, and since the probability that |J| is too large is at most 1/4 we see that any fixed i ∈ [n] will get an incoming edge in G with probability at least α = 1/4. For any fixed i, j ∈ [n], since the distribution of the special q-subset which is contributing an edge to i is uniform in M i (conditioned on J containing a q-subset from M i ), we can conclude that (j, i) ∈ E(G) with probability at most 1/(δn) = β/n. This proves the claim. Now, applying Lemma 3.4, we get that, after t = O(log 2 n) steps, the graph process G µ t will contain at most O(1/δ) sources with positive probability. Let J 1 , . . . , J t be the sets chosen in the different steps of the process and, w.l.o.g, remove any of them that were too big (i.e., when the graph sampled by µ was empty). Hence, all of the sets satisfy |J i | ≤ 4 · δ − 1 q−1 n q−2 q−1 . Now, let S be the intersection of all sets S j such that j belongs to at least one of the sets J i . Then, restricted to S, each of the sets S i contains one of the sources in the graph G µ t . Hence, if we add to our intersection a representative form each of the sources, we will get a set that is contained in all the sets S j . The total number of sets we end up intersecting is bounded by This completes the proof of the theorem.

Constructing q-LCSs with high rank
In this section we prove the lower bound part of Theorem 1.10 (the lower bound for the 2-query case follows from the Hadamard code construction). We will in fact generate this spanoid at random by picking, for each i ∈ [n], a random q-matching M i on [n] and, for each q-subset T ∈ M i add the rule T |= i. The resulting spanoid will thus have, by design, the structure of a q-LCS. The reason why this spanoid should have high rank (with high probability) relies on the following observation. Suppose A ⊂ [n] is a set that spans [n]. This means that there is a sequence of derivations T i |= i with each q-subset T i in the matching M i that eventually generates all of [n]. We can limit ourselves to the first C · |A| such derivations for some large C. These derivations generate a set A ′ of size (C + 1)|A| (including the original A and the C|A| newly derived elements). Now, the set A ′ must contain all of the q-subsets T i for C|A| values of i. However, the union of randomly chosen C|A| q-subsets will generally have size much larger than (C + 1)|A| (closer to q · C|A|). Proof. Let m = r · log 2 (n) = cn q−1 q−2 . If the rank of S is at most r then there exists a set A ⊂ [n] of size r that spans (using the rules obtained from the n random matchings M 1 , . . . , M n ) the entire base set [n]. We will upper bound the probability that such a set exists by bounding the smaller event given by the existence of a set of m rules that can be applied one after another starting with the original set A. That is, let E denote the event that there exists a set A of size r on which one can sequentially apply m rules of the form T j i |= j i with each T j i belonging to the matching M j i and for m different values j 1 , . . . , j m ∈ [n] arriving at the final setÂ = A ∪ {j 1 , . . . , j m }. If A spans [n] then clearly the event E must hold and so, it is enough to show that E happens with probability approaching zero.
We will present the event E as the union of (possibly overlapping) smaller events and then use the union bound, bounding the probability that each one occurs and multiplying by the number of bad events. Given a set A ⊂ [n] of size r, a tuple of m indicesĴ = {j 1 , j 2 , . . . , j m } and a family of q-subsetsT = {T j 1 , . . . , T jm } with T j i ∈ M j i denote by E(A,Ĵ ,T ) the event in which the set A spans the setÂ = A ∪Ĵ using the rules T j i |= j i applied in order with i going from 1 to m. For every fixing of A,Ĵ,T we can bound W.l.o.g suppose we sample the random matchings iteratively, picking a new q-subset at random among the available elements not covered by any previously chosen q-subsets in the current matching. Since the number of q-subsets in each matching is ⌊n/2q⌋ we have, at each step, at least n/2 available elements to chose from and so Taking the product over all m q-subsets inT we get To complete the proof we bound the number of tuples (A,Ĵ ,T ) as above by n r · n m · ⌊n/2q⌋ m ≤ n r · (en/m) m · n m ≤ 6n 2 m m , where the last inequality used the fact that r/m ≤ 1/ log 2 (n). Putting these bounds together we get that which is exponentially decreasing in m for the given choice of m = c · n (q−2)/(q−1) and for c a sufficiently small constant.
One could ask for a more explicit construction of an LCS with rank equal to (or even close to) that stated above. We are not able to give such a construction but can relate this problem to a longstanding open problem in explicit construction of expander graphs. A bipartite (balanced) expander of degree q, is a bipartite graph with n left vertices L and n right vertices R such that the degree of each vertex is q and such that sets A ⊂ L of size 'not too large' have many neighbors in R. More specifically, one typically asks that sets with |A| ≤ n/2 have at least (1 + ǫ)|A| right neighbors for some constant ǫ > 0. It is quite easy to see that a random graph of this form will be a good expander with high probability and, by now, there are also many explicit constructions [HLW06]. One can also consider unbalanced bipartite expanders in which |L| ≫ |R|. Take, for example, the setting in which |L| = n 2 , |R| = n and when the degree of every vertex in L is some constant q. A simple probabilistic argument shows that sufficiently small sets in L, namely sets of size |A| ≤ n αq with α q < 1 a constant depending on q and approaching 1 as q grows, have many neighbors in R (say, at least 2|A|). However, no explicit constructions of such graphs are known (for any constant q and any α q > 0). The property we needed in our random construction of LCSs can be thought of as an 'easier' variant of the expander construction problem. Given q-matchings M 1 , . . . , M n each of size δn consider the bipartite graph with L = [n] × [δn] and R = [n]. We identify each vertex (i, j) ∈ L with the j'th q-subset T ij of M i and connect it to the q neighbors in R given by that q-subset. For our proof to work we need the property that there is no small set containing many q-subsets from different matchings. This corresponds to asking for the above graph to be an expander for a restricted family of sets, namely to sets that have at most one vertex (i, j) for a given i (with each subgraph (i, * ) defining a matching).

Functional-rank vs. spanoid rank
In this section we analyze the five element spanoid Π 5 described in the introduction ( Figure 1) and show that its rank is strictly larger than its functional rank. Along the way we formulate the LP relaxation LP cover (S) which lower bounds the functional rank in general and another linear program LP entropy (S) which upper bounds the functional rank.
First we give the lower bound by constructing a consistent code over an alphabet of size 4 with 32 = 4 2.5 codewords.
Proof. We will construct a consistent code over the alphabet Σ = {0, 1} 2 . Each codeword will be indexed by an element of {0, 1} 5 . The codeword w(x) ∈ Σ 5 corresponding to (x 1 , x 2 , x 3 , x 4 , x 5 ) ∈ {0, 1} 5 will be ((x 5 , x 2 ), (x 1 , x 3 ), (x 2 , x 4 ), (x 3 , x 5 ), (x 4 , x 1 )) as shown in Figure 2. In other words, we place the bits x 1 , . . . , x 5 on the vertices of the cycle and then assign to each vertex the symbol of Σ = {0, 1} 2 comprised of the bits of its two neighbors on the cycle. It is now straight forward to verify that one can compute the coordinate w i , i ∈ [5] from the two coordinates spanning it in Π 5 . For example, the span rule {1, 2} |= 4 requires us to compute w 4 = (x 3 , x 5 ) from w 1 = (x 5 , x 2 ) and w 2 = (x 1 , x 3 ), which can be easily done (by symmetry, this is the situation in all the other rules).

An upper bound on functional rank via LP entropy
In this section, we will give a linear programming upper bound for f-rank(S) using properties of Shannon entropy. We will show that this upper bound matches the lower bound of 2.5 for f-rank(Π 5 ) shown in Claim 5.1, thus proving that f-rank(Π 5 ) = 2.5. We will begin by recollecting some properties of Shannon entropy.
Given a random variable X supported on some domain A, its (Shannon) entropy is defined as The Shannon entropy of a random variable measures its information content. The conditional entropy of X given an other random variable Y is defined as: H(X|Y ) = H(X, Y ) − H(Y ). And Figure 2: A consistent code for Π 5 ; each coordinate can be recovered from the coordinates of the opposite edge.
the conditional mutual information between X and Y given a third random variable Z is defined as: . Shannon proved that conditional entropy and conditional mutual information are always non-negative [CT91]. These are called basic information inequalities. Let X = (X 1 , X 2 , . . . , X n ) be a random variable made up of n coordinates. Define a function f : . This is because, In fact, the monotone increasing submodular property of f captures all the inequalities that can be obtained by using the basic information inequalities. But when n ≥ 4, the entropies H(X S ) satisfy some extra linear inequalities that are not captured by the basic information inequalities. These mysterious inequalities are called non-Shannon type inequalities and a few such inequalities are known [ZY98], but they are not well understood. The set where X ranges over all n jointly distributed random variables is called the entropic region for n random variables. Γ * n is a convex cone, but neither Γ * n nor its closure Γ * n are polyhedral for n ≥ 4 [Mat07] i.e. they are not defined by a finite number of linear inequalities. See [Yeu08] for more information about non-Shannon type information inequalities and the entropic region.
We are now ready to set up the linear program for upper bounding the functional rank of a spanoid. Let S be a spanoid on [n] and let C ⊂ Σ n be a code over some alphabet Σ which is consistent with the spanoid S i.e. whenever A → i in the spanoid, for every codeword c ∈ C, c i is determined by c| S . Let X = (X 1 , X 2 , . . . , X n ) be a random variable with uniform distribution over C. Then A → i in S implies that H(X A∪{i} ) = H(X A ). The dimension of the code C is So upper bounding the functional rank of S is equivalent to upper bounding H(X 1 , X 2 , . . . , X n ) where X is a random variable distributed over Σ n such that H( Clearly f (φ) = 0 and f ({i}) ≤ 1 for all i ∈ [n]. The basic information inequalities are equivalent to saying that f is a monotone increasing submodular function i.e. for every subsets A, B ⊂ [n], Thus the best upper bound we can derive on H(X) using the basic information inequalities is captured by the following linear program.
Note that LP entropy (S) is always at most rank(S). This is because, if A ⊂ [n] is such that span(A) = [n], then any feasible f in the LP (3), should satisfy f ([n]) ≤ |A|. The following claim formally states that LP entropy (S) upper bounds the functional rank of the spanoid S and lower bounds rank(S), the proof of which follows immediately from the above discussion. Note that the functional rank could be smaller than LP entropy (S) (though we do not know of an explicit example). This is because the basic information inequalities do not characterize the entropic region of more than 3 random variables. It might be possible to obtain better upper bounds on the functional rank by using non-Shannon type information inequalities. But in the case of the pentagon spanoid Π 5 defined in Figure 1, we will show that LP entropy (S) gives the tight upper bound.
Proof. We will show that value of the LP in Equation (3) is at most 2. 5. By the spanoid rules, we know that f ({i, i + 2, i + 3}) = f ({i + 2, i + 3}) for every i ∈ [5] where the addition is modulo 5. By rotational symmetry, we can obtain five inequalities of this form. Summing them, and observing that every adjacent pair of vertices is counted three times on the l.h.s, we get 10f ({1, 2, 3, 4, 5}).

A lower bound on functional rank via LP cover
In this section, we will prove lower bounds on functional rank by constructing consistent codes.
And the best code one can construct in this way is captured by a very natural linear programming relaxation of the spanoid rank called LP cover . The code constructed in Claim 5.1 for the pentagon spanoid Π 5 can be viewed as an instance of a more general scheme based on a union set-representation of a spanoid. x i Since the LP is a relaxation, rank(S) ≥ LP cover (S).
We can round an LP solution to get an integral hitting set by losing a factor of O(VC-dim(O * ) · log(LP cover (S))) where VC-dim(O * ) is the VC-dimension of O * [ERS05,BG95]. Therefore, Note that VC-dim(O * ) ≤ log |O * | always. By LP duality, we can write a dual LP for LP cover (S) with a dual variable λ S for every S ∈ O * , The following lemma shows that LP cover (S) is the largest dimension of a consistent code one can obtain using union set-representation of a spanoid as shown in Construction 5. 4. Since the construction of a consistent code for the pentagon spanoid Π 5 in Claim 5.1 is obtained by Construction 5.4 and since we know that f-rank(Π 5 ) = 2.5, it turns out that LP cover (Π 5 ) = 2.5.
Lemma 5. 5. Let C be a code consistent with S obtained by using Construction 5.4, then dim(C) ≤ LP cover (S). Moreover, there exists a code C obtained by using Construction 5.4 giving equality and so f-rank(S) ≥ LP cover (S).
Proof. We will first show that one can get a code whose rank is at least LP cover (S). Let  . . , F m } then a codeword will be indexed by a tuple of bits x = (x 1 , . . . , x m ) ∈ {0, 1} m . To construct the corresponding codeword w(x) ∈ Σ n we need to specify the value of w(x) i ∈ Σ = {0, 1} N for each i = 1, 2, . . . , n. We set that value to be w(x) i = (x j | i ∈ F j ) (ordered in increasing order of j). That is, we associate a bit with each open set in H, and assign the value of a coordinate i to be the list of bits for all sets in H containing i. Note that the alphabet size is bounded by N because |{j : i ∈ F j }| = S∋i λ S N ≤ N .
We now need to show that, if T |= i is a rule of S then one can recover w(x) i from (w(x) t ) t∈T . Suppose F j is an open set containing i so that x j appears in the symbol w(x) i . Then, there must be an element t ∈ T so that F j contains t (otherwise, F j cannot contain i). Thus, the value x j can be computed from w(x) t . Since this holds for any x j appearing in w(x) i we are done.
We will now show that the dimension of a code obtained using Construction 5.4 cannot be better than LP cover (S). Let S 1 , . . . , S n be the subsets of size at most ℓ from some universe U obtained from Construction 5.4 such that whenever T |= i in S, S i ⊂ ∪ j∈T S j . Such a set system gives a code with dimension |U |/ℓ. We will show that such a code also gives a feasible solution to the dual LP (Eqn. 5). For u ∈ U , let F u = {i : u ∈ S i }. The complement of F u is closed since the union of sets which don't have u cannot contain any set which has u, therefore F u is an open set in S. Let F * u be a minimal open set contained in F u . For a minimal open set F ∈ O * , set Thus this assignment is a feasible solution to the dual LP (Eqn. 5) with objective value This implies that LP cover (S) ≥ |U |/ℓ.
The code construction based on sets achieving dimension equal to LP cover (S) constructed in the above above lemma needed very large alphabet. The following lemma shows that by using random sampling, one can get very small alphabet and still achieve dimension close to LP cover (S).
Lemma 5. 6. Let S be a spanoid on [n], then there exists a code C ⊂ Σ n consistent with S, obtained as in Construction 5.4 of dimension r over an alphabet Σ such that log |Σ| log n/ log log n and r log log n log n · LP cover (S).
Proof. Let us assume that LP cover (S) ≥ log n/ log log n, since otherwise the bound is trivial. Form a random subset H ⊂ O * by sampling each set S ∈ O * with probability λ S where λ S is the optimal solution to the dual LP (5). Let Z S be the random variable that S is included in H.
Let ∆(H) be the maximum number of sets in H that an element of S belongs to. We want to show that ∆(H) is small with good probability. Fix some i ∈ [n]. The number of subsets in H which contain i is S∋i Z S .
By union bound, if t = e log n/ log log n. Therefore there exists an H ⊂ O * such that |H| ≥ LP cover (S)/2 and ∆(H) ≤ e log n/ log log n. By imitating the proof of Lemma 5.5 using this H, we get the required result.
Gaps between LP cover and functional rank: Any code that is obtained by using Construction 5.4 from a q-LCS with error-tolerance δ will have dimension at most O(q/δ). Indeed, suppose S 1 , . . . , S n ⊂ U are the sets used in the construction. Then each set S i is contained in (δ/q)n disjoint q-subsets of sets in {S j : j = i} such that S i ⊂ ∪ j∈T S j for each q-subset T . Therefore each element of U occurs in (δ/q) fraction of sets in S 1 , . . . , S n . Therefore a typical set in S 1 , . . . , S n contains (δ/q) fraction of elements from U . Therefore the dimension of such a code can be at most q/δ. This also implies that the LP cover of a q-LCS is O(q/δ). Since the rank of a q-LCS (with some constant δ) can beΩ(n 1− 1 q−1 ) (Theorem 1.10), this shows that LP cover can be much smaller than the rank of a spanoid. In fact LP cover can be much smaller than the functional rank (f-rank) of a spanoid. exp(Õ( √ log n))-query LCCs (for some constant δ) of length n with linear dimension are known to exist [KMRS17]. This implies that functional rank of the corresponding spanoid is Ω(n) whereas LP cover is at most exp(Õ( √ log n)).

Products of Spanoids
We have shown that functional rank can be strictly smaller than the rank of a spanoid (Claim 5.3), but it was only a constant factor gap. Can we construct spanoids whose rank(S) and f-rank(S) have a polynomial gap? One way to achieve this is by constructing product operations on spanoids which amplify the gap between rank and f-rank. Suppose that given two spanoids S 1 and S 2 on [n 1 ] and [n 2 ] respectively, we can construct a product spanoid S 1 × S 2 on [n 1 ] × [n 2 ] such that, rank(S 1 × S 2 ) ≥ rank(S 1 ) · rank(S 2 ) and f-rank(S 1 × S 2 ) ≤ f-rank(S 1 ) · f-rank(S 2 ). Then by starting with the pentagon spanoid and taking the above product several times, one can get a spanoid on n elements with n Ω(1) gap between rank and f-rank. With this motivation, we study a few ways natural ways to define products of spanoids and study how various notions of rank we defined behave under these products.The following lemma will be useful to compare the rank measures of two spanoids. It shows that a spanoid with more open sets has higher rank, f-rank, LP cover and LP entropy . Intuitively, this is because in the spanoid with more open (closed) sets, you can make fewer inferences.

Proof.
Since rank(S) is the smallest hitting set for O S and LP cover (S) is the smallest fractional hitting set for O S and O S ⊂ O ′ S ′ , the first two inequalities easily follow. To show the inequality for functional ranks, we will show that if A |= S ′ i then A |= S i i.e. S has more inference rules than S ′ . Thus any code consistent with S is also consistent with S ′ . This also implies the inequality for LP entropy because, the corresponding maximization LP for S has more constraints than for S ′ and so the maximum is smaller.
The closed sets C S and C S ′ of S and S ′ also satisfy C S ⊂ C S ′ . A |= S i iff i ∈ span S (A). By  We will now show that under this product, LP cover is multiplicative.
The minimal open sets in S are cartesian products of minimal open sets in S 1 and S 2 i.e.
LP cover (S) = min n i,j=1 Let x, y be the optimal solutions to the LPs (of the form (4)) corresponding to LP cover (S 1 ) and LP cover (S 2 ) respectively. Then z = x ⊗ y (i.e. z ij = x i y j ) is a feasible solution to the above LP for LP cover (S). Therefore LP cover (S) ≤ LP cover (S 1 ) · LP cover (S 2 ).
We can also write the dual LP for LP cover (S).
Let α, β be the optimal solutions to the dual LPs (of the form (5)) corresponding to LP cover (S 1 ) and LP cover (S 2 ) respectively. Then λ = α ⊗ β (i.e. λ AB = α A β B ) is a feasible solution to the above dual LP for LP cover (S). Therefore LP cover (S) ≥ LP cover (S 1 ) · LP cover (S 2 ).

Product S 1 ⊗ S 2
We will now define a different product operation inspired by the following tensor product operation on codes.
Definition 6. 4. Given two codes C 1 ⊂ Σ n 1 and C 2 ⊂ Σ n 2 , the tensor code C 1 ⊗ C 2 ⊂ Σ n 1 ×n 2 is defined as the set of all n 1 × n 2 matrices over Σ where each column is a codeword in C 1 and each row is a codeword in C 2 .
If C 1 and C 2 are linear codes over some field F i.e. they are linear subspaces of F n 1 and F n 2 respectively, then the tensor code C 1 ⊗ C 2 ⊂ F n 1 ×n 2 is exactly the tensor product of the subspaces C 1 ⊗ C 2 .
We will now define a product operation on spanoids which mimics the above operation on codes.
Definition 6. 5. Let S 1 and S 2 be spanoids on X 1 , X 2 respectively. The product S 1 ⊗S 2 is a spanoid on X 1 × X 2 generated by the following inference rules: 1. For A ⊂ X 1 , i ∈ X 1 , if A |= i in S 1 then for every j ∈ X 2 , A × {j} |= (i, j).
The following claim follows easily from the above definitions.
How does S 1 ⊗ S 2 compare with S 1 ⊙ S 2 ? The following claim shows that S 1 ⊗ S 2 has more open sets (or closed sets) than S 1 ⊙ S 2 . This shows that all the rank measures are smaller for S 1 ⊙ S 2 than for S 1 ⊗ S 2 .
Proof. It is enough to show that if A ∈ O S 1 and B ∈ O S 2 then A × B ∈ O S 1 ⊗S 2 . This is equivalent to showing (A × B c ) ∪ (A c × B) being closed in S 1 ⊗ S 2 , which is easy to see given the inference rules.

Product S 1 ⋉ S 2
In this subsection, we define a product operation called the semi-direct product, denoted by S 1 ⋉S 2 . Under this product we will show that rank is multiplicative and f-rank is sub-multiplicative. Thus by taking repeated semi-direct product of the pentagon spanoid Π 5 with itself, we create a spanoid with polynomial gap between its rank and f-rank which proves Theorem 1.13. Additionally, we can also show that LP entropy is sub-multiplicative under this product which gives a spanoid with polynomial gap between LP entropy and rank. We will begin with formal definition of semi-direct product.
Definition 6.8 (Semi-direct product of spanoids). Let S 1 , S 2 be two spanoids on X 1 , X 2 resp. Define S 1 ⋉ S 2 to be the spanoid on X 1 × X 2 generated by the following rules.
Now we return to the proof of |T | ≥ rank(S 1 )rank(S 2 ). Let l be the integer in the above claim. Because steps k ≥ l + 1 are all of type (1), we have |{i : {i} × X 2 ⊂ T l }| ≥ rank(S 1 ).
We will now show that f-rank is sub-multiplicative under semi-direct product.
Proof. Let C ∈ Σ X 1 ×X 2 be a code consistent with S 1 ⋉ S 2 . For i ∈ X 1 , define code C i 2 ⊂ Σ X 2 as C i 2 = {c {i}×X 2 : c ∈ C}. Because C i 2 is consistent with S 2 , we have log |C i 2 |/log |Σ| ≤ f-rank(S 2 ). Let N = max i∈X 1 |C i 2 | and for each i ∈ X 1 , choose an injection φ i : C i 2 ֒→ [N ]. Clearly, we have log N log |Σ| ≤ f-rank(S 2 ).
We can now prove Theorem 1.13.
We also show that LP entropy is sub-multiplicative under semi-direct product.
Thus by repeatedly taking semi-direct product of the pentagon spanoid Π 5 with itself, we can obtain a polynomial gap between LP entropy and rank, which is a stronger statement than Theorem 1.13 because f-rank(S) ≤ LP entropy (S). Corollary 6.13. There exists a spanoid S on n elements with rank(S) ≥ n c LP entropy (S) where c = log 5 3 − log 5 2.5 ≥ 0.113.
Because S 1 ⋉ S 2 has fewer inferences than S 1 ⊗ S 2 , every closed set in S 1 ⊗ S 2 is also closed in S 1 ⋉ S 2 i.e. O S 1 ⊗S 2 ⊂ O S 1 ⋉S 2 . Therefore we have the following relationship between the different products we constructed, By Lemma 6.1, all the rank measures should follow the same order. Therefore, we have the following corollary.
Thus LP cover is super-multiplicative under semi-direct product, while rank is multiplicative and the other two are sub-multiplicative.
Example 6.15. In contrast to the semi-direct product, rank is not multiplicative under S 1 ⊙ S 2 and S 1 ⊗ S 2 .