Perfect Simulation of Las Vegas Algorithms via Local Computation
Abstract
The notion of Las Vegas algorithms was introduced by Babai (1979) and can be defined in two ways:
-
In Babai’s original definition, a randomized algorithm is called Las Vegas if it has a finitely bounded running time and certifiable random failure.
-
Another definition widely accepted today is that Las Vegas algorithms refer to zero-error randomized algorithms with random running times.
The equivalence between the two definitions is straightforward. Specifically, for randomized algorithms with certifiable failures, repeatedly running the algorithm until no failure is encountered allows for faithful simulation of the correct output when it executes successfully.
We show that a similar perfect simulation can also be achieved in distributed local computation. Specifically, in the model, with a polylogarithmic overhead in time complexity, any Las Vegas algorithm with finitely bounded running time and locally certifiable failures can be converted to a zero error Las Vegas algorithm. This transformed algorithm faithfully reproduces the correct output of the original algorithm in successful executions. This is achieved by a reduction to a distributed sampling problem under the Lovász Local Lemma (LLL), where the objective is to sample from the joint distribution of random variables avoiding all bad events. We then design the first efficient algorithm to solve this sampling problem in the model.
Keywords and phrases:
Las Vegas algorithms, perfect simulation, Lovász Local Lemma, samplingCopyright and License:
2012 ACM Subject Classification:
Theory of computation Distributed algorithmsEditor:
Shubhangi SarafSeries and Publisher:
Leibniz International Proceedings in Informatics, Schloss Dagstuhl – Leibniz-Zentrum für Informatik
1 Introduction
The Las Vegas algorithm, introduced by Babai [2], stands as a cornerstone in the theory of computing, defining the ZPP class for decision problems efficiently solvable by Las Vegas algorithms. Beyond decision problems, Las Vegas algorithms are pivotal in tackling optimization [24, 6], searching [28, 31], or sampling [34, 16] problems.
Las Vegas algorithms can be defined in two distinct yet related ways. In Babai’s original work [2], Las Vegas algorithms are defined as randomized algorithms whose failures are certifiable:
-
A Las Vegas algorithm produces the correct output or reports failure within a finite bounded time.
Another definition of the Las Vegas algorithm, widely accepted today and utilized in various contexts, e.g. in [27, 32, 29], defines Las Vegas algorithms as zero-error randomized algorithms:
-
A Las Vegas algorithm may exhibit a random running time but always produces the correct output.
The equivalence between these two definitions is evident. Through truncation, we can transform a zero-error Las Vegas algorithm with random running time into a Las Vegas algorithm with bounded running time and certifiable failure. Conversely, restarting the algorithm upon failure allows us to convert a Las Vegas algorithm with certifiable failure into a zero-error Las Vegas algorithm. This strategy, retrying with independent random choice until success, defines a rejection sampling procedure that faithfully simulates the random output of the Las Vegas algorithms, provided that they return successfully without failure.
This strategy for perfectly simulating the Las Vegas algorithm relies on a global coordination machinery: Every part of the algorithm must be notified if a failure occurs anywhere. However, our focus is on exploring how this could be accomplished through local computations, without the need for global coordination.
The LOCAL model.
Local computations are formally characterized by the model [25, 33]. An instance consists of a network , which is an undirected graph, and a vector of local inputs. Each node receives and as input and can access private random bits. Communications are synchronized and proceed in rounds. In each round, each node may perform arbitrary local computation based on all information collected so far and send messages of arbitrary sizes to all its neighbors. This gives a algorithm. The time complexity is measured by the number of rounds that the algorithm spends until all nodes are terminated. A algorithm is said to be a -round algorithm on a class of instances, if it always terminates within rounds on every instance from that class, where represents the number of nodes of the instance.
The (Babai’s) Las Vegas algorithm can be defined in the mode with locally certifiable failures. A -round algorithm is called Las Vegas if each node returns a pair , where stands for the local output at , and indicates whether the algorithm failed locally at . The algorithm successfully returns if none of the nodes fails. Furthermore, it is guaranteed that . This notion of Las Vegas algorithm was formulated in [14].
In this paper, we try to answer the following fundamental question:
Can we faithfully simulate the correct output avoiding all local failures via local computation?
Specifically, we wonder whether a fixed-round Las Vegas algorithm with locally certifiable failures, can be converted into a zero-error Las Vegas algorithm that produces the correct output conditioned on , where the distribution of the correct output is faithfully preserved. This can be viewed as the analog of rejection sampling, carried out without any form of global coordination.
In this paper, for the first time, we answer this question affirmatively. We prove the following result for the perfect simulation of Las Vegas algorithms via local computation.
Theorem 1 (main theorem, informal).
Any -round Las Vegas algorithm can be converted to a zero-error Las Vegas algorithm, which terminates within rounds with probability , and returns the output of the -round Las Vegas algorithm conditioned on no failure.
In the above theorem, the output of the zero-error Las Vegas algorithm is identically distributed as the output of the -round Las Vegas algorithm conditioned on that none of the nodes fails, i.e. the zero-error algorithm perfectly simulates a successful running of the algorithm that may locally fail.
To see how nontrivial this is, consider a weakened task: to generate an assignment of random bits so that under this random choice the algorithm terminates without failure. One may think of this as “solving” for the random bits under which the algorithm successfully returns, which is weaker than our goal, where the generated random bits are further required to follow the correct distribution. However, for local computation, just solving the feasible random bits without changing their distribution is already highly nontrivial.
In a seminal work [14], Ghaffari, Harris, and Kuhn gave a systematic approach to solve the good random bits under which a Las Vegas algorithm successfully returns. Their derandomization based approach preserves the support of the distribution, hence was more suitable for the distributed graph problems for constructing feasible solutions on graphs. Specifically, -round reductions were established between the two types of Las Vegas algorithms for such problems.
In contrast, the perfect simulation guaranteed in Theorem 1 preserves the distribution, therefore, the result can apply to problems beyond constructing feasible solutions, for example, the sampling problems.
Consider the Gibbs distributions defined on the network . Each node corresponds to a variable with finite domain . Let be a class of constraints, where each is a nonnegative-valued function defined on the variables in . This defines a Gibbs distribution over all assignments by:
Such Gibbs distribution is said to be local, if: (1) for any , the diameter of in graph is bounded by a constant; and (2) for any partial assignment specified on , if is locally feasible, i.e. if for all with , then is (globally) feasible, which means that can be extended to a feasible full assignment such that and .111In [13], this property of local feasibility implying global feasibility in Gibbs distribution was called “locally admissible”.
A Gibbs distribution is said to have strong spatial mixing with exponential decay if the discrepancy (measured in the total variation distance) between the marginal distributions at any given the respective feasible boundary conditions on that differ over an arbitrary , satisfies:
The strong spatial mixing is a key property for sampling algorithms. Its implication for efficient sampling from general Gibbs distributions is a major open problem. In [13, Corollary 5.3], a -round Las Vegas algorithm with bounded local failures was given, for perfect sampling from local Gibbs distributions that have strong spatial mixing with exponential decay. By Theorem 1, this immediately implies the following result for the perfect simulation of Gibbs distributions via local computation.
Corollary 2.
For any class of local Gibbs distributions that have strong spatial mixing with exponential decay, there is a algorithm for perfect sampling from the Gibbs distribution, which terminates within rounds with probability .
Remark 3.
The computational tractability of sampling from general Gibbs distributions with strong spatial mixing remains unresolved, even in the sequential and centralized setting. Corollary 2 certainly does not resolve the computational complexity of sampling. This is because the model does not impose any bounds on the computational costs, and generic reductions such as the one stated in Theorem 1 must apply to all algorithms. Such a lack of limitation on computational power was indeed common in general (black-box) reductions in the model and may be necessary. For instance, the derandomization of general algorithms in [14] also relied on a local exhaustive search of random bits. Here, our objective is even more ambitious: to preserve the distribution of the random bits of a successful run.
On a positive note, if it is known that the correct output has strong spatial mixing, as in the case of Corollary 2, then the perfect simulation in Theorem 1 can be simplified (using Algorithm 3 instead of Algorithm 4), avoiding exhaustive enumerations. Consequently, the computational cost of the algorithm in Corollary 2 is dominated by approximately sampling from the marginal distributions and estimating the marginal probabilities within an inverse-polynomial accuracy. Indeed, the non-trivial computation costs in both the sampling algorithm in [13] and our perfect simulation algorithm when there is a strong spatial mixing, are dominated by these tasks which are computationally equivalent to sampling.
In summary, the focus of Corollary 2 is the locality of Gibbs sampling. Our proof of it may also imply: locality of computation imposes no additional significant barrier to the task of Gibbs sampling with strong spatial mixing, provided the sampling problem is computationally tractable.
1.1 A sampling Lovász Local Lemma in the LOCAL model
The perfect simulation of Las Vegas algorithms stated in Theorem 1 is achieved by resolving a more general problem, namely, generating a random sample avoiding all bad events. This problem is formulated as a natural sampling problem in the variable-framework of the Lovász Local Lemma.
An instance for the variable-framework Lovász Local Lemma (LLL) is given by , where is a set of mutually independent random variables, such that each follows a distribution over a finite domain ; and is a set of bad events, such that for each , the occurrence of is determined by the evaluation of , where denotes the subset of variables on which is defined. The LLL instance defines a dependency graph , such that each vertex represents a bad event and each iff .
An LLL instance is said to be -satisfiable, if the probability avoiding all bad events is bounded as:
| (1) |
The Lovász Local Lemma [8] states a sufficient condition on the dependency graph for to be positive.
A satisfiable LLL instance gives rise to a natural probability distribution over satisfying assignments. Let denote the distribution of the random vector conditioning on that none of the bad events occurs. Formally, denote by the space of all satisfying assignments, and the product measure. Here, the symbol represents the Cartesian product over sets. Then
| (2) |
This distribution of random satisfying assignment was referred to as the LLL distribution in [17, 18]. It is a Gibbs distribution defined by hard constraints.
The following defines a computational problem to generate a sample according to the distribution .
Sampling Satisfying Solution of Lovász Local Lemma
| Input: | a -satisfiable LLL instance with dependency graph ; |
|---|---|
| Output: | a random satisfying assignment distributed as . |
When the problem is solved in the model, the input is presented to the algorithm as follows:
-
1.
The network of the model is just the dependency graph .
-
2.
Each node receives as input the values of and , along with the definition of the bad event , and the distributions of the random variables on which is defined so that it can locally draw independent evaluations of the random variables or check the occurrence of on such evaluation.
Our main technical result is an efficient algorithm for sampling satisfying solution according to the LLL distribution, for any LLL instance that is not prohibitively hard to satisfy (i.e., is not too small). We call this result a “ sampling lemma” since it uses the local lemma framework to give a sampling algorithm.
Theorem 4 ( sampling lemma).
There is a randomized algorithm such that for any LLL instance with bad events, if is -satisfiable, then the algorithm returns a random satisfying assignment drawn from , within rounds in expectation, and within rounds with probability at least for any .
Remark 5.
The sampling variant of LLL has recently drawn considerable attention [35, 20, 21, 22, 11, 30], which aims to generate a satisfying solution distributed as . This is harder than just finding a satisfying solution. It requires a stronger LLL condition (with polynomials of dependency degree) to ensure that the sampling is tractable in polynomial time [3, 16, 18].
Alternatively, in our setting, we do not impose any LLL-like conditions, but instead only assume that the LLL instance is -satisfiable for some suitably large (e.g. or ). This is because the LLL instance here arises from a Las Vegas algorithm which succeeds with probability . In centralized model, such assumption may trivialize the problem because a sample can be drawn using trials of rejection sampling. However, the sampling problem remains highly nontrivial for local computation.
1.2 Perfect simulation via LOCAL sampling lemma
Recall that for algorithms, an instance consists of a network and a vector specifying the local input to each node . Let be a class of instances. A -round Las Vegas algorithm with success probability on instance class , is a randomized algorithm such that on every instance , where is a network with nodes, at every node the algorithm terminates within rounds and outputs a pair where indicates whether the algorithm failed locally at , and the probability that the algorithm succeeds is
Denote by the output of the Las Vegas algorithm on instance with network . The following theorem is a formal restatement of Theorem 1, which gives a zero-error Las Vegas algorithm that perfectly simulates the good output when there is no failure everywhere in the network.
Theorem 6 (formal restatement of Theorem 1).
Let and be two functions. Let be a -round Las Vegas algorithm with success probability on instance class . There is a algorithm such that on every instance of nodes, for any ,
-
terminates in rounds with probability at least ;
-
upon termination, returns an output that is identically distributed as , which stands for the output of on the same instance conditioned on that none of the nodes fails.
Proof.
Let be the instance of the algorithm, where is a network with nodes and the vector specifies the local inputs. For each , let denote the local random bits at node used by algorithm .
Since is a -round Las Vegas algorithm, at any , the algorithm deterministically maps the inputs and the random bits within the -ball , to the local output , where indicates the failure at . This defines a bad event for every , on the random variables for , i.e. , by
Together, this defines an LLL instance , which is -satisfiable because the probability that has no failure everywhere is at least .
We simulate the sampling algorithm in Theorem 4 (which we call the LLL sampler) on this LLL instance . Rather than executing it on the dependency graph as in Theorem 4, here we simulate the LLL sampler on the network , where each node holds an independent random variable and a bad event . Note that any 1-round communication in the dependency graph can be simulated by -round communications in this network . Also note that at each , the values of and can be computed locally by knowing and enumerating all network instances with nodes. The LLL sampler can thus be simulated with -multiplicative overhead. In the end it outputs an , which is identically distributed as , i.e. the random bits used in the algorithm conditioned on no failure. The final output is computed by simulating within locality deterministically using as random bits.
The LLL sampler in Theorem 4 is a Las Vegas algorithm with random terminations. Each node can continue updating using the current random bits it has collected within its -local neighborhood . Once the LLL sampler for generating terminates at all nodes, the updating of will stabilize within additional rounds. And this final is identically distributed as . This gives us the zero-error Las Vegas algorithm as claimed in Theorem 6.
1.3 Related work
The perfect simulation of Las Vegas algorithms is a fundamental problem. In the celebrated work of Luby, Sinclaire, and Zuckerman [27], an optimal strategy was given for speeding up Las Vegas algorithms. Their approach was based on stochastic resetting, which requires global coordination and works for the Las Vegas algorithms with deterministic outputs, or the interruptible random outputs.
The distribution of satisfying solutions of the Lovász local lemma (LLL) has drawn much attention, e.g. in [16, 18]. Its perfect simulation was studied in [16, 20, 19, 9], where several key approaches for perfect sampling were applied, including partial rejection sampling (PRS) [16], “lazy depth-first” method of Anand and Jerrum (a.k.a. the AJ algorithm) [1], coupling from the past (CFTP) [34], and coupling towards the past (CTTP) [9].
The connection between LLL and distributed computing is well-established. The foundational work of Chung, Pettie, and Su [5] pioneered the application of distributed LLL for solving Locally Checkable Labeling (LCL) problems in the model. Then, Chang and Pettie [4] proposed that any Las Vegas randomized algorithm can be represented as an LLL instance. Recently, Davies-Peck [7] present a novel analysis that leads to improved constructive LLL algorithms with variable running times for the LOCAL model, further enriching this domain.
In the model, Ghaffari, Harris and Kuhn [14] showed that for distributed graph problems, where the goal is to construct feasible graph configurations, the fixed-round Las Vegas algorithms and the zero-error Las Vegas algorithms are equivalent to polylogarithmic rounds. Their approach was based on a derandomization by conditional expectations, and hence was especially suitable for the tasks where the support of the output distribution, instead of the output distribution itself, is concerned, such as the searching problems for constructing feasible solutions.
The algorithms and Gibbs distributions are intrinsically related. For example, the distributions of the random bits on which a fixed-round Las Vegas algorithm successfully returns are Gibbs distributions, where the certifiers of local failures are the local constraints defining the Gibbs distribution. In [13], Feng and Yin gave a sampler with local failures for the Gibbs distributions with strong spatial mixing by parallelizing the JVV sampler [23] using the network decomposition [26].
1.4 Technique overview
We provide an overview of the main sampling algorithm stated in Theorem 4. The design and analysis of this algorithm will be detailed in subsequent sections.
Recall that the input instance for the algorithm in Theorem 4 is an LLL instance with bad events. We assume that is -satisfiable.
Initially, a random assignment is generated according to the product distribution , which can be achieved by each node independently drawing its own variable . If none of the bad events in occurs on this , then is a correct sample that follows the LLL distribution . However, in general, some bad events in may occur. In this case, the crux of the sampling algorithm lies in how to locally resample the independently generated to make it follow the correct joint distribution .
Locally resample via the Bayes filter.
We first propose a resampling procedure under the idealized assumption that there is only one node with its bad event occurring on . This algorithm serves as the principal subroutine in our main sampling algorithm. To locally resample the random assignment around node , a natural attempt would be to resample locally according to the marginal distribution induced by on conditioned on the outer boundary . However, this naïve resampling would introduce a bias to the sample, because does not follow the marginal distribution induced by on , but rather follows the conditional marginal distribution on given .
To rectify this, the idea of Bayes filter was introduced in [10]. In this approach, a (locally checkable) filter is constructed according to Bayes’ law to cancel the bias introduced by the naïve resampling. This new resampling procedure guarantees the production of a correct that follows the desired distribution upon termination. Moreover, the procedure terminates in finite steps if the distribution exhibits a strong enough decay of correlation property. The only problem is that such decay of correlation may not necessarily hold for general LLL instances.
De-correlate distant variables by LLL augmentation.
The decay of correlation is a key property that is widely assumed by sampling algorithms. However, in our main applications, namely simulating Las Vegas algorithms via local computation, such decay of correlation may not always hold. Instead, we assume that the LLL instance is sufficiently satisfiable (which corresponds to a Las Vegas algorithm that succeeds with a sufficiently large probability).
With this new assumption, we show that it is possible to establish suitably strong correlation decay between far-apart regions of variables by introducing a new bad event on the boundary between these two regions. Intuitively, we shows that strong correlation between distant regions can only persist due to some specific low-probability bad blocking assignments on the variables between them. We then de-correlate the regions by augmenting the LLL instance with a new locally constructed bad event that explicitly prohibits these bad blocking assignments. Crucially, because these bad blocking assignments must have small marginal probabilities, the new bad event itself is rare, preserving the overall satisfiability of the instance. A precise example illustrating this mechanism is provided in the full version of the paper for further insight. Furthermore, we show how to utilize this LLL augmentation in a carefully designed resampling procedure to guarantee both the correctness and efficiency of the resampling.
Resample in general case using the SLOCAL paradigm.
Finally, to handle the general case where multiple bad events may occur on , we first cluster the bad events into small enough balls far apart using network decomposition. We then apply the resampling procedure to fix each ball one by one. This final step of sequentially resampling on balls is presented in the sequential local () paradigm. We introduce the notion of algorithm as an extension of the model, where stands for the term sequential Las Vegas. Finally, we show that this algorithm can be simulated in the model with bounded time complexity.
1.5 Organization of the paper
We begin with preliminaries in Section 2. Then, in Section 3, we illustrate the core ideas behind our main sampling algorithm (stated in Theorem 4) by analyzing a special case where only a single bad event occurs initially. The algorithm for the general case involving multiple bad events initially, along with the complete proofs, is deferred to the full version of this paper. We conclude with a discussion of our work in Section 4.
2 Preliminary
2.1 Graph and LLL notations
Let be an undirected graph. The following notations are used throughout.
-
Neighborhoods: and inclusive neighborhood .
-
Distances: represents the shortest path distance between and in , and . The diameter of in is .
-
Balls: and for .
-
Shells/Spheres: and for .
Let be an LLL instance with dependency graph . The following defines a notion of rings of variables enclosing a nonempty subset of bad events, where .
-
Rings: and in particular, . Furthermore, we define
(3)
We also define the rings of bad events intersected or contained by a ring of variables:
| (4) |
In all the above notations, we omit the graph or the LLL instance if they are clear in the context.
Inspired by the induced subgraph, we define the sub-instance of induced by a subset of bad events:
where is the LLL sub-instance induced by the bad events .
2.2 Marginal distribution
Let be a LLL instance, where each random variable follows the distribution over domain . For , define and , and write and .
For nonempty and , define
| (5) |
For disjoint , and , denote by the direct concatenation of and , that is, satisfying for and for .
The following defines a notion of marginal distribution in the LLL instance.
Definition 7 (marginal distribution).
For , a is said to be a feasible boundary condition if , where is defined in (5). Given and feasible boundary condition , for any nonempty , the marginal distribution on induced by conditioned on , denoted by , is defined as:
2.3 Decay of correlation in LLL
We consider the following notion of correlation decay in the LLL instance.
Definition 8 (-correlated sets).
A pair of disjoint with , is said to be -correlated, if one of is empty, or for any and ,
To see that this indeed defines a decay of correlation, note that it is equivalent to the following property: For drawn according to the product distribution that avoids all bad events satisfying ,
Recall that we want to bound the correlation between and in a random distributed as . Here, Definition 8 is slightly different by ignoring the bad events defined on the variables within .
2.4 Las Vegas SLOCAL algorithm
Our main sampling algorithm is described in a sequential local () paradigm. The model introduced by Ghaffari, Kuhn, and Maus [15] captures local computations where the inherent parallelism of distributed systems is intentionally suppressed, allowing nodes to be processed sequentially rather than concurrently. We extend this notion to the algorithms with random locality of computation.
SLOCAL-LV algorithms.
Here, stands for the term sequential Las Vegas. An -scan algorithm runs on a network with a subset of active nodes. Each node maintains a local memory state , initially storing ’s local input, random bits, its ID, and neighbors’ IDs. An arbitrary total order is assumed over , such that the relative order between any can be deduced from the contents of and .
The algorithm operates in scans. Within each scan, the active nodes in are processed one after another in the ordering. Upon each node being processed, for , the node tries to grow an -ball and update the memory states for all based on the information observed so far by , until some stopping condition has been met by the information within the current ball . Finally, each outputs a value based on its memory state .
Compared to the standard (Monte Carlo) algorithm, whose locality is upper bounded by a fixed value, in algorithm, the local neighborhoods are randomly constructed in a sequential and local fashion. The next theorem gives a simulation of algorithms in the model.
Proposition 9 (simulation of in ).
Let be an -scan algorithm that assumes an arbitrary ordering of nodes. Then there is a randomized algorithm , such that starting from the same initial memory states , the algorithm terminates and returns the same output as within rounds, where is the set of active nodes and is the radius of the ball accessed by the active node in the th scan of algorithm , both fully determined by .
Compared to the simulation of Monte Carlo algorithm in the model proved in [15], which relies on the network decomposition to parallelize the procedure, Proposition 9 provides a rather straightforward simulation that does not parallelize the local computations. An advantage of such an easy simulation is that it does not require a worst-case complexity upper bound for all scan orders of nodes. This translation from to algorithm makes it more convenient to describe algorithms where there are multiple randomly growing local neighborhoods interfering with each other.
A formal proof of Proposition 9 is given in the full version of the paper.
3 Exposition of the Algorithm: Special Case
In this section, we expose the key ideas of the main sampling algorithm stated in Theorem 4 within a special case. The detailed and formal construction of the algorithm will be presented in the next section.
As stated in Theorem 4, the algorithm deals with an LLL instance with bad events. Assume that is -satisfiable, and let be its dependency graph.
Our sampling algorithm follows a natural two-step framework:
-
Initially, the algorithm generates a random assignment according to the product distribution . This can be easily achieved by having each node independently generate its own variables.
-
If none of the bad events in occurs, then follows the correct distribution . Otherwise, if some bad events have occurred on , the algorithm locally applies some corrections to to ensure that the corrected assignment follows the correct distribution .
Obviously, the nontrivial part of the algorithm is the above second step, in which the algorithm locally modifies a random assignment to a new assignment when some bad events occur on .
To simplify our exposition, we start by making the following idealized assumption.
Assumption 10.
Only one node has its corresponding bad event occurs on .
Next, we will expose the main idea of our sampling algorithm by explaining how to resolve the sampling problem under this idealized assumption. The rest of this section is organized as follows:
-
1.
We begin by approaching the sampling problem with modest objectives. Our first goal is to sample with a bounded expected complexity under Assumption 10, while also assuming a correlation decay property. With these assumptions, we present a sampling algorithm. (Section 3.1)
-
2.
The correlation decay assumed above may not always hold. To address this, we introduce an augmentation of the LLL instance, inducing desirable correlation decay by properly augmenting the LLL instance. This enables us to remove the correlation decay assumption from the earlier mentioned algorithm, ensuring correct sampling from despite augmenting the LLL instance . (Section 3.2)
-
3.
Finally, we introduce a recursive sampling framework, which upgrades the aforementioned sampling algorithms with bounded expected complexity to algorithms with exponentially convergent running time. This proves Theorem 4 under Assumption 10. (Section 3.3)
In the full version of the paper, we eliminate the need for Assumption 10 through the introduction of a new section that addresses the general case involving multiple bad events.
3.1 Warm-up: Sampling with idealized correlation decay
In this part, we assume the correlation decay property as formally defined in Definition 8. This idealized correlation decay property is assumed for exposition purpose, and reliance on it will be eliminated later.
Assumption 11.
Any pair of disjoint are -correlated.
Let be generated according to the product distribution . Under Assumption 10, there is only one node having occur on . Let and . Notice that we have
| (6) |
because is distributed as and all bad events except do not occur on .
An attempt to correct resampling.
Our goal is to locally fix the random assignment around to make it distributed as . A natural attempt is to resample evaluation of according to the correct marginal distribution . This would produce a new assignment which is distributed as:
whereas, our goal is that each is sampled with probability .
To remedy this, we apply the Bayes filters introduced in [10]. A Bayes filter is a trial whose success or failure is determined by . The probability that succeeds satisfies
| (7) |
where are taken over all possible with .
Now given a Bayes filter satisfying (7), we conduct an experiment of , and resample if succeeds. This will produce a due to the Bayes law derived as follows:
| (8) | ||||
Otherwise, if fails, we trivially resample the entire , which still ensures the correctness of sampling but uses global information. Overall, the above procedure correctly produces a .
It remains to construct a good Bayes filter using local generation and succeeding with high probability.
Construction of the Bayes filter .
Condition (7) of the Bayes filter can be expressed as
| (9) | ||||
Note that can be computed locally from . It is thus natural to define as
where denotes the maximum value of taken over all all possible with .
It is obvious to see that for the Bayes filter constructed as above, an experiment of can be conducted and observed locally from . Furthermore, due to the correlation decay assumed in Assumption 11, succeeds with high probability. Let be the set of possible assignments on the set of variables:
Let be the assignment on that achieve the maximum . It holds that
| (10) | ||||
where the last inequality is derived from that and are -correlated, guaranteed by Assumption 11.
This simple sampling algorithm under Assumption 10 and Assumption 11 is described in Algorithm 1. By (10) and the locality of the Bayes filter, the algorithm terminates within rounds in expectation.
3.2 Sampling without correlation decay via LLL augmentation
The correlation decay asked by Assumption 11 may not always hold for general -satisfiable LLL instances. A key idea is then to properly “augment” the LLL instance by including new bad events to enforce desirable decay of correlation. This is highlighted by the following LLL augmentation lemma.
Recall the notions of balls , shells , and rings formally defined in Section 2.1.
Lemma 12 (LLL augmentation).
Let be a LLL instance. Let , and where . For any nonempty , if the sub-instance is -satisfiable, then a new bad event with can be constructed such that:
-
1.
is defined on , and is constructed using local information on ;
-
2.
has probability at most on independent random variables , i.e. ;
-
3.
the variable sets and are -correlated after including into .
Definition 13 (the augmenting event).
Let be a LLL instance. Let , , and . Fix any nonempty that is -satisfiable. We use to denote the new bad event constructed in Lemma 12.
Remark 14.
Lemma 12 constitutes a critical component of our sampling approach. Beyond the algorithmic implications, it provides new insights into correlation between variables constrained by local constraints.
This lemma aims to establish a decay of correlation between two distant regions and . However, it only assumes that all local constraints sandwiched between and are collectively easy to satisfy. Merely knowing this fact does not prevent and from being strongly correlated. Remarkably, Lemma 12 shows that the only obstacle to achieving correlation decay between and in this case lies in rarely occurring “bad” assignments between them. Consequently, and can be effectively de-correlated by introducing a new locally defined bad event over the region between and to prohibit those bad assignments that cause significant correlation between them.
This argument will be formalized in the full version of the paper, where a formal restatement of Lemma 12 will be introduced, and the lemma will be rigorously proved.
Sampling using augmented LLL.
Now we show how to utilize this LLL augmentation stated in Lemma 12 to get rid of Assumption 11 in Algorithm 1, while still assuming Assumption 10.
As before, let be generated according to the product distribution ; and let and , where is the only node (Assumption 10) with the bad event occurring on .
Our goal is to utilize Lemma 12 to induce the necessary decay of correlation required for sampling. However, augmenting LLL would inevitably alter the distribution . The saving grace is the following observation, suggesting that sampling may not be affected by certain local changes to the distribution.
Observation 15.
The sampling in Algorithm 1 is correct and efficient as long as:
-
(correctness) follows the marginal distribution ;
-
(efficiency) and are -correlated.
Using this observation, we can apply the LLL augmentation described in Lemma 12 to ensure correlation decay between and , while preserving the marginal distribution . Consequently, the resulting sampling process is both correct and efficient without relying on Assumption 11.
Note the new bad event in Definition 13, and define its complement :
| (11) |
Correspondingly, define the following two augmented LLL instances:
| (12) |
Let be a sufficiently small constant. We define the choice of parameter
| (13) |
In the following, let be defined as above on with parameter . Let and , where .
By the same argument for (6), we still have . Notice that is identical except for the new bad event . Then, conditioned on , we have in the augmented instance . By Lemma 12 on our choice of parameter, and are -correlated in . Thus, a calling to Sampling-with-decay() (Algorithm 1) will produce a random assignment within rounds in expectation.
An idealized sampling algorithm.
Then an idealized sampling procedure may proceed as follows. Our goal is to modify the input to a new . This is achieved differently depending on whether the new bad event occurs on . If occurs on (which happens with a small probability), we generate an entire new using global information. Otherwise, if does not occur on , we can use the following cleverer approach to produce a :
-
With probability , generate a , where is defined as
(14) Since avoids , this can be achieved by calling Sampling-with-decay() to produce .
-
Otherwise, generate entire using global information.
Overall, this correctly generates a This procedure for sampling is idealized because it uses a value as defined in (14) that is not easy to compute locally.
Bootstraping the unknown threshold .
The probability in (14) can be lower bounded. By Lemma 12, occurs with probability at most . With the parameter specified in (13), we have
Therefore, when does not occur on , the above idealized sampling procedure can be realized as: first call the subroutine Sampling-with-decay() defined in Algorithm 1 to produce , and then with probability , compute the value of (which uses global information) and generate the entire with probability . The resulting algorithm is described in Algorithm 2.
Algorithm 2 may use global information, particularly in Line 5 with probability , and in Line 7 when occurs on , which happens with probability at most due to Lemma 12.222Technically, 10 may bias the probability of on . However, Algorithm 2 is solely used for exposition purposes, and such bias caused by 10 will no longer be an issue beyond in Algorithm 2. As we discussed, the call to Sampling-with-decay() at Line 3 returns within rounds in expectation. At last, the construction of the augmented instance takes rounds. Overall, Algorithm 2 returns a within rounds in expectation without relying on Assumption 11.
3.3 Recursive sampling with exponential convergence
Both Algorithm 1 and Algorithm 2 occasionally rely on global information with a probability of . While the expected time complexity remains well bounded, achieving exponential convergence, as outlined in Theorem 4, would be more desirable in randomized running time.
We introduce a recursive sampling framework to achieve such exponential convergence in the time complexity. And more importantly, this framework is also crucial for finally getting rid of Assumption 10.
First, we adapt Algorithm 1 to the recursive sampling framework. This adaptation is direct and simple. Then, we achieve the recursive sampling without Assumption 11. Although part of this has already been embodied in Algorithm 2, there are still some substantial difficulties that need to be overcome.
Recursive sampling assuming correlation decay.
At first, we still assume Assumption 11. The recursive sampling algorithm in this scenario is described in Algorithm 3. There are two differences between this algorithm and Algorithm 1: First, instead of taking just a node as input in Algorithm 1, now Algorithm 3 takes a subset as input. Second and more importantly, the original step of sampling using global information in Algorithm 1, is now replaced by a recursive call in Algorithm 3.
The correctness of recursive sampling relies on the following conditional Gibbs property, which is adapted from the same property introduced in [12, 10] for Gibbs distributions.
Definition 16 (conditional Gibbs).
Let be a LLL instance. The random pair , where is an assignment and is a subset of events, is said to satisfy conditional Gibbs property on instance , if for any and with , conditioned on , it holds that .
Intuitively, the random pair represents a “partially correct” sample with its problematic part covered by . The conditional Gibbs property guarantees that except for this problematic part, the sample is always distributed correctly. This gives a key invariant condition for the correctness of recursive sampling.
The following is easy to verify for satisfying Assumption 10 with .
Condition 17.
satisfies conditional Gibbs property on instance .
The following can be verified routinely by a structural induction: As long as Condition 17 is satisfied by the input, Algorithm 3 terminates with probability 1 and returns a that is distributed as . This guarantees the correctness of Algorithm 3 as a sampling algorithm.
To further bound the complexity of Algorithm 3, we need to assume Assumption 11. With such assumption, by the same analysis as in (10), the Bayes filter in Line 2 succeeds with probability at least . Hence, for any , Algorithm 3 terminates in rounds with probability .
Recursive sampling with LLL augmentation.
Our goal here is to adopt Algorithm 2 for sampling without the additional assumption about correlation decay, into the recursive sampling framework. The resulting algorithm will solve the problem in Theorem 4 under Assumption 10.
As in Algorithm 2, we need LLL augmentation. The following notion of augmented conditional Gibbs property is a variant of the conditional Gibbs property in Definition 16, tailored with LLL augmentation.
Definition 18 (augmented conditional Gibbs).
Let , , and , where is defined as in Lemma 12. Let be a LLL instance. The random pair , where and , is said to satisfy augmented conditional Gibbs property on instance with parameter , if for any with :
-
1.
the sub-instance is -satisfiable;
-
2.
for , , for any with , conditioned on that , follows the distribution , i.e.,
where stands for the augmented LLL instance with as in Definition 13.
Recall the choice of parameter in (13) and let . Let be the minimal integer that avoids the bad event . The following can be routinely verified on the instance , the random assignment , , along with the parameter .
Condition 19.
The following hold:
-
, and ;
-
the LLL instance is -satisfiable and the sub-instance is -satisfiable;
-
satisfies the augmented conditional Gibbs property on instance with parameter .
Our goal is then to modify the random assignment around the region to make it follow the correct distribution , as long as Condition 19 is satisfied. Adopting the definitions of and as in Equations 11, 12, and 14 with and . By our choice of , the bad event may never occur on . Then, Algorithm 2 can be simiplied to the case where always aoids .
Recursive bootstrapping of marginal probabilities.
A challenge, as we mentioned in Section 3.2, is that the true value of the marginal probability as defined in (14) is hard to compute locally. This is now resolved by a more sophisticated bootstrapping of the threshold , by recursively estimating it within a progressively more accurate interval . The estimation is formally stated by the following lemma, which is proved in the full version of the paper. It shows that the threshold can be estimated exponentially more accurately as the radius of local information grows.
Lemma 20.
Let be a LLL instance. Let , , and , where the function is as defined in Lemma 12. For any nonempty and an arbitrary event defined on the variables in , assuming that is -satisfiable and is -satisfiable, there is a determined only by , , and , such that
The high-level strategy for sampling can be outlined as follows. Given the current estimate interval of the threshold , as provided by Lemma 20, the algorithm operates as follows:
-
With probability , the algorithm is determined to enter the branch for sampling . This can be resolved efficiently as in Algorithm 3, because of the correlation decay in the augmented instance .
-
With probability , the algorithm is determined to enter the branch for sampling . This can be resolved recursively with an appropriately enlarged neighborhood containing , ensuring that Condition 19 is still satisfied invariantly.
-
Otherwise, the algorithm enters the so-called “zone of indecision”, where it enlarges the local radius to gather more information in order to obtain a more accurate estimate of the threshold .
Overall, the above procedure generates a random assignment . Throughout the recursive calls, we ensure to maintain the invariant Condition 19. The procedure RecursiveSampling(; ) is detailed in Algorithm 4.
Our use of “zones of indecision” for recursive bootstrapping of marginal probabilities is inspired from the Anand-Jerrum algorithm introduced in [1] for solving perfect simulation of Gibbs distributions.
The correctness of Algorithm 4 follows from a structural induction. The complexity of Algorithm 4 is challenging to analyze due to the random recursion, nevertheless, we apply a potential method to bound it.
Lemma 21.
Assume the input of RecursiveSampling satisfy Condition 19.
-
1.
After RecursiveSampling returns, follows the distribution .
-
2.
For any , with probability , RecursiveSampling accesses and updates , where
Remark 22.
It is important to note that Assumption 10 is not assumed in the statement of Lemma 21. Instead, the lemma holds as long as that Condition 19 is satisfied. The scenario described in Assumption 10 can in fact be incorporated into Condition 19 as a special case where .
4 Conclusion
In this paper, we show that for local computation, the successful output of any fixed-round Las Vegas computation, where failures are reported locally, can be perfectly simulated with polylogarithmic overheads.
As by-products, this gives perfect simulations, via efficient (polylogarithmic-round) local computation, for several fundamental classes of high-dimensional joint distributions, including:
-
random satisfying solutions of Lovász local lemma with non-negligible satisfiability;
-
uniform locally checkable labelings (LCLs) with non-negligible feasibility;
-
Gibbs distributions satisfying the strong spatial mixing with exponential decay.
We develop a novel approach for augmenting Lovász local lemma (LLL) instances by introducing locally-defined new bad events, to create the desirable decay of correlation. We also give a recursive local sampling procedure, which utilizes the correlation decay to accelerate the sampling process, and meanwhile still keeps the sampling result correct, without being biased by the change to the LLL instance.
At first glance, it almost looks like we are creating mixing conditions out of nothing. In particular, the approach seems to bypass the local-lemma-type conditions for sampling (e.g. the one assumed in [20]). But indeed, our augmentation of LLL instances relies on that the LLL instances are fairly satisfiable (or at least the separator between the regions that we want to de-correlate should be enough satisfiable). Sampling in such instances might already be tractable in conventional computation models, e.g. in polynomial-time Turing machine, but the problem remains highly nontrivial for local computation.
This new approach for perfect simulation works especially well in the models where the locality is the sole concern. A fundamental question is how this could be extended to the models where the computation and/or communication costs are also concerned, e.g. model or model.
References
- [1] Konrad Anand and Mark Jerrum. Perfect sampling in infinite spin systems via strong spatial mixing. SIAM Journal on Computing, 51(4):1280–1295, 2022. doi:10.1137/21M1437433.
- [2] László Babai. Monte-carlo algorithms in graph isomorphism testing. Université tde Montréal Technical Report, DMS, (79-10), 1979.
- [3] Ivona Bezáková, Andreas Galanis, Leslie Ann Goldberg, Heng Guo, and Daniel Stefankovic. Approximation via correlation decay when strong spatial mixing fails. SIAM Journal on Computing, 48(2):279–349, 2019. doi:10.1137/16M1083906.
- [4] Yi-Jun Chang and Seth Pettie. A time hierarchy theorem for the local model. SIAM J. Comput., 48(1):33–69, January 2019. doi:10.1137/17M1157957.
- [5] Kai-Min Chung, Seth Pettie, and Hsin-Hao Su. Distributed algorithms for the Lovász local lemma and graph coloring. In Proceedings of the 33th ACM Symposium on Principles of Distributed Computing (PODC), pages 134–143, 2014. doi:10.1145/2611462.2611465.
- [6] Kenneth L Clarkson. Las vegas algorithms for linear and integer programming when the dimension is small. Journal of the ACM (JACM), 42(2):488–499, 1995. doi:10.1145/201019.201036.
- [7] Peter Davies-Peck. On the locality of the lovász local lemma. In Proceedings of the 57th Annual ACM Symposium on Theory of Computing, STOC ’25, pages 1271–1282, New York, NY, USA, 2025. Association for Computing Machinery. doi:10.1145/3717823.3718103.
- [8] Paul Erdos and László Lovász. Problems and results on 3-chromatic hypergraphs and some related questions. Infinite and finite sets, 10(2):609–627, 1975.
- [9] Weiming Feng, Heng Guo, Chunyang Wang, Jiaheng Wang, and Yitong Yin. Towards derandomising markov chain monte carlo. In FOCS, 2023.
- [10] Weiming Feng, Heng Guo, and Yitong Yin. Perfect sampling from spatial mixing. Random Structures & Algorithms, 61(4):678–709, 2022. doi:10.1002/RSA.21079.
- [11] Weiming Feng, Heng Guo, Yitong Yin, and Chihao Zhang. Fast sampling and counting k-sat solutions in the local lemma regime. Journal of the ACM (JACM), 68(6):1–42, 2021. doi:10.1145/3469832.
- [12] Weiming Feng, Nisheeth K Vishnoi, and Yitong Yin. Dynamic sampling from graphical models. SIAM Journal on Computing, 50(2):350–381, 2021. doi:10.1137/20M1315099.
- [13] Weiming Feng and Yitong Yin. On local distributed sampling and counting. In PODC, 2018.
- [14] Mohsen Ghaffari, David G Harris, and Fabian Kuhn. On derandomizing local distributed algorithms. In FOCS, 2018.
- [15] Mohsen Ghaffari, Fabian Kuhn, and Yannic Maus. On the complexity of local distributed graph problems. In STOC, 2017.
- [16] Heng Guo, Mark Jerrum, and Jingcheng Liu. Uniform sampling through the Lovász local lemma. Journal of the ACM (JACM), 66(3):18:1–18:31, 2019. doi:10.1145/3310131.
- [17] Bernhard Haeupler, Barna Saha, and Aravind Srinivasan. New constructive aspects of the Lovász local lemma. Journal of the ACM (JACM), 58(6):28, 2011.
- [18] David G Harris. New bounds for the moser-tardos distribution. Random Structures & Algorithms, 57(1):97–131, 2020. doi:10.1002/RSA.20914.
- [19] Kun He, Xiaoming Sun, and Kewen Wu. Perfect sampling for (atomic) lov’asz local lemma. arXiv preprint arXiv:2107.03932, 2021.
- [20] Kun He, Chunyang Wang, and Yitong Yin. Sampling lovász local lemma for general constraint satisfaction solutions in near-linear time. In FOCS, 2022.
- [21] Vishesh Jain, Huy Tuan Pham, and Thuy Duong Vuong. On the sampling Lovász local lemma for atomic constraint satisfaction problems. arXiv, abs/2102.08342, 2021. arXiv:2107.03932.
- [22] Vishesh Jain, Huy Tuan Pham, and Thuy Duong Vuong. Towards the sampling lovász local lemma. In FOCS, 2021.
- [23] Mark R. Jerrum, Leslie G. Valiant, and Vijay V. Vazirani. Random generation of combinatorial structures from a uniform distribution. Theoret. Comput. Sci., 43:169–188, 1986. doi:10.1016/0304-3975(86)90174-X.
- [24] Gil Kalai. A subexponential randomized simplex algorithm. In STOC, 1992.
- [25] Nathan Linial. Locality in distributed graph algorithms. SIAM Journal on Computing (SICOMP), 21(1):193–201, 1992. doi:10.1137/0221015.
- [26] Nathan Linial and Michael Saks. Low diameter graph decompositions. Combinatorica, 13(4):441–454, 1993. doi:10.1007/BF01303516.
- [27] Michael Luby, Alistair Sinclair, and David Zuckerman. Optimal speedup of las vegas algorithms. Information Processing Letters, 47(4):173–180, 1993. doi:10.1016/0020-0190(93)90029-9.
- [28] D Mitchell, B Selman, and H Leveque. A new method for solving hard satisfiability problems. In AAAI, 1992.
- [29] Michael Mitzenmacher and Eli Upfal. Probability and Computing: Randomized Algorithms and Probabilistic Analysis. Cambridge University Press, 2005. doi:10.1017/CBO9780511813603.
- [30] Ankur Moitra. Approximate counting, the lovász local lemma, and inference in graphical models. Journal of the ACM (JACM), 66(2):1–25, 2019. doi:10.1145/3268930.
- [31] Robin A Moser and Gábor Tardos. A constructive proof of the general Lovász local lemma. Journal of the ACM (JACM), 57(2):11, 2010.
- [32] Rajeev Motwani and Prabhakar Raghavan. Randomized algorithms. Cambridge university press, 1995. doi:10.1017/CBO9780511814075.
- [33] David Peleg. Distributed computing: a locality-sensitive approach. SIAM, 2000.
- [34] James G. Propp and David B. Wilson. Exact sampling with coupled Markov chains and applications to statistical mechanics. Random Structures Algorithms, 9(1-2):223–252, 1996. doi:10.1002/(SICI)1098-2418(199608/09)9:1/2\%3C223::AID-RSA14\%3E3.0.CO;2-O.
- [35] Chunyang Wang and Yitong Yin. A sampling lovász local lemma for large domain sizes. In FOCS, 2024.
