Near-Universally-Optimal Differentially Private
Minimum Spanning Trees
Abstract
Devising mechanisms with good beyond-worst-case input-dependent performance has been an important focus of differential privacy, with techniques such as smooth sensitivity, propose-test-release, or inverse sensitivity mechanism being developed to achieve this goal. This makes it very natural to use the notion of universal optimality in differential privacy. Universal optimality is a strong instance-specific optimality guarantee for problems on weighted graphs, which roughly states that for any fixed underlying (unweighted) graph, the algorithm is optimal in the worst-case sense, with respect to the possible setting of the edge weights.
In this paper, we give the first such result in differential privacy. Namely, we prove that a simple differentially private mechanism for approximately releasing the minimum spanning tree is near-optimal in the sense of universal optimality for the neighbor relation. Previously, it was only known that this mechanism is nearly optimal in the worst case. We then focus on the neighbor relation, for which the described mechanism is not optimal. We show that one may implement the exponential mechanism for MST in polynomial time, and that this results in universal near-optimality for both the and the neighbor relations.
Keywords and phrases:
differential privacy, universal optimality, minimum spanning treesFunding:
Richard Hladík: Supported in part by the European Research Council (ERC) under the European Union’s Horizon 2020 Research and Innovation Programme (grant agreement No. 949272).Copyright and License:
2012 ACM Subject Classification:
Security and privacy Privacy-preserving protocols ; Mathematics of computing Graph algorithmsAcknowledgements:
We would like to thank Rasmus Pagh for helpful discussions and hosting Richard at the University of Copenhagen. We would like to thank Bernhard Haeupler for helpful discussions. Jakub worked on this paper while affiliated with BARC, University of Copenhagen. Richard partially worked on this paper while visiting BARC, University of Copenhagen.Funding:
Partially funded by the Ministry of Education and Science of Bulgaria’s support for INSAIT as part of the Bulgarian National Roadmap for Research Infrastructure. Partially supported by the VILLUM Foundation grants 54451 and 16582.Editors:
Mark BunSeries and Publisher:
Leibniz International Proceedings in Informatics, Schloss Dagstuhl – Leibniz-Zentrum für Informatik
1 Introduction
The minimum spanning tree (MST) problem is one of the classic combinatorial problems, making the release of an MST a fundamental question in differential privacy. Unfortunately, if the edge set is private, this problem cannot be solved, as it would require us to release a subset of the edges (note that we want to release the edges of the MST and not just their total weight). It is therefore natural to consider this problem in the (standard) setting where the underlying unweighted graph is public, but the weights are private.
The following simple near-linear-time mechanism has been proposed for this problem [26]: add Laplacian noise to the edge weights, making them private, then find the MST with the noisy weights. At the same time, the author shows that this mechanism is near-optimal in the worst case.
In this paper, we prove a much stronger optimality claim for this algorithm, namely that it is universally optimal up to a logarithmic factor for the neighbor relation (i.e., two graphs on the same edge set are neighboring if the edge weights differ by in the norm). An algorithm (or differentially private mechanism) that works on weighted graphs is said to be universally optimal, if for any fixed underlying unweighted graph , it is worst-case optimal with respect to the possible settings of edge weights . That is, if we write a weighted graph , universal optimality states that for any fixed , we are worst-case optimal w.r.t. . Standard worst-case optimality, on the other hand, is worst-case w.r.t. both and .
We then focus on differential privacy with the neighbor relation (defined analogously to the neighbor relation), for which the above algorithm is not universally optimal. We instead prove that one may implement the exponential mechanism for MST in polynomial time by relying on a known sampling result [6]. We prove that this more complicated and somewhat slower -time algorithm does achieve universal optimality up to a logarithmic factor for both the and the neighbor relations. For the neighbor relation, this improves upon the PAMST algorithm of Pinot [23] which is only known to be near-optimal in the worst-case sense.
Our results are the first to prove the universal optimality of a differentially private mechanism.111The name universal optimality unfortunately has different meaning in different contexts. There are several results that are universally optimal with one of these different meanings, as we discuss in Section 1.2. We stick with the meaning commonly used in distributed algorithms. This is despite the fact that previous work in differential privacy has put a lot of emphasis on instance-specific performance guarantees: smooth sensitivity, propose-test-release, privately bounding local sensitivity, and inverse sensitivity mechanism are all examples of this trend. This makes it very natural to focus on universal optimality, perhaps making it somewhat surprising that universal optimality is not already being commonly used in differential privacy.
As a side note, we also prove that the above-mentioned linear-time algorithm is optimal up to a constant factor in the worst-case sense. This improves upon previous results which had a logarithmic gap [26]. Similarly, we prove that the exponential mechanism is for MST worst-case optimal up to a constant factor for both the and the neighbor relations.
1.1 Technical Overview
In this section, we briefly discuss the intuition and techniques behind our results. Here, we focus mostly on the neighbor relation and universal near-optimality. The lower bound arguments for and the worst-case optimality are similar. The comparison between ours and previous results is summarized in Table 1.
Our lower and upper bounds rely on the properties of the set of all spanning trees of a given (unweighted) graph . We define a Hamming-like metric in this space, defined as . It turns out that the diameter of with respect to is a natural parameter that determines the “hardness” of . Namely, we show an lower bound and an upper bound on the expected error of the optimal -differentially private algorithm.
Upper bound
In Section 3, we use the diameter to obtain a sharper analysis of the Laplace mechanism of Sealfon [26]. The mechanism is very simple: add Laplacian noise to every edge, then return the MST with respect to the noisy weights. Its standard analysis uses the fact that with high probability, the noise on every edge is , and thus the total error of the spanning tree returned is . Our improvement follows from the fact that the returned spanning tree differs from the MST in at most edges, and thus the total error accumulated is actually only .
Lower bound
The most technically interesting part of this paper is our lower bound in Section 4. We have a fixed underlying graph and for any -differentially private mechanism, we want to find a weight assignment on which the error will be large. Our lower bound builds on a general packing-based lower bound for differentially private algorithms, which we briefly paraphrase in the language of MSTs and universal optimality: Given a set of weight vectors and a parameter , define for each the set of spanning trees that are -light for this , i.e., that are heavier than the MST by at most . Moreover, assume that the sets are pairwise disjoint, that is, each spanning tree is -light with respect to at most one weight vector in . Then the expected error of any -differentially private algorithm is on at least one weight assignment , where is the diameter of in the metric induced by the neighbor relation .
Intuitively speaking, we have a set of weight assignments and for each of them, we consider a ball of outputs that are “good” in the sense that their error is “small” (with respect to ) for this input. Our goal is to find a large collection of weights that are “close” (i.e., is small), but which induce disjoint balls that are “wide” (i.e., we can set to be large).
Reduction to finding many dissimilar spanning trees
The problem of finding that maximizes the expression is somewhat complicated by the fact that we have a trade-off between , and . In order to solve this problem, we first show how to reduce it to a combinatorial problem of finding a large set of dissimilar spanning trees.
The main idea of the reduction is as follows: assume we have such that every two in have . We construct by, for each , creating a weight vector by defining if and otherwise. Now the crucial observation is that . That is, for any , the weight of under the weight vector is exactly the Hamming distance between and . One can then verify that for every , there is at most one weight under which its weight is at most , as otherwise we would, by the triangle inequality, for some have that , which would be a contradiction with the definition of . Therefore, we can set as large as while still ensuring the disjointness property of all required by the original lower bound.
An upper bound of can be argued as follows: We have that any two spanning trees differ in edges. Combined with having zero-one weights, any two spanning trees’ weight vectors thus also have -distance at most , and thus .
Using these ideas, we have reduced the original problem to the problem of finding a large set of spanning trees that is sparse in the sense that no two spanning trees in are similar. More specifically, we have reduced the original problem to the problem of finding a set of spanning trees which maximizes .
How to find many dissimilar spanning trees?
Finally, the problem of finding a large, yet sparse , is reducible to a standard problem: We show that there are always at least spanning trees, and the problem of finding a sparse subset among them can be reduced to finding a binary code of length and minimum Hamming distance that has many codewords. Such a code exists by the Gilbert–Varshamov bound [14, 28]. The that we get from this reduction then allows us to prove an lower bound on the expected error, which nearly matches the upper bound mentioned above.
1.2 Related Work
| neighborhood | on a fixed graph topology | worst-case | previous work (worst-case) | |||
|---|---|---|---|---|---|---|
| lower bound | upper bound | lower and upper bound | lower bound | upper bound | reference | |
| [26] | ||||||
| — | [23] | |||||
Differentially private minimum spanning trees
The problem of privately releasing the MST in the “public graph, private weights” setting was first studied by Sealfon [26]. The author shows a worst-case lower bound of on the expected error for the neighbor relation, proposes a simple mechanism based on adding Laplacian noise to all weights and calculating the MST with respect to the noisy weights, and proves that it is worst-case optimal up to an factor. In Sections 3 and 4, we show that this mechanism is in fact worst-case optimal up to a constant factor and also universally optimal up to an factor.
Pinot [23] gives a mechanism for the neighbor relation by running a noisy version of the Jarník-Prim algorithm that, in each step, selects the edge to be included in the tree by running the exponential mechanism. It has an expected error, which, by our results from Section 4, is worst-case optimal. However, the analysis does not seem to be easily modifiable to show universal optimality. We remark that the author claims an expected error of , but this is only true if one normalizes all weights by .
We also note that releasing the weight of the minimum spanning tree is easy. One may easily show that the global sensitivity is (with ) or (with ), allowing us to simply find the MST and release its weight using the Laplace mechanism. In their work on smooth sensitivity, Nissim, Raskhodnikova, and Smith [21] consider the problem of privately releasing the weight of the MST in a slightly different setting where the weights are bounded and neighboring datasets differ by changing the weight of one edge.
Other differential privacy notions on graphs
The notion of privacy used in this work was introduced by Sealfon [26] and was since used widely [4, 23, 24, 5, 10, 22]. To paraphrase a real-world motivation, the graph may represent a city’s (publicly known) metro network, with edge weights representing how busy each line is. User’s commute data is private information which contributes to the edge weights and should be protected.222Based on what specific data are stored for each user, this example motivates both the and neighbor relation. Other common privacy notions include edge differential privacy and node differential privacy [18], where the graph itself is unweighted and private, and two graphs and are neighboring if one can be obtained from the other by deleting an edge (for edge privacy) or a node and all its adjacent edges (for node privacy).
Instance-optimality in differential privacy
The notion of universal optimality is closely linked to that of instance optimality, which states that our algorithm is “as good as any mechanism could be” on every single instance. Universal optimality can then be seen as a combination of instance-optimality w.r.t. the underlying graph and worst-case optimality w.r.t. the edge weights.
In the last few years, several instance-optimality results have been proven in differential privacy [19, 3, 7, 1]. We highlight here Huang, Liang, and Yi [19] which give an instance-optimal differentially private mechanism for releasing the mean. We also highlight Asi and Duchi [1] who introduce the inverse sensitivity mechanism, and give instance-optimality results for mean estimation, performing linear regression, and the principal component analysis.
It should be noted that there are subtleties in the precise definitions of instance optimality that these papers use. The reason is that under the definition of instance-optimality that is commonly used in other areas, often no instance-optimal mechanism exists in differential privacy for trivial reasons. Therefore, the precise definitions used in the mentioned papers differ somewhat.
Issues with nomenclature
The name “universal optimality” has unfortunately been used to mean different things in different contexts. Throughout this paper, we use universal optimality in the sense in which it is commonly used in distributed algorithms [17]. It should be noted that there have been several works in differential privacy that use the name “universal optimality” to denote completely unrelated concepts [13, 11]. Namely, these papers use the name universal optimality in a Bayesian setting, where universal optimality states that a given mechanism is optimal no matter the prior. This is a completely unrelated notion to what we consider in this paper.
Universal optimality
The notion of universal optimality started in distributed algorithms where multiple classic combinatorial problems are now known to have universally optimal algorithms in various settings [17, 29, 16, 25]. We highlight here the paper by Haeupler, Wajc, and Zuzic [17] which gives a universally optimal algorithm in the supported CONGEST model for the minimum spanning tree; the techniques used in that paper are different from those that we use, despite both papers considering the MST problem.
Recently, it was shown that Dijkstra’s algorithm is universally optimal for a version of the single-source shortest paths problem [15] in the standard Word-RAM model. To the best of our knowledge, that paper was the first universal optimality result in a non-distributed setting. This makes this paper the second such result.
1.3 Future work
We believe that our technique can be applied to other graph problems. Specifically, the framework that we use for our lower bound seems to be quite general. We believe that it is likely that the same techniques could be used for releasing shortest-path trees. With some additional tweaks, we believe our techniques could be useful for problems such as maximum-weight matching or minimum-weight perfect matching. Extending our results to approximate differential privacy would also be of great interest.
2 Preliminaries
In this paper, we consider all graphs to be simple, undirected and connected. We consider the setting introduced by Sealfon [26], where the unweighted graph is public and fixed, and the only private information are the edge weights . We also assume that has at least two different spanning trees, i.e., it is not itself a tree. We denote by the set of all spanning trees of , and identify each with the set of its edges. It is folklore that for all connected graphs, , with the maximum attained when is a clique. We write as a shortcut for , and write to denote the minimum spanning tree under weights . If is clear from context, we write just . For a spanning tree , its error is defined as .
2.1 Differential Privacy
We define two notions of adjacency for weight vectors: , defined so that if and only if , and , defined so that if and only if .
A mechanism for MST is any randomized algorithm that, for a fixed and public unweighted graph , takes as input a weight vector and outputs a spanning tree of , specified by the list of its edges.333Formally, we have infinitely many mechanisms, each for one graph . What we will do instead is to pretend that there is a single mechanism that accepts as an additional parameter. We are interested in minimizing the expected error of , defined as . Furthermore, is -differentially private with respect to (which is either or ), if, for every and every ,
where is a parameter that controls the tradeoff between privacy and the expected error of .
2.2 Spanning Trees
We define the following metric on the space of spanning trees: .444The reader is invited to verify that is indeed a metric. We call the Hamming metric or Hamming distance between and . Note that and we could have used either in the previous definition. We then define as the diameter of the space of spanning trees of with respect to . Trivially, , but it can be much smaller: for example, we have when is a cycle, and generally, if has at most edges. In Section 5, we provide an exponential lower and upper bound on the relationship between and .
In the lower bounds, we will make use of special zero-one weight vectors, derived from spanning trees:
Definition 1.
For a spanning tree , define the weights as if , and otherwise.
Fact 2.
For two trees , it holds . In particular, . It also holds that and (with equality if ).
2.3 Universal Optimality
Universal optimality is a notion that, intuitively speaking, says that an algorithm is optimal for any fixed graph topology. It is most commonly used with respect to the time complexity, but here, we instead define it in terms of an expected error of an -differentially private mechanism.
Definition 3.
Let be any optimization problem on weighted graphs where for every input ,555Here is the set of all problem inputs; in our specific problem, this is the (uncountably infinite) set of all possible weighted connected graphs with all possible choices of edge weights. the goal is to produce an output minimizing some scoring function . Define . For a fixed graph and an algorithm , define worst-case expected error of on as
We say that an -differentially private mechanism is universally optimal for , if there exists a constant such that for any unweighted graph and any other -differentially private mechanism , we have We instead say that is universally optimal up to factor if
3 Simple Mechanism for Privately Releasing the MST
In this section, we show a tighter analysis of a very simple mechanism for privately releasing an MST, which originally appeared in Sealfon [26]. Whereas the original analysis proves that the expected error is at most for the neighbor relation, we prove a tighter bound of for (note that ). We prove in Section 4 a lower-bound of , showing that the algorithm is universally near-optimal. On the other hand, in Claim 26 we prove that for , the algorithm is neither worst-case nor universally optimal.
Our goal is to prove the following corollary. It follows from Corollary 7 (which states the upper bound), Theorem 9 (which states the lower bound) and the fact that the time complexity of Algorithm 5 is dominated by the runtime of an MST algorithm.
Corollary 4.
Algorithm 5 is -differentially private under the neighbor relation for any . It is universally optimal up to an factor for releasing the MST under the neighbor relation. It runs in near-linear time.
Algorithm 5 (MST via postprocessing).
Given the weight vector , let for and for . Release the noisy weights obtained by, for each edge, independently sampling , where is the Laplacian distribution with mean 0 and scale . Calculate the MST of and return it.
Theorem 6.
Algorithm 5, denoted here as , is -differentially private for both and . For , and for any , it holds that
where is the MST of and .
The idea of our proof is analogous to the original proof by Sealfon [26], except that in the last step, instead of bounding the error incurred by all edges of and , we bound the (smaller) error incurred by the edges in which and differ.
Proof.
By the guarantees of the Laplace mechanism [9], we can privately release the noised weight vector, where for , we additionally use that for . The privacy of Algorithm 5 then follows from the privacy of postprocessing.
Fix . For each edge , it holds that by the definition of . By the union bound, the probability that holds for all edges is at least . Let us condition on this event.
Let be the spanning tree returned by Algorithm 5, i.e., the MST of , and let be the MST of . We can write
We used, respectively, the fact that edges in do not contribute to the error, the fact that , rewriting, and the fact that is an MST under . Noting that finishes the proof.
Corollary 7.
Let be Algorithm 5 in the setting with . It holds that where is the MST of and .
Proof.
Lemma 8.
Let be a random variable and let us have , , such that for every , we have . Then .
Proof.
Equivalently, we can write Now let be such that . We have Denote and let , where is the exponential distribution. For any , we have , i.e., stochastically dominates , and thus . Thus, by linearity of expectation, , as needed.
4 Lower Bounds for MST
In this section, we focus on proving the lower bound. For , this is a lower bound that nearly matches the performance of Algorithm 5, for it nearly matches the performance of the mechanism from Section 6.
In both cases, there is a gap between the lower and upper bounds. This gap is inherent to our approach, but for some graphs, and in particular, for , we can make it disappear and get worst-case lower bounds of (for ) and (for ) that match our worst-case upper bounds from Section 6 up to a constant factor, as well as that of Sealfon [26] in the case of . Overall, we prove the following theorem:
Theorem 9.
Fix an unweighted graph and let be mechanism for MST on . If is -differentially private with respect to , then there exist weights such that where . Moreover, there exists at least one and where
If is -differentially private with respect to instead, then, for every fixed , there exist weights such that Moreover, there exists at least one and where
The rough outline of the proof is as follows: in Lemmas 10 and 11, we state (and restate in a language of MSTs) a general packing-based lower bound that applies whenever we can find many inputs (in our case, weight vectors) close in the metric induced by the neighbor relation such that, for each input, the set of all outputs (in our case, spanning trees) that give small error with respect to this input, is disjoint with all the other sets. In Lemma 12, we prove that instead of searching for a good set of weight vectors, we can search for a large set of spanning trees in which every two spanning trees differ in many edges. Lemmas 13 and 14 prove that such a large set always exists, with the latter being a special case providing stronger guarantees when the graph is a clique. We postpone their proofs to Section 5, where we also build the needed theory.
At the core of our lower bound is the following lemma, based on a general packing argument.
Lemma 10 (Vadhan [27], Theorem 5.13).
Let be a collection of datasets all at (neighbor-relation-induced) distance at most from some fixed dataset , and let be a collection of disjoint subsets of . If there is an -differentially private mechanism such that for every , then
The following rephrases Lemma 10 in the language of our problem. It also picks a specific way of constructing the sets , namely, by including precisely those outputs that have a “small enough” error with respect to .
Lemma 11 (Adaptation of Lemma 10 for MST).
Fix an unweighted graph and let be an arbitrary neighbor relation on . Given a collection of weight vectors, all at (-induced) distance at most from some fixed weight vector , and a parameter , denote for each the MST of by , and define the set of spanning trees that are “light” under . Assume that and are such that all are pairwise disjoint. Then for any -differentially private (with respect to ) mechanism , there exist weights , such that
It turns out that there is a very natural class of sets that give a reasonably good lower bound. Namely, the we use in our lower bound will be of the form for some choice of . This is formalized in the following statement. It says that to create our , we can start with any , and then use the mapping (recall that is the indicator function of a set , see Definition 1). The quality of the lower bound provided by depends solely on two things: the size of , and the minimum Hamming distance between distinct . The parameter can then be used to control the tradeoff between and the probability that the error will be smaller than in Lemma 11.
Lemma 12.
Fix an unweighted graph and denote . Given a set of spanning trees , let be such that for all . Finally, let be a mechanism for releasing a minimum spanning tree of . If is -differentially private with respect to , then there exist weights such that
If is -differentially private with respect to , then there exist weights such that
Proof.
Let us walk through the proof with the assumption that is -differentially private with respect to . At the very end, we will discuss the needed changes for .
Fix a parameter and define as . Later we will specify the value of that gives the needed bound.
Fix arbitrarily. In order to apply Lemma 11, we need to determine and . For each , we have , and thus .
Set . Then for every , we have:
We can conclude that all are disjoint: If we had for distinct and , then we would have , which would contradict the properties of , as .
Now we apply Lemma 11 to get that, for every -DP mechanism for releasing the MST of there exist weights , such that
Setting yields:
Finally, let us deal with the case when is -differentially private with respect to instead. The idea of the proof is identical and we only point out the differences. When calculating , we now get that , and thus . By Lemma 11, and as , there exist weights such that
Setting yields:
Lemma 12 reduces our problem into a problem of finding a set , which is as large as possible, and at the same time, the Hamming distance between any two spanning trees in the set is large. In Section 5, we prove the following two results:
Lemma 13.
For any unweighted graph with , there exists a set of size such that for any distinct , we have .
Lemma 14.
For any unweighted clique on vertices, there exists a set of size such that for any distinct , we have .
Now we are ready to prove Theorem 9.
Proof of Theorem 9.
Lemma 12 says that for any mechanism , there exist weights such that, if is -differentially private with respect to , then:
for and such that for any distinct . This means that , and thus, as :
Now, if is from Lemma 13, we have and . If then instead we take from Lemma 14, and and . Substituting twice in the equation above proves the first half of the theorem.
5 Finding a Large Set of Dissimilar Trees
In this section, our goal is to prove Lemmas 13 and 14, which assert the existence of a large set of spanning trees such that every two spanning trees differ in many edges. In Section 5.1, we state some common properties of spanning trees and prove that there are at least spanning trees (for ). In Section 5.2, we show how to embed binary block codes into the spanning trees from Section 5.1, which leads to our first method of constructing , and proving Lemma 13; this is the lemma that we need in order to prove our universal optimality results.
In Section 5.3, we bound the number of trees in the -ball around some tree , and use this in conjunction with a greedy packing argument to provide a different method of constructing that gives slightly different guarantees than the one in Section 5.2. We use this to prove Lemma 14; this is the lemma that we need to prove our worst-case optimality results.
5.1 Properties of Spanning Trees
Here we state some simple properties of spanning trees. Although (some of) these results could be considered folklore, we give proofs for completeness.
Lemma 15 (Exchange lemma).
Let be two spanning trees and let . Then there exists such that is also a spanning tree. Furthermore, .
Proof.
The graph has exactly one cycle . does not have cycles, and must thus contain an edge ; furthermore, , as . But then and thus as needed, and is again a tree. Finally, , as removing had no effect on and adding decreased it by .
Lemma 16 (Iterated exchange lemma).
Given two spanning trees , and a set of edges , there exists a spanning tree such that and .
Proof.
Let . The claim follows by induction on . If , then we can set . Otherwise, take any , and let . By induction, let be the tree such that and . Now invoke Lemma 15 with , and to obtain and a tree satisfying .
We claim that , since by induction, and the only edge additionally contains is . But simultaneously, by triangle inequality, Hence, and , and thus as needed and we can set .
Corollary 17.
For every graph , it holds that for .
Proof.
Fix any two such that . By Lemma 16, we have that for any , there exists a spanning tree such that . As there are possible choices of , and each of them yields a unique spanning tree, we can conclude that there are at least different spanning trees.
5.2 Dissimilar Trees via Binary Codes
In this section, we show that the set of trees from Corollary 17, behaves, in some sense, as the space . Namely, there is a correspondence between and , such that if differ in positions, then their corresponding spanning trees have Hamming distance at least . In this way, we reduce the problem of finding a set of dissimilar spanning trees to the problem of finding a good binary block code.
Lemma 18.
Given two spanning trees , , let . Let and be trees such that and . Then it holds that .
We note that and always exist thanks to Lemma 16.
Proof.
By rules for set subtraction, we have
We briefly recall the definition of a block code. Note that we use the less common definition of an code where is not the message length, but the (exponentially larger) number of codewords.
Definition 19.
An code is a set such that and every two different vectors differ in at least positions.
Next, we prove the reduction between finding a set of dissimilar spanning trees and finding a good block code:
Lemma 20.
Let be a code and let be an unweighted graph such that . Then there exists a set such that and for any two different .
Proof.
We will construct as follows: first, we fix and such that , as in Corollary 17. We number the edges of in any order as . Now, for every , define , and let be the tree obtained by invoking Lemma 16. Namely, satisfies . Finally, we set .
Clearly , and we need to show that for all distinct . Note that the number of positions in which and differ is exactly . Now we can use Lemma 18, to write
and thus , as needed.
Finally, we show that there always exists a good block code:
Lemma 21.
For every , there exists an code.
Proof.
First assume that is divisible by . By the Gilbert–Varshamov bound [14, 28], there exists a code for some that satisfies
Using the bound on the sum of binomial coefficients (see e.g. Flum and Grohe, p. 427 [12]), we obtain with being the binary entropy function . One can verify that and thus as needed.
Finally, if is not divisible by , we can apply the lemma with and then pad every codeword of the resulting code with zeros.
Combination of these results proves Lemma 13:
Proof of Lemma 13.
5.3 Dissimilar Trees via Greedy Packing
In this section, we provide a different approach for finding a large set of dissimilar spanning trees. It works by producing a crude upper bound on the number of trees in the -ball around a tree , and then using a greedy packing argument to show that we can always find of size .
Lemma 22 (Volume of a -ball around a spanning tree).
For a graph , , and , it holds that
Proof.
Any with can be fully (and possibly non-uniquely) described by a list of edges to be added to and another list of edges to be removed. Furthermore, both lists can be padded to have length exactly by repeating arbitrary entries. As there are at most possible pairs of lists of length , there must be at most possible trees .
Lemma 23 (Greedy packing).
For a graph , and a parameter , there exists a set such that for all distinct and furthermore,
Proof.
We will construct greedily: start with . As long as is nonempty, pick arbitrary and add it to . Then, set , and repeat. By Lemma 22, in each step we remove at most elements from a set of size , and therefore only stop after contains at least elements.
Now we are ready to prove Lemma 14:
Proof of Lemma 14.
We invoke Lemma 23 on with . Since is a clique, we have . Clearly, and , and we can write
Lemma 22 also gives us the following relationship between and :
Lemma 24.
For a graph with , it holds that .
Proof.
Take any and define . Necessarily by the definition of . Now we invoke Lemma 22 to conclude that .
6 Universal Near-Optimality via the Exponential Mechanism
In this section, we start by showing that Algorithm 5 is in fact not universally optimal for . The main contribution of this section is then that we prove that the exponential mechanism is universally near-optimal with respect to both and . We also show that it can be implemented in polynomial time by relying on a result of Colbourn, Myrvold, and Neufeld [6]. The properties of the exponential mechanism that we rely on are summarized in Fact 31 in Appendix A.
Our goal is to prove the following corollary. It follows from Theorem 29 (which states the upper bounds and time complexity) and Theorem 9 (which states the lower bounds).
Corollary 25.
For any , the exponential mechanism with loss function is universally optimal up to an factor for releasing the MST, in both the and the neighbor relations. It can be implemented in the matrix multiplication time .
We now show that Algorithm 5 is neither worst-case, nor universally optimal. Note also that when we are using the neighbor relation, the noise magnitude used by the algorithm is indeed optimal in the sense that any lower noise magnitude will not lead to the weights themselves being private after adding the noise. This can be easily seen as follows: With the current amount of noise added, if each weight changes by 1, we lose up to privacy on each edge. Since the composition theorem for pure differential privacy is tight, we thus may indeed lose up to privacy in total. Any lower amount of noise would not give -differential privacy.
Claim 26.
Denote Algorithm 5 used with the neighbor relation as . For every graph , there exist weights such that
Proof.
The claim follows immediately from the fact that is also -differentially private with respect to , and thus by the first part of Theorem 9 with , such weights must exist.
Below, we will prove that one can in fact achieve an expected error of . This implies that Algorithm 5 is in fact neither universally, nor worst-case optimal for .
In the rest of this section, our goal is to prove that the exponential mechanism can be implemented in polynomial time and that it is universally optimal for releasing the MST. We start by stating a useful result on efficiently sampling spanning trees.
Lemma 27.
There is an algorithm that, given an unweighted graph , a weight vector , and a parameter , samples a spanning tree of such that the probability that is returned is proportional to . It runs in matrix multiplication time .
Proof.
Such an algorithm is provided in Colbourn, Myrvold, and Neufeld [6]. The original paper only deals with unweighted graphs, but it is mentioned in Durfee et al. [8] that this approach is actually easily generalized to the weighted case.
Note that generally, any exact spanning tree sampling algorithm that supports weighted graphs works for Lemma 27. There are faster algorithms available, but every faster algorithm known to us either does not sample from the exact distribution (failing or sampling from a different distribution with some small probability ), or does not support weighted graphs out of the box.
The following lemma allows us to analyze the exponential mechanism more tightly by changing the loss function so that the outcome probabilities do not change, but the global sensitivity decreases.
Lemma 28.
Let be two loss functions related in the following way: for each , there exists such that for all , .
Given , let and be instantiations of the exponential mechanism that, given , sample with probability proportional to and , respectively. Then and are equivalent, that is, for each , they return the same distribution on .
Proof.
returns with probability
which is exactly the probability of returning .
Theorem 29.
There is an -time mechanism for MST, -differentially private with respect to , such that, for every weighted graph ,
where is the MST of and . Furthermore, there is an -time mechanism for MST, -differentially private with respect to , such that, for every weighted graph ,
Proof.
The asymptotic bounds on the error follow from the exact ones by Lemma 24, thus we will only focus on the exact inequalities. Let us assume the case first; we will deal with the case at the end of the proof.
will be an instantiation of the exponential mechanism from Lemma 27, with , and with determined later.
We will use Lemma 28 to analyze an equivalent exponential mechanism that uses the loss function defined as follows: fix globally some and define . It holds that, for every fixed , we can write , where does not depend on , and thus and are equivalent by Lemma 28.
Let us choose the right for (and thus also for ). By the standard properties of the exponential mechanism (see Fact 31), is -differentially private if we choose , where is the global sensitivity of , as defined in Fact 31. As a next step, we bound . For each , we have:
for . We used the fact that edges present in both and do not count towards the result. Hence, . If we thus choose in , we immediately get from Fact 31 that is -differentialy private and the expected error is at most , as needed. By the equivalence of and , the same holds for .
Finally, note that can compute (and thus ) quickly: namely, if we denote by the MST of a graph , then . That is because minimizes the expression , which, by Fact 2, is equal to , just as needed. The MST can be computed in linear time using e.g. the Jarník-Prim algorithm with a double-ended queue as the priority queue, as all weights are either or .
Let us now deal with the case. will again be an instantiation of the exponential mechanism. This time, we set and analyze directly, without the use of Lemma 28. For any and , we immediately have , and thus the global sensitivity of is . By Fact 31, is -differentially private and the expected error is at most , exactly as needed.
Since computed in the above proof satisfies , we immediately get the following corollary:
Corollary 30.
A 2-approximation of can be computed in linear time.
Note that the above algorithm is actually strictly stronger than Algorithm 5, as there are graphs where . We suspect that, in fact, the exponential mechanism is universally optimal for both and , but we were not able to prove a stronger lower bound.
References
- [1] Hilal Asi and John C. Duchi. Instance-optimality in differential privacy via approximate inverse sensitivity mechanisms. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, volume 33, pages 14106–14117, 2020. URL: https://proceedings.neurips.cc/paper/2020/hash/a267f936e54d7c10a2bb70dbe6ad7a89-Abstract.html.
- [2] Raef Bassily, Kobbi Nissim, Adam D. Smith, Thomas Steinke, Uri Stemmer, and Jonathan R. Ullman. Algorithmic stability for adaptive data analysis. In Proceedings of the 48th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2016, Cambridge, MA, USA, June 18-21, 2016, pages 1046–1059. ACM, 2016. doi:10.1145/2897518.2897566.
- [3] Jaroslaw Błasiok, Mark Bun, Aleksandar Nikolov, and Thomas Steinke. Towards instance-optimal private query release. In Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2019, San Diego, California, USA, January 6-9, 2019, pages 2480–2497. SIAM, SIAM, 2019. doi:10.1137/1.9781611975482.152.
- [4] Solenn Brunet, Sébastien Canard, Sébastien Gambs, and Baptiste Olivier. Edge-calibrated noise for differentially private mechanisms on graphs. In 14th Annual Conference on Privacy, Security and Trust, PST 2016, Auckland, New Zealand, December 12-14, 2016, pages 42–49. IEEE, IEEE, 2016. doi:10.1109/PST.2016.7906935.
- [5] Justin Y. Chen, Shyam Narayanan, and Yinzhan Xu. All-pairs shortest path distances with differential privacy: Improved algorithms for bounded and unbounded weights. CoRR, abs/2204.02335, 2022. doi:10.48550/arXiv.2204.02335.
- [6] Charles J. Colbourn, Wendy J. Myrvold, and Eugene Neufeld. Two algorithms for unranking arborescences. Journal of Algorithms, 20(2):268–281, 1996. doi:10.1006/jagm.1996.0014.
- [7] Wei Dong and Ke Yi. A nearly instance-optimal differentially private mechanism for conjunctive queries. In PODS ’22: International Conference on Management of Data, Philadelphia, PA, USA, June 12 – 17, 2022, pages 213–225. ACM, 2022. doi:10.1145/3517804.3524143.
- [8] David Durfee, Rasmus Kyng, John Peebles, Anup B. Rao, and Sushant Sachdeva. Sampling random spanning trees faster than matrix multiplication. In Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2017, Montreal, QC, Canada, June 19-23, 2017, pages 730–742. ACM, 2017. doi:10.1145/3055399.3055499.
- [9] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam D. Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography, Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006, Proceedings, volume 3876 of Lecture Notes in Computer Science, pages 265–284. Springer, Springer, 2006. doi:10.1007/11681878_14.
- [10] Chenglin Fan, Ping Li, and Xiaoyun Li. Private graph all-pairwise-shortest-path distance release with improved error rate. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 – December 9, 2022, volume 35, pages 17844–17856, 2022. URL: http://papers.nips.cc/paper_files/paper/2022/hash/71b17f00017da0d73823ccf7fbce2d4f-Abstract-Conference.html.
- [11] Natasha Fernandes, Annabelle McIver, Catuscia Palamidessi, and Ming Ding. Universal optimality and robust utility bounds for metric differential privacy. In 35th IEEE Computer Security Foundations Symposium, CSF 2022, Haifa, Israel, August 7-10, 2022, pages 348–363. IEEE, August 2022. doi:10.1109/CSF54842.2022.9919647.
- [12] Jörg Flum and Martin Grohe. Parameterized Complexity Theory. Texts in Theoretical Computer Science. An EATCS Series. Springer, 2006. doi:10.1007/3-540-29953-X.
- [13] Arpita Ghosh, Tim Roughgarden, and Mukund Sundararajan. Universally utility-maximizing privacy mechanisms. In Proceedings of the 41st Annual ACM Symposium on Theory of Computing, STOC 2009, Bethesda, MD, USA, May 31 – June 2, 2009, pages 351–360. ACM, 2009. doi:10.1145/1536414.1536464.
- [14] Edgar N Gilbert. A comparison of signalling alphabets. The Bell system technical journal, 31(3):504–522, 1952.
- [15] Bernhard Haeupler, Richard Hladík, Václav Rozhon, Robert E. Tarjan, and Jakub Tetek. Universal optimality of dijkstra via beyond-worst-case heaps. In 65th IEEE Annual Symposium on Foundations of Computer Science, FOCS 2024, Chicago, IL, USA, October 27-30, 2024, pages 2099–2130. IEEE, 2024. doi:10.1109/FOCS61266.2024.00125.
- [16] Bernhard Haeupler, Harald Räcke, and Mohsen Ghaffari. Hop-constrained expander decompositions, oblivious routing, and distributed universal optimality. In STOC ’22: 54th Annual ACM SIGACT Symposium on Theory of Computing, Rome, Italy, June 20 – 24, 2022, pages 1325–1338. ACM, 2022. doi:10.1145/3519935.3520026.
- [17] Bernhard Haeupler, David Wajc, and Goran Zuzic. Universally-optimal distributed algorithms for known topologies. In STOC ’21: 53rd Annual ACM SIGACT Symposium on Theory of Computing, Virtual Event, Italy, June 21-25, 2021, pages 1166–1179. ACM, 2021. doi:10.1145/3406325.3451081.
- [18] Michael Hay, Chao Li, Gerome Miklau, and David D. Jensen. Accurate estimation of the degree distribution of private networks. In ICDM 2009, The Ninth IEEE International Conference on Data Mining, Miami, Florida, USA, 6-9 December 2009, pages 169–178. IEEE, IEEE Computer Society, 2009. doi:10.1109/ICDM.2009.11.
- [19] Ziyue Huang, Yuting Liang, and Ke Yi. Instance-optimal mean estimation under differential privacy. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, volume 34, pages 25993–26004, 2021. URL: https://proceedings.neurips.cc/paper/2021/hash/da54dd5a0398011cdfa50d559c2c0ef8-Abstract.html.
- [20] Frank McSherry and Kunal Talwar. Mechanism design via differential privacy. In 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS 2007), October 20-23, 2007, Providence, RI, USA, Proceedings, pages 94–103. IEEE, IEEE Computer Society, 2007. doi:10.1109/FOCS.2007.41.
- [21] Kobbi Nissim, Sofya Raskhodnikova, and Adam D. Smith. Smooth sensitivity and sampling in private data analysis. In Proceedings of the 39th Annual ACM Symposium on Theory of Computing, San Diego, California, USA, June 11-13, 2007, pages 75–84. ACM, 2007. doi:10.1145/1250790.1250803.
- [22] Rasmus Pagh and Lukas Retschmeier. Faster private minimum spanning trees. CoRR, abs/2408.06997, 2024. doi:10.48550/arXiv.2408.06997.
- [23] Rafael Pinot. Minimum spanning tree release under differential privacy constraints. CoRR, abs/1801.06423, 2018. doi:10.48550/arXiv.1801.06423.
- [24] Rafael Pinot, Anne Morvan, Florian Yger, Cédric Gouy-Pailler, and Jamal Atif. Graph-based clustering under differential privacy. In Proceedings of the Thirty-Fourth Conference on Uncertainty in Artificial Intelligence, UAI 2018, Monterey, California, USA, August 6-10, 2018, pages 329–338. AUAI Press, 2018. URL: http://auai.org/uai2018/proceedings/papers/132.pdf.
- [25] Václav Rozhon, Christoph Grunau, Bernhard Haeupler, Goran Zuzic, and Jason Li. Undirected -shortest paths via minor-aggregates: near-optimal deterministic parallel and distributed algorithms. In STOC ’22: 54th Annual ACM SIGACT Symposium on Theory of Computing, Rome, Italy, June 20 – 24, 2022, pages 478–487. ACM, 2022. doi:10.1145/3519935.3520074.
- [26] Adam Sealfon. Shortest paths and distances with differential privacy. In Proceedings of the 35th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, PODS 2016, San Francisco, CA, USA, June 26 – July 01, 2016, pages 29–41. ACM, 2016. doi:10.1145/2902251.2902291.
- [27] Salil Vadhan. The complexity of differential privacy. Tutorials on the Foundations of Cryptography: Dedicated to Oded Goldreich, pages 347–450, 2017. doi:10.1007/978-3-319-57048-8_7.
- [28] Rom Rubenovich Varshamov. Estimate of the number of signals in error correcting codes. Doklady Akad. Nauk, SSSR, 117:739–741, 1957.
- [29] Goran Zuzic, Gramoz Goranci, Mingquan Ye, Bernhard Haeupler, and Xiaorui Sun. Universally-optimal distributed shortest paths and transshipment via graph-based -oblivious routing. In Proceedings of the 2022 ACM-SIAM Symposium on Discrete Algorithms, SODA 2022, Virtual Conference / Alexandria, VA, USA, January 9 – 12, 2022, pages 2549–2579. SIAM, SIAM, 2022. doi:10.1137/1.9781611977073.100.
Appendix A Exponential Mechanism
For completeness, we restate the guarantees of the exponential mechanism.
Fact 31 (Guarantees of the exponential mechanism [20, 2]).
Let be a function, and let be a neighbor relation on . The exponential mechanism that, given , samples with probability proportional to , is -differentially private and satisfies:
where is the minimizer of and is the global sensitivity of , defined as
