Smooth Sensitivity Revisited: Towards Optimality
Abstract
Smooth sensitivity is one of the most commonly used techniques for designing practical differentially private mechanisms. In this approach, one computes the smooth sensitivity of a given query on the given input and releases with noise added proportional to this smooth sensitivity. One question remains: what distribution should we pick the noise from?
In this paper, we give a new class of distributions suitable for the use with smooth sensitivity, which we name the distribution. This distribution improves upon the state-of-the-art Student’s T distribution in terms of standard deviation by arbitrarily large factors, depending on a “smoothness parameter” , which one has to set in the smooth sensitivity framework. Moreover, our distribution is defined for a wider range of parameter , which can lead to significantly better performance.
Furthermore, we prove that the distribution converges for to the Laplace distribution and so does its variance. This means that the Laplace mechanism is a limit special case of the mechanism. This implies that our mechanism is in a certain sense optimal for .
Keywords and phrases:
differential privacy, smooth sensitivityFunding:
Richard Hladík: Supported in part by the European Research Council (ERC) under the European Union’s Horizon 2020 Research and Innovation Programme (grant agreement No. 949272)Copyright and License:
2012 ACM Subject Classification:
Security and privacy Privacy-preserving protocolsAcknowledgements:
We would like to thank Rasmus Pagh and Adam Sealfon for helpful discussions. We would like to thank Rasmus Pagh for hosting Richard at the University of Copenhagen. Jakub worked on this paper while affiliated with BARC, University of Copenhagen. Richard partially worked on this paper while visiting BARC, University of Copenhagen.Funding:
Partially funded by the Ministry of Education and Science of Bulgaria’s support for INSAIT as part of the Bulgarian National Roadmap for Research Infrastructure. Partially supported by the VILLUM Foundation grants 54451 and 16582.Editors:
Mark BunSeries and Publisher:
Leibniz International Proceedings in Informatics, Schloss Dagstuhl – Leibniz-Zentrum für Informatik
1 Introduction
Differential privacy has in recent years become the golden standard for data privacy. Possibly the most common way of achieving differential privacy for numerical queries is the Laplace mechanism, which adds noise from the Laplace distribution scaled with the global sensitivity of the query.111The global sensitivity is defined as the greatest difference in the query value between some two neighboring datasets. However, global sensitivity has one significant shortcoming: it is not adaptive to the dataset at hand. Namely, the error on every input is the same as that on the most difficult input, which determines the global sensitivity.
One of the most common ways to get practical mechanisms with instance-adaptive performance is to use smooth sensitivity. In contrast with global sensitivity, the smooth sensitivity can adapt to a specific dataset, meaning that easier inputs can have significantly lower errors. The approach works by computing the -smooth sensitivity (Definition 2) of the given query on the input at hand for a smoothness parameter , and releasing with added noise scaled with the -smooth sensitivity, picked from an appropriately chosen distribution. The parameter here controls a tradeoff between how large the -smooth sensitivity is on one hand, and how concentrated a distribution we can use for the noise on the other. This leaves one important question unanswered: what distribution should the noise be sampled from?
In this paper, we give a new distribution class suitable for use with smooth sensitivity, which we name the distribution (Definition 6). The specific distribution from the class that we use depends on the smoothness parameter and the privacy parameter . Our distribution has two advantages over the state-of-the-art Student’s T distribution: (1) it achieves significantly smaller error, having standard deviation smaller by arbitrarily large factors (depending on ), as shown in Figure 1, and (2) it is defined for any , whereas the previously used distributions all required . While this may seem as an improvement by a constant factor, it may in fact bring asymptotic decrease of the error as smooth sensitivity can change by an arbitrarily large amount even with a small change in . Our method also significantly improves upon the Laplace distribution for all but tiny values of (Laplace distribution moreover provides only approximate DP when used with smooth sensitivity). One of the main strengths of our mechanism is that it is completely general and it provides an improvement in any application relying on smooth sensitivity.
We also prove that for , our distributions converge in distribution to the Laplace distribution and the variance of also converges to that of the Laplace distribution. At the same time, with , the -smooth sensitivity converges to the global sensitivity. This means that in some sense, our proposed mechanism generalizes the Laplace mechanism, which is a limit special case of our mechanism for . Since the Laplace mechanism is optimal for pure differential privacy [8], this means that our method is also optimal for in the sense that any quantile of converges to that of the optimal distribution. Our result can be summarized as follows:
Theorem 1 (Informal version of Theorem 8).
Let us have a query and two parameters . Let be a dataset. There exists a class of distributions such that releasing
is -DP. Moreover, for , the variance is up to a factor the same as scaling the Laplace distribution with the smooth sensitivity. Also, for any value it holds that .
Note that this convergence result is meaningful despite the fact that the smooth sensitivity converges to the global sensitivity for : This result implies that for fixed small values of , the distribution is close to optimal, while the smooth sensitivity will, in general, still be asymptotically smaller than the global sensitivity. This refutes the following argument, that one might naturally make against our result: Namely, that since the smooth sensitivity is, for , equal to the global sensitivity, this result does not show anything that would not already hold for the Laplace mechanism.
Smooth Sensitivity vs. Inverse Sensitivity Mechanism
Recently, an alternative to smooth sensitivity appeared in the form of the inverse sensitivity mechanism [1, 2]. This approach has nice theoretical guarantees, and mechanisms that are optimal up to logarithmic factors have been designed by relying on inverse sensitivity. Admittedly, this reduces the overall impact of our approach. However, we are still convinced that smooth sensitivity has an important place in the repertoire of differential privacy techniques, and that it will continue to be used in practice.
The inverse sensitivity mechanism comes with its own problems. First, inverse sensitivity may be difficult to compute [1, 7], even in cases when smooth sensitivity is easy to compute, such as for counting triangles in a graph. While it is often possible to efficiently approximately implement the inverse sensitivity mechanism [1, 7], this already leads to worsened performance – it remains to be seen whether this approach will be competitive in practice for a wide range of problems.
Moreover, there are practical considerations that often favor the smooth sensitivity approach. First, it is possible to have an efficient generic implementation just based on an oracle that returns the smooth sensitivity. This is clearly not possible for the inverse sensitivity mechanism – the inverse sensitivity mechanism instantiates the exponential mechanism which in general cannot be implemented in time sublinear in the size of the universe. It is also due to this ease of use that there is extensive literature on the uses of smooth sensitivity on the applied side, as we discuss in Section 1.2. We think that even just the existence of this large amount of literature on smooth sensitivity would make improving smooth sensitivity an important undertaking.
Another issue with the inverse sensitivity mechanism is that it may in some cases have worse constants than smooth sensitivity. Specifically, it relies on the exponential mechanism which loses a factor of 2 in some instances. There are other mechanisms that could be used in place of the exponential mechanism, most notably the report-noisy-max mechanism [5], that may have better constants, but even those lose a factor of 2 in the standard deviation in some cases. For example, the Laplace mechanism can be seen as an instantiation of the exponential mechanism or report noisy max, but in doing so, one would lose a factor of 2. Our mechanism for smooth sensitivity does not have this issue (unlike the previously known smooth sensitivity mechanisms that achieved pure differential privacy).
1.1 Technical Overview
The principle that motivated the distribution is that we wanted to make sure that for , the privacy loss will be the same regardless of what value we released (recall that the privacy loss random variable is a function of the value that we release). Indeed, this is precisely the property that the Laplace distribution has in the context of global sensitivity. We formalize this property as a claim about some derivatives involving the density of the distribution (Lemma 12). Intuitively speaking, these derivatives capture an “infinitesimal privacy loss” as if we had two neighboring datasets whose query values as well as smooth sensitivities differ infinitesimally. In Lemma 9, we essentially integrate this infinitesimal privacy loss in order to express the true privacy loss. Doing so takes bounds on the infinitesimal privacy loss and gives bounds on the actual privacy loss, proving differential privacy.
Intuitively speaking, our improvement comes from two sources. The first is unsurprisingly that our distribution simply has a smaller variance than the Student’s distribution. The second source of our improvement is more subtle: we do not split the privacy budget the way previous work did. Intuitively speaking, there are two sources of privacy loss when we use smooth sensitivity: from the change in the query value between adjacent inputs (corresponding to shifting of noise) and from the change in the smooth sensitivity between neighboring datasets (corresponding to scaling noise). The standard approach is to split the privacy budget between the privacy loss caused by the shifting and the privacy loss caused by scaling. Surprisingly, if the scaling is not too large, it does not contribute anything to the worst-case privacy loss with the distribution. This allows us in some sense to use all of the privacy budget to “pay” for the change in query value between neighboring datasets, leading to a more efficient use of the privacy budget.
1.2 Related Work
Smooth sensitivity has been proposed by Nissim, Raskhodnikova, and Smith [12] as an alternative to global sensitivity, rectifying the global sensitivity’s drawback that it is determined by the most difficult input. They have shown how it can be used to privately approximately release the median, minimum, minimum spanning tree, and the number of triangles in a graph. Bun and Steinke [3] later gave new distributions suitable for use with smooth sensitivity. Notably, they propose the use of the Student’s T distribution. They also propose other distributions but those only work with the weaker concentrated differential privacy [6], which we do not focus on in this paper.
Since its inception, smooth sensitivity has been used in many applications. An incomplete list includes: principal component analysis [9], deep learning [13], statistics from genome-wide association studies [17], generating synthetic datasets using Bayesian models [11], generating synthetic graphs [15], random forests [16], or some graph centrality measures [10].
Other frameworks for beyond-worst-case differentially private mechanisms include propose-test-release [4], or privately bounding local sensitivity [14, Section 3.4]. However, these techniques only give approximate differential privacy and they generally speaking tend to need larger datasets to work. Namely, to get non-trivial guarantees, they usually need a dataset of size .222This is the case for propose-test-release as it otherwise always fails. For privately bounding local sensitivity, this is the case assuming the local sensitivity has global sensitivity 1 (it often has bigger sensitivity in which case the situation is even worse). For these reasons as well as for the sake of brevity, we choose not to focus on these techniques.
2 Preliminaries
2.1 Differential Privacy
Let us have a symmetric notion of dataset adjacency . A randomized algorithm is said to be -differentially private if, for any two adjacent datasets , and for any measurable set , the following condition holds:
where controls the trade-off between privacy and data utility. Smaller values of provide stronger privacy guarantees, but may limit the accuracy of the analysis. One of the most notable ways of achieving differential privacy is using the notion of smooth sensitivity:
Definition 2 (Smooth sensitivity).
Let us have a query . We define the local sensitivity of at as
In turn, we define the -smooth sensitivity of at as
where denotes distance induced by .
2.2 Calculus
We now give several statements from calculus. The following is a standard result.
Fact 3.
Let us have . If at a point , all partial derivatives exist and are continuous, then the function is differentiable at .
For the following two statements, we choose to give statements of special cases that suffice for our needs, instead of giving the more general cases. We defer their proofs to Appendix A.
Fact 4.
Let us have a function . At any at which is differentiable, it holds
Fact 5.
Let us have a real function . It holds that
3 Improved Smooth Sensitivity Mechanism
Definition 6.
We define the distribution with scale and shape by its probability density function
for .
We plot a few example members of this family of distributions in Figure 2, where we also compare it with the Laplace distribution.
We start by claiming that this distribution is correctly defined. We defer the proof to Appendix A.
Lemma 7.
The distribution is correctly defined for . That is, the density is non-negative and integrates to .
We now state our main result.
Theorem 8.
Let us have a query . Let us have two parameters . Then the mechanism
is -DP. Moreover, for , it holds that
and for any value it holds that .
Proof.
We prove the asymptotic expansion of the variance in Corollary 14. We show that the distribution converges to Laplace in Lemma 15.
It remains to prove privacy. Let denote the density function of . We then have
where final inequality holds by Lemma 9. This implies -differential privacy.
Proving Privacy
In this section, we prove Lemma 9, which is the result we used above to prove that our mechanism is differentially private. The technical parts of the proof of the lemma appear in Lemmas 12, 10, and 11. The casual reader may wish to skip the proof of Lemma 9 as well as the other lemmas in this subsection.
Lemma 9.
Let for being the set of all possible datasets, be a query function. Let us have two neighboring datasets and . Then
where is defined in Definition 6.
Proof.
Let and define such that . Let also and . Note that for any and , it holds
| (1) |
We now give a bound and we explain the individual steps below.
| (2) | |||
| (3) | |||
| (4) | |||
| (5) | |||
| (6) | |||
| (7) | |||
| (8) |
We now justify the manipulations that may not be clear. Equality (2) holds by Equation 1. Equality (3) holds by the definition of and . The equality (4) holds because of the fundamental theorem of calculus; we use here the fact that by Lemma 11, the function
is differentiable except at points where , at which points it is continuous; furthermore, the function is continuous everywhere and differentiable everywhere except for at most one point where . The equality (5) holds by (4); we again used that is differentiable everywhere except for the at most one satisfying . The equality (6) holds by viewing each summand as a univariate function in and applying Equation 5. The bound (7) holds because we argue below that the removed constants are . The equality (8) holds by the Lemma 12 below.
It remains to prove . The definition of smooth sensitivity ensures that , and thus it holds that . It also holds
and thus
It thus holds as we needed to prove.
In the following lemma, we explicitly calculate the derivatives of with respect to and , which is useful for Lemmas 11 and 12.
Lemma 10.
For and defined as , it holds for any :
Proof.
The statement follows from direct computation. Full proof is deferred to Appendix A. We proceed with a technical lemma that justifies the invocation of the fundamental theorem of calculus in the proof of Lemma 9.
Lemma 11.
Let ; . The function
is continuous everywhere. It is differentiable everywhere, except when .
Proof.
Set and take from Lemma 10. Define and . Then and we have
Now we can apply Lemma 10 on the right-hand sides, together with the chain rule, to obtain that the partial derivatives on the left exist and are continuous for all such that . By Fact 3 and the fact that around every , there is a sufficiently small neighborhood where , we get that is differentiable (and thus continuous) in every with .
It remains to prove that is also continuous for . It suffices to show that is continuous at for all choices of . The first branch of is continuous everywhere, and for any , there exists a sufficiently small -neighborhood of where , that is, where . Therefore, is continuous at .
Lemma 12.
Let and let . It holds
| (9) |
Proof sketch.
For the full proof, see Appendix A. The main idea of the proof is to calculate both expressions in the sum using Lemma 10, with and . Then, we distinguish two cases: either , or . For both cases separately, and for each absolute value, we can argue that the expression inside this absolute value will always have the same sign, and can be therefore written without the absolute value. This simplifies the analysis, and we can then verify that the left-hand side of Equation 9 always sums up to . Finally, we substitute and back to get .
3.1 Properties of the Distribution
We now give a lemma that among others states the variance of . To make sure we have not made a mistake in the very technical manipulations, we have checked that we get the same result if we approximate the variance by numerical integration.333Specifically, by scaling, we can assume w.l.o.g. that . For this value of , we have plotted the variance as calculated by numeric integration and the expression above. We checked that they are indeed the same. We defer the proof to Appendix A.
Lemma 13.
has mean and variance
Corollary 14.
Assume . It holds
Proof.
We substitute into the above lemma with , , and take the Taylor expansion of order one w.r.t. around . We skip routine steps in the computation and just state that the second-order Taylor expansion is
This is exactly as we claim.
Note that the variance of the Laplace distribution scaled to the smooth sensitivity is , meaning that for , the variance of our mechanism approaches that of the Laplace mechanism scaled to the smooth sensitivity. In fact, the distribution converges to the Laplace distribution for .
Lemma 15.
For and any , it holds that converges pointwise to the probability density function (pdf) of .
Proof.
Let us denote the pdf of by . We prove pointwise convergence, that is, that for any , . This in turn implies the desired convergence in distribution.444This holds because by Scheffé’s lemma, this means that the TV distance of the two distributions goes to zero, which in turn implies convergence in distribution. Since is bounded, this follows if we prove that for and any fixed and , it holds that , denoted . We prove this claim in the rest of this proof.
Set , . It holds that
where the first uses the fact that the base of the exponential converges to and the second uses the standard limit . At the same time, it holds that
where the first again uses the limit . Overall, we can thus for write
as we wanted to show. The argument for is very similar:
and
where the second uses the standard limit for .
References
- [1] Hilal Asi and John C. Duchi. Instance-optimality in differential privacy via approximate inverse sensitivity mechanisms. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, volume 33, pages 14106–14117, 2020. URL: https://proceedings.neurips.cc/paper/2020/hash/a267f936e54d7c10a2bb70dbe6ad7a89-Abstract.html.
- [2] Hilal Asi and John C. Duchi. Near instance-optimality in differential privacy. CoRR, abs/2005.10630, 2020. doi:10.48550/arXiv.2005.10630.
- [3] Mark Bun and Thomas Steinke. Average-case averages: Private algorithms for smooth sensitivity and mean estimation. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, volume 32, pages 181–191, 2019. URL: https://proceedings.neurips.cc/paper/2019/hash/3ef815416f775098fe977004015c6193-Abstract.html.
- [4] Cynthia Dwork and Jing Lei. Differential privacy and robust statistics. In Proceedings of the 41st Annual ACM Symposium on Theory of Computing, STOC 2009, Bethesda, MD, USA, May 31 – June 2, 2009, pages 371–380. ACM, 2009. doi:10.1145/1536414.1536466.
- [5] Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science, 9(3-4):211–407, 2014. doi:10.1561/0400000042.
- [6] Cynthia Dwork and Guy N. Rothblum. Concentrated differential privacy. CoRR, abs/1603.01887, 2016. doi:10.48550/arXiv.1603.01887.
- [7] Juanru Fang, Wei Dong, and Ke Yi. Shifted inverse: A general mechanism for monotonic functions under user differential privacy. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, CCS 2022, Los Angeles, CA, USA, November 7-11, 2022, pages 1009–1022. ACM, 2022. doi:10.1145/3548606.3560567.
- [8] Natasha Fernandes, Annabelle McIver, and Carroll Morgan. The Laplace mechanism has optimal utility for differential privacy over continuous queries. In 36th Annual ACM/IEEE Symposium on Logic in Computer Science, LICS 2021, Rome, Italy, June 29 – July 2, 2021, pages 1–12. IEEE, IEEE, 2021. doi:10.1109/LICS52264.2021.9470718.
- [9] Alon Gonem and Ran Gilad-Bachrach. Smooth sensitivity based approach for differentially private PCA. In Algorithmic Learning Theory, ALT 2018, 7-9 April 2018, Lanzarote, Canary Islands, Spain, volume 83 of Proceedings of Machine Learning Research, pages 438–450. PMLR, PMLR, 2018. URL: http://proceedings.mlr.press/v83/gonem18a.html.
- [10] Jesse Laeuchli, Yunior Ramírez-Cruz, and Rolando Trujillo-Rasua. Analysis of centrality measures under differential privacy models. Applied Mathematics and Computation, 412:126546, 2022. doi:10.1016/j.amc.2021.126546.
- [11] Mingzhu Li and Xuebin Ma. Bayesian networks-based data publishing method using smooth sensitivity. In IEEE International Conference on Parallel & Distributed Processing with Applications, Ubiquitous Computing & Communications, Big Data & Cloud Computing, Social Computing & Networking, Sustainable Computing & Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018, Melbourne, Australia, December 11-13, 2018, pages 795–800. IEEE, IEEE, 2018. doi:10.1109/BDCloud.2018.00119.
- [12] Kobbi Nissim, Sofya Raskhodnikova, and Adam D. Smith. Smooth sensitivity and sampling in private data analysis. In Proceedings of the 39th Annual ACM Symposium on Theory of Computing, San Diego, California, USA, June 11-13, 2007, pages 75–84. ACM, 2007. doi:10.1145/1250790.1250803.
- [13] Lichao Sun, Yingbo Zhou, Philip S. Yu, and Caiming Xiong. Differentially private deep learning with smooth sensitivity. CoRR, abs/2003.00505, 2020. doi:10.48550/arXiv.2003.00505.
- [14] Salil P. Vadhan. The complexity of differential privacy. In Tutorials on the Foundations of Cryptography, pages 347–450. Springer International Publishing, 2017. doi:10.1007/978-3-319-57048-8_7.
- [15] Yue Wang and Xintao Wu. Preserving differential privacy in degree-correlation based graph generation. Trans. Data Priv., 6(2):127–145, 2013. URL: http://www.tdp.cat/issues11/abs.a113a12.php.
- [16] Bangzhou Xin, Wei Yang, Shaowei Wang, and Liusheng Huang. Differentially private greedy decision forest. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2019, Brighton, United Kingdom, May 12-17, 2019, pages 2672–2676. IEEE, IEEE, 2019. doi:10.1109/ICASSP.2019.8682219.
- [17] Akito Yamamoto and Tetsuo Shibuya. Privacy-preserving publication of GWAS statistics using smooth sensitivity. In 20th Annual International Conference on Privacy, Security and Trust, PST 2023, Copenhagen, Denmark, August 21-23, 2023, pages 1–12. IEEE, IEEE, 2023. doi:10.1109/PST58708.2023.10320160.
Appendix A Deferred Proofs
We start by proving the calculus facts from Section 2.2.
Proof of Fact 4.
The statement follows from the chain rule. Its special case states that for and differentiable and , and all such that , and is differentiable at , it holds
where the notation on right-hand side is an equivalent way of writing partial derivatives. In our case, , and , and substituting these facts in the equation above concludes the proof.
Proof of Fact 5.
The statement follows from the chain rule. Its special case states that if , is differentiable everywhere and is differentiable at , then . If we apply the chain rule on , then and we get that . Evaluated at , this becomes , as needed.
Next, we prove Lemma 7, which states that the distribution is correctly defined.
Proof of Lemma 7..
By scaling, we may assume without loss of generality that . By symmetry, it then suffices to integrate from to and verify that it integrates to . The first branch in the definition of integrates to555We do not show the detailed steps of how to compute the antiderivative. However, it should be noted that it is easy to check correctness by differentiating and checking that this results in the respective expressions we were integrating.
and the second integrates to
We now write the integral as
where in the last step we skipped some elementary manipulations.
Next, we prove Lemma 10, which provides a formula for the derivatives of .
Proof of Lemma 10.
Define
Then one can verify that
When taking derivatives, we can ignore the additive in the second branch as is constant.
We start with analyzing the derivatives of with respect to . For :
Now, for , , it holds:
| (10) |
Finally, for , each branch gives us a one-sided derivative at , and one can verify that both derivatives are equal and therefore Equation 10 holds for all .
Similarly, for :
Thus, for :
| (11) |
Again, the two branches are equal for and Equation 11 hence holds for all .
Proof of Lemma 12.
In this whole proof, fix . We will carry through the proof with and , and in the very end substitute and .
Per Lemma 10, it holds:
| (12) |
To justify the removal of the absolute value on the right-hand side, note that and thus and . Then consider two cases: if , then and thus , and if , then and thus .
Similarly,
To again justify the removal of absolute values, we distinguish two cases. If , then
and thus, . If , then, similarly,
and thus, .
Now we can combine both results to conclude that
To conclude the proof, set , and observe that and as needed. Thus, all previous claims hold, and we have
Now we prove Lemma 13, which states that the distribution has mean 0 and calculates its variance.
Proof of Lemma 13..
Assuming the expectation is defined, it has to be as the distribution is symmetric around this point. We now focus on the variance; by proving that the variance is finite, we also get that the expectation is defined and therefore . Similarly to proof of Lemma 7, we first separately give the indefinite integrals of the two branches, this time of :
We may now use this to write the integral as
Appendix B Other Mechanisms for Smooth Sensitivity
In this section, we calculate the variances of the generalized Cauchy distribution of [12], the Student’s T distribution [3], and the Laplace distribution, when used with the smooth sensitivity framework. We use these variances in Figure 1 in order to compare our distribution with these two distributions.
Claim 16.
When scaled according to Nissim, Raskhodnikova, and Smith [12], the generalized Cauchy distribution has, for and , a standard deviation
For other values of , the standard deviation is infinite.
Proof.
The generalized Cauchy distribution is defined in Nissim, Raskhodnikova, and Smith [12] as having density . Using a Computer Algebra System (such as Mathematica), it is easy to verify that the density is in fact equal to
Writing the variance as an integral and again using a CAS to evaluate it, one gets that the variance is
for and infinite otherwise. For two neighboring datasets , let us have such that and let . It is shown in Nissim, Raskhodnikova, and Smith [12] that the privacy loss is at most . If we substitute the bound , the remaining privacy budget is , meaning that we need to scale the distribution with in order to ensure the bound on privacy loss is at most , assuming . This results in a standard deviation of
Claim 17.
When scaled according to Bun and Steinke [3], the Student’s T distribution has, for and , a standard deviation
For other values of , the standard deviation is infinite.
Proof.
The variance of the T distribution with degrees of freedom is . At the same time, for , defined as in the proof above, the privacy loss is at most
We have . The remaining budget for privacy loss caused by the distribution shift is , meaning that we need
assuming . To achieve -differential privacy, for a fixed value of , we have to scale the distribution with . This results in a standard deviation of
Claim 18.
When scaled according to Nissim, Raskhodnikova, and Smith [12], the Laplace distribution has, for , a standard deviation
Proof.
The proof is essentially the same as the proofs above. It is shown by [12] that the privacy loss is at most . This means we have to scale the noise to be inversely proportional to . This leads to the claimed standard deviation.
