Document Open Access Logo

Infinitely Divisible Noise for Differential Privacy: Nearly Optimal Error in the High ε Regime

Authors Charlie Harrison , Pasin Manurangsi

PDF

File

Thumbnail PDF
PDF
LIPIcs.FORC.2025.12.pdf
  • Filesize: 1.8 MB
  • 24 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Mathematics of computing → Distribution functions
  • Security and privacy
Keywords
  • Differential Privacy
  • Distributed Noise Addition

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

Differential privacy (DP) can be achieved in a distributed manner, where multiple parties add independent noise such that their sum protects the overall dataset with DP. A common technique here is for each party to sample their noise from the decomposition of an infinitely divisible distribution. We analyze two mechanisms in this setting: 1) the generalized discrete Laplace (GDL) mechanism, whose distribution (which is closed under summation) follows from differences of i.i.d. negative binomial shares, and 2) the multi-scale discrete Laplace (MSDLap) mechanism, a novel mechanism following the sum of multiple i.i.d. discrete Laplace shares at different scales. For ε ≥ 1, our mechanisms can be parameterized to have O(Δ³ e^{-ε}) and O(min(Δ³ e^{-ε}, Δ² e^{-2ε/3})) MSE, respectively, where Δ denote the sensitivity; the latter bound matches known optimality results. Furthermore, the MSDLap mechanism has the optimal MSE including constants as ε → ∞. We also show a transformation from the discrete setting to the continuous setting, which allows us to transform both mechanisms to the continuous setting and thereby achieve the optimal O(Δ² e^{-2ε / 3}) MSE. To our knowledge, these are the first infinitely divisible additive noise mechanisms that achieve order-optimal MSE under pure DP for either the discrete or continuous setting, so our work shows formally there is no separation in utility when query-independent noise adding mechanisms are restricted to infinitely divisible noise. For the continuous setting, our result improves upon Pagh and Stausholm’s Arete distribution which gives an MSE of O(Δ² e^{-ε/4}) [Pagh and Stausholm, 2022]. Furthermore, we give an exact sampler tuned to efficiently implement the MSDLap mechanism, and we apply our results to improve a state of the art multi-message shuffle DP protocol from [Balle et al., 2020] in the high ε regime.

Cite As Get BibTex

Charlie Harrison and Pasin Manurangsi. Infinitely Divisible Noise for Differential Privacy: Nearly Optimal Error in the High ε Regime. In 6th Symposium on Foundations of Responsible Computing (FORC 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 329, pp. 12:1-12:24, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.FORC.2025.12

Author Details

Charlie Harrison
  • Google, Austin, TX, USA
Pasin Manurangsi
  • Google Research, Bangkok, Thailand

Acknowledgements

Thanks to Peter Kairouz for useful discussions and pointing out the prior work, especially [Bagdasaryan et al., 2022]. This work was supported by Google.

References

  1. John M Abowd. The US Census Bureau adopts differential privacy. In KDD, pages 2867-2867, 2018. Google Scholar
  2. Naman Agarwal, Peter Kairouz, and Ziyu Liu. The skellam mechanism for differentially private federated learning. NeurIPS, pages 5052-5064, 2021. Google Scholar
  3. Horst Alzer and Graham Jameson. A harmonic mean inequality for the digamma function and related results. Rendiconti del Seminario Matematico della Università di Padova, 137:203-209, 2017. Google Scholar
  4. Apple Differential Privacy Team. Learning with privacy at scale. Apple Machine Learning Journal, 2017. Google Scholar
  5. Andreas Athanasiou, Konstantinos Chatzikokolakis, and Catuscia Palamidessi. Enhancing metric privacy with a shuffler. In PETS 2025 - 25th Privacy Enhancing Technologies Symposium, 2025. Google Scholar
  6. Eugene Bagdasaryan, Peter Kairouz, Stefan Mellem, Adrià Gascón, Kallista Bonawitz, Deborah Estrin, and Marco Gruteser. Towards sparse federated analytics: Location heatmaps under distributed differential privacy with secure aggregation. Proceedings on Privacy Enhancing Technologies, 4:162-182, 2022. URL: https://doi.org/10.56553/POPETS-2022-0104.
  7. Borja Balle, James Bell, Adrià Gascón, and Kobbi Nissim. Private summation in the multi-message shuffle model. In CCS, pages 657-676, 2020. URL: https://doi.org/10.1145/3372297.3417242.
  8. James Henry Bell, Kallista A. Bonawitz, Adrià Gascón, Tancrède Lepoint, and Mariana Raykova. Secure single-server aggregation with (poly)logarithmic overhead. In CCS, pages 1253-1269, 2020. URL: https://doi.org/10.1145/3372297.3417885.
  9. Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy-preserving machine learning. In CCS, pages 1175-1191, New York, NY, USA, 2017. URL: https://doi.org/10.1145/3133956.3133982.
  10. Mark Bun and Thomas Steinke. Concentrated differential privacy: Simplifications, extensions, and lower bounds. In TCC, pages 635-658, 2016. URL: https://doi.org/10.1007/978-3-662-53641-4_24.
  11. Clément L Canonne, Gautam Kamath, and Thomas Steinke. The discrete gaussian for differential privacy. NeurIPS, pages 15676-15688, 2020. Google Scholar
  12. Clément L. Canonne and Hongyi Lyu. Uniformity testing in the shuffle model: Simpler, better, faster. In SOSA, pages 182-202, 2022. URL: https://doi.org/10.1137/1.9781611977066.13.
  13. Albert Cheu, Adam D. Smith, Jonathan R. Ullman, David Zeber, and Maxim Zhilyaev. Distributed differential privacy via shuffling. In EUROCRYPT, pages 375-403, 2019. URL: https://doi.org/10.1007/978-3-030-17653-2_13.
  14. Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor. Our data, ourselves: Privacy via distributed noise generation. In EUROCRYPT, pages 486-503, 2006. URL: https://doi.org/10.1007/11761679_29.
  15. Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In TCC, pages 265-284, 2006. URL: https://doi.org/10.1007/11681878_14.
  16. Úlfar Erlingsson, Vitaly Feldman, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Abhradeep Thakurta. Amplification by shuffling: From local to central differential privacy via anonymity. In SODA, pages 2468-2479, 2019. URL: https://doi.org/10.1137/1.9781611975482.151.
  17. Quan Geng, Peter Kairouz, Sewoong Oh, and Pramod Viswanath. The staircase mechanism in differential privacy. IEEE Journal of Selected Topics in Signal Processing, 9(7):1176-1184, 2015. URL: https://doi.org/10.1109/JSTSP.2015.2425831.
  18. Quan Geng and Pramod Viswanath. The optimal mechanism in differential privacy. In ISIT, pages 2371-2375, 2014. URL: https://doi.org/10.1109/ISIT.2014.6875258.
  19. Quan Geng and Pramod Viswanath. The optimal noise-adding mechanism in differential privacy. IEEE Transactions on Information Theory, 62(2):925-951, 2016. URL: https://doi.org/10.1109/TIT.2015.2504967.
  20. Badih Ghazi, Noah Golowich, Ravi Kumar, Rasmus Pagh, and Ameya Velingker. On the power of multiple anonymous messages: Frequency estimation and selection in the shuffle model of differential privacy. In EUROCRYPT, pages 463-488, 2021. URL: https://doi.org/10.1007/978-3-030-77883-5_16.
  21. Badih Ghazi, Ravi Kumar, and Pasin Manurangsi. Pure-dp aggregation in the shuffle model: Error-optimal and communication-efficient. In ITC, pages 4:1-4:13, 2024. URL: https://doi.org/10.4230/LIPICS.ITC.2024.4.
  22. Badih Ghazi, Ravi Kumar, Pasin Manurangsi, and Rasmus Pagh. Private counting from anonymous messages: Near-optimal accuracy with vanishing communication overhead. In ICML, pages 3505-3514, 2020. URL: http://proceedings.mlr.press/v119/ghazi20a.html.
  23. Badih Ghazi, Ravi Kumar, Pasin Manurangsi, Rasmus Pagh, and Amer Sinha. Differentially private aggregation in the shuffle model: Almost central accuracy in almost a single message. In ICML, pages 3692-3701, 2021. URL: https://proceedings.mlr.press/v139/ghazi21a.html.
  24. Badih Ghazi, Pasin Manurangsi, Rasmus Pagh, and Ameya Velingker. Private aggregation from fewer anonymous messages. In Anne Canteaut and Yuval Ishai, editors, EUROCRYPT, pages 798-827, 2020. URL: https://doi.org/10.1007/978-3-030-45724-2_27.
  25. Arpita Ghosh, Tim Roughgarden, and Mukund Sundararajan. Universally utility-maximizing privacy mechanisms. SIAM J. Comput., 41(6):1673-1693, 2012. URL: https://doi.org/10.1137/09076828X.
  26. Slawomir Goryczka and Li Xiong. A comprehensive comparison of multiparty secure additions with differential privacy. IEEE transactions on dependable and secure computing, 14(5):463-477, 2015. Google Scholar
  27. Charlie Harrison. Distributed noise addition and infinite divisibility. https://charlieharrison.xyz/2024/08/05/distributed-noise.html, 2024. Accessed: 2025-02-11.
  28. Creighton Heaukulani and Daniel M Roy. Black-box constructions for exchangeable sequences of random multisets. arXiv preprint arXiv:1908.06349, 2019. Google Scholar
  29. Wolfram Research, Inc. Mathematica, Version 12.0, 2019. Champaign, IL, 2019. Google Scholar
  30. Yuval Ishai, Eyal Kushilevitz, Rafail Ostrovsky, and Amit Sahai. Cryptography from anonymity. In FOCS, pages 239-248, USA, 2006. URL: https://doi.org/10.1109/FOCS.2006.25.
  31. Norman L. Johnson, Samuel Kotz, and Narayanaswamy Balakrishnan. Discrete Multivariate Distributions, chapter 40, pages 200-203. Wiley, 1997. Google Scholar
  32. Charles H Jones. Generalized hockey stick identities and n-dimensional blockwalking. The Fibonacci Quarterly, 34(3):280-288, 1996. Google Scholar
  33. Peter Kairouz, Ziyu Liu, and Thomas Steinke. The distributed discrete gaussian mechanism for federated learning with secure aggregation. In ICML, pages 5201-5212, 2021. URL: http://proceedings.mlr.press/v139/kairouz21a.html.
  34. Peter Kairouz, Brendan McMahan, Shuang Song, Om Thakkar, Abhradeep Thakurta, and Zheng Xu. Practical and private (deep) learning without sampling or shuffling. In ICML, pages 5213-5225, 2021. URL: http://proceedings.mlr.press/v139/kairouz21b.html.
  35. Seetha Lekshmi V and Simi Sebastian. A skewed generalized discrete laplace distribution. IJMSI, 2:95-102, 2014. Google Scholar
  36. Ilya Mironov. On significance of the least significant bits for differential privacy. In CCS, pages 650-661, 2012. URL: https://doi.org/10.1145/2382196.2382264.
  37. Ilya Mironov. Rényi Differential Privacy. In CSF, pages 263-275, 2017. URL: https://doi.org/10.1109/CSF.2017.11.
  38. F. W. J. Olver, A. B. Olde Daalhuis, D. W. Lozier, B. I. Schneider, R. F. Boisvert, C. W. Clark, B. V. Saunders B. R. Mille and, H. S. Cohl, and eds. M. A. McClain. NIST Digital Library of Mathematical Functions. https://dlmf.nist.gov/15.2, Release 1.2.2 of 2024-09-15, 2024.
  39. Rasmus Pagh and Nina Mesing Stausholm. Infinitely divisible noise in the low privacy regime. In ALT, pages 881-909, 2022. URL: https://proceedings.mlr.press/v167/pagh22a.html.
  40. Feng Qi. Bounds for the ratio of two gamma functions. Journal of Inequalities and Applications, 2010:1-84, 2010. Google Scholar
  41. Aaron Schein, Zhiwei Steven Wu, Alexandra Schofield, Mingyuan Zhou, and Hanna Wallach. Locally private bayesian inference for count models. In ICML, pages 5638-5648, 2019. URL: http://proceedings.mlr.press/v97/schein19a.html.
  42. F William Townes. Review of probability distributions for modeling count data. arXiv preprint arXiv:2001.04343, 2020. Google Scholar
  43. Filipp Valovich and Francesco Alda. Computational differential privacy from lattice-based cryptography. In International Conference on Number-Theoretic Methods in Cryptology, pages 121-141, 2017. Google Scholar
  44. Zheng Xu, Yanxiang Zhang, Galen Andrew, Christopher A. Choquette-Choo, Peter Kairouz, H. Brendan McMahan, Jesse Rosenstock, and Yuanbo Zhang. Federated learning of gboard language models with differential privacy. In ACL: Industry Track, pages 629-639, 2023. URL: https://doi.org/10.18653/V1/2023.ACL-INDUSTRY.60.
  45. Mingyuan Zhou. Nonparametric bayesian negative binomial factor analysis. Bayesian Analysis, 13, April 2016. URL: https://doi.org/10.1214/17-BA1070.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact