Document Open Access Logo

Additive Noise Mechanisms for Making Randomized Approximation Algorithms Differentially Private

Author Jakub Tětek

PDF
Thumbnail PDF

File

LIPIcs.APPROX-RANDOM.2024.73.pdf
  • Filesize: 0.77 MB
  • 20 pages

Document Identifiers

Author Details

Jakub Tětek
  • INSAIT, Sofia, Bulgaria

Funding

This work was done while the author was at BARC, University of Copenhagen. He was supported by the VILLUM Foundation grant 54451. This research was partially funded from the Ministry of Education and Science of Bulgaria (support for INSAIT, part of the Bulgarian National Roadmap for Research Infrastructure).

Acknowledgements

The author would like to thank Rasmus Pagh and anonymous reviewers for helping to improve this paper.

Cite AsGet BibTex

Jakub Tětek. Additive Noise Mechanisms for Making Randomized Approximation Algorithms Differentially Private. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2024). Leibniz International Proceedings in Informatics (LIPIcs), Volume 317, pp. 73:1-73:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2024)
https://doi.org/10.4230/LIPIcs.APPROX/RANDOM.2024.73

Abstract

The exponential increase in the amount of available data makes taking advantage of them without violating users' privacy one of the fundamental problems of computer science. This question has been investigated thoroughly under the framework of differential privacy. However, most of the literature has not focused on settings where the amount of data is so large that we are not even able to compute the exact answer in the non-private setting (such as in the streaming setting, sublinear-time setting, etc.). This can often make the use of differential privacy unfeasible in practice. In this paper, we show a general approach for making Monte-Carlo randomized approximation algorithms differentially private. We only need to assume the error R of the approximation algorithm is sufficiently concentrated around 0 (e.g. 𝔼[|R|] is bounded) and that the function being approximated has a small global sensitivity Δ. Specifically, if we have a randomized approximation algorithm with sufficiently concentrated error which has time/space/query complexity T(n,ρ) with ρ being an accuracy parameter, we can generally speaking get an algorithm with the same accuracy and complexity T(n,Θ(ε ρ)) that is ε-differentially private. Our technical results are as follows. First, we show that if the error is subexponential, then the Laplace mechanism with error magnitude proportional to the sum of the global sensitivity Δ and the subexponential diameter of the error of the algorithm makes the algorithm differentially private. This is true even if the worst-case global sensitivity of the algorithm is large or infinite. We then introduce a new additive noise mechanism, which we call the zero-symmetric Pareto mechanism. We show that using this mechanism, we can make an algorithm differentially private even if we only assume a bound on the first absolute moment of the error 𝔼[|R|]. Finally, we use our results to give either the first known or improved sublinear-complexity differentially private algorithms for various problems. This includes results for frequency moments, estimating the average degree of a graph in subliinear time, rank queries, or estimating the size of the maximum matching. Our results raise many new questions and we state multiple open problems.

Subject Classification

ACM Subject Classification
  • Security and privacy
  • Theory of computation → Streaming, sublinear and near linear time algorithms
Keywords
  • Differential privacy
  • Randomized approximation algorithms

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Related Versions

References

  1. Daniel Alabi, Omri Ben-Eliezer, and Anamay Chaturvedi. Bounded space differentially private quantiles. arXiv preprint, 2022. URL: https://arxiv.org/abs/2201.03380.
  2. Noga Alon, Yossi Matias, and Mario Szegedy. The space complexity of approximating the frequency moments. In Proceedings of the twenty-eighth annual ACM symposium on Theory of computing, pages 20-29, 1996. Google Scholar
  3. Raef Bassily, Kobbi Nissim, Uri Stemmer, and Abhradeep Guha Thakurta. Practical locally private heavy hitters. Advances in Neural Information Processing Systems, 30, 2017. Google Scholar
  4. Petra Berenbrink, Bruce Krayenhoff, and Frederik Mallmann-Trenn. Estimating the number of connected components in sublinear time. Information Processing Letters, 114(11):639-642, 2014. Google Scholar
  5. J Blocki, E Grigorescu, and T Mukherjee. Privately estimating graph parameters in sublinear time. In 49th International Colloquium on Automata, Languages, and Programming (ICALP 2022)., 2022. Google Scholar
  6. Jeremiah Blocki, Avrim Blum, Anupam Datta, and Or Sheffet. The johnson-lindenstrauss transform itself preserves differential privacy. In 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science, pages 410-419. IEEE, 2012. Google Scholar
  7. Jeremiah Blocki, Elena Grigorescu, Tamalika Mukherjee, and Samson Zhou. How to Make Your Approximation Algorithm Private: A Black-Box Differentially-Private Transformation for Tunable Approximation Algorithms of Functions with Low Sensitivity. In Nicole Megow and Adam Smith, editors, Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2023), volume 275 of Leibniz International Proceedings in Informatics (LIPIcs), pages 59:1-59:24, Dagstuhl, Germany, 2023. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.APPROX/RANDOM.2023.59.
  8. Jonas Boehler and Florian Kerschbaum. Secure sublinear time differentially private median computation, February 1 2022. US Patent 11,238,167. Google Scholar
  9. Seung Geol Choi, Dana Dachman-Soled, Mukul Kulkarni, and Arkady Yerukhimovich. Differentially-private multi-party sketching for large-scale statistics. Cryptology ePrint Archive, 2020. Google Scholar
  10. Charlie Dickens, Justin Thaler, and Daniel Ting. (nearly) all cardinality estimators are differentially private. arXiv preprint, 2022. URL: https://arxiv.org/abs/2203.15400.
  11. Marianne Durand and Philippe Flajolet. Loglog counting of large cardinalities. In Algorithms-ESA 2003: 11th Annual European Symposium, Budapest, Hungary, September 16-19, 2003. Proceedings 11, pages 605-617. Springer, 2003. Google Scholar
  12. Cynthia Dwork. Differential Privacy. In Michele Bugliesi, Bart Preneel, Vladimiro Sassone, and Ingo Wegener, editors, Automata, Languages and Programming, pages 1-12, Berlin, Heidelberg, 2006. Springer Berlin Heidelberg. Google Scholar
  13. Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference, pages 265-284. Springer, 2006. Google Scholar
  14. Talya Eden, Dana Ron, and C Seshadhri. On approximating the number of k-cliques in sublinear time. In Proceedings of the 50th annual ACM SIGACT symposium on theory of computing, pages 722-734, 2018. Google Scholar
  15. Alessandro Epasto, Jieming Mao, Andres Munoz Medina, Vahab Mirrokni, Sergei Vassilvitskii, and Peilin Zhong. Differentially private continual releases of streaming frequency moment estimations. arXiv preprint, 2023. URL: https://arxiv.org/abs/2301.05605.
  16. geetha290krm (https://math.stackexchange.com/users/1064504/geetha290krm). Does f_X+Y(z) = e[f_y(z-x)] hold? Mathematics Stack Exchange, 2022. URL:https://math.stackexchange.com/q/4544852 (version: 2022-10-04). URL: https://arxiv.org/abs/https://math.stackexchange.com/q/4544852.
  17. Badih Ghazi, Noah Golowich, Ravi Kumar, Rasmus Pagh, and Ameya Velingker. On the power of multiple anonymous messages: Frequency estimation and selection in the shuffle model of differential privacy. In Annual International Conference on the Theory and Applications of Cryptographic Techniques, pages 463-488. Springer, 2021. Google Scholar
  18. Ziyue Huang, Yuan Qiu, Ke Yi, and Graham Cormode. Frequency estimation under multiparty differential privacy: One-shot and streaming. arXiv preprint, 2021. URL: https://arxiv.org/abs/2104.01808.
  19. Haim Kaplan and Uri Stemmer. A note on sanitizing streams with differential privacy. arXiv preprint, 2021. URL: https://arxiv.org/abs/2111.13762.
  20. Zohar Karnin, Kevin Lang, and Edo Liberty. Optimal quantile approximation in streams. In 2016 ieee 57th annual symposium on foundations of computer science (focs), pages 71-78. IEEE, 2016. Google Scholar
  21. Kasper Green Larsen, Rasmus Pagh, and Jakub Tětek. Countsketches, feature hashing and the median of three. In International Conference on Machine Learning, pages 6011-6020. PMLR, 2021. Google Scholar
  22. Christian Janos Lebeda and Jakub Tětek. Better differentially private approximate histograms and heavy hitters using the misra-gries sketch. arXiv preprint, 2023. URL: https://arxiv.org/abs/2301.02457.
  23. Alexander J McNeil, Rüdiger Frey, and Paul Embrechts. Quantitative risk management: concepts, techniques and tools-revised edition. Princeton university press, 2015. Google Scholar
  24. Luca Melis, George Danezis, and Emiliano De Cristofaro. Efficient private statistics with succinct sketches. arXiv preprint, 2015. URL: https://arxiv.org/abs/1508.06110.
  25. Darakhshan Mir, Shan Muthukrishnan, Aleksandar Nikolov, and Rebecca N Wright. Pan-private algorithms via statistics on sketches. In Proceedings of the thirtieth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pages 37-48, 2011. Google Scholar
  26. Jayadev Misra and David Gries. Finding repeated elements. Science of computer programming, 2(2):143-152, 1982. Google Scholar
  27. Rasmus Pagh and Mikkel Thorup. Improved utility analysis of private countsketch. arXiv preprint, 2022. URL: https://arxiv.org/abs/2205.08397.
  28. Ryan M Rogers, Aaron Roth, Jonathan Ullman, and Salil Vadhan. Privacy odometers and filters: Pay-as-you-go composition. Advances in Neural Information Processing Systems, 29, 2016. Google Scholar
  29. Harry Sivasubramaniam, Haonan Li, and Xi He. Differentially private sublinear average degree approximation, 2020. Google Scholar
  30. Adam Smith, Shuang Song, and Abhradeep Guha Thakurta. The flajolet-martin sketch itself preserves differential privacy: Private counting with minimal space. Advances in Neural Information Processing Systems, 33:19561-19572, 2020. Google Scholar
  31. Nina Mesing Stausholm. Improved differentially private euclidean distance approximation. In Proceedings of the 40th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, pages 42-56, 2021. Google Scholar
  32. Roman Vershynin. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018. Google Scholar
  33. Lun Wang, Iosif Pinelis, and Dawn Song. Differentially private fractional frequency moments estimation with polylogarithmic space. arXiv preprint, 2021. URL: https://arxiv.org/abs/2105.12363.
  34. Yuichi Yoshida, Masaki Yamamoto, and Hiro Ito. An improved constant-time approximation algorithm for maximum~ matchings. In Proceedings of the forty-first annual ACM symposium on Theory of computing, pages 225-234, 2009. Google Scholar
  35. Fuheng Zhao, Dan Qiao, Rachel Redberg, Divyakant Agrawal, Amr El Abbadi, and Yu-Xiang Wang. Differentially private linear sketches: Efficient implementations and applications. arXiv preprint, 2022. URL: https://arxiv.org/abs/2205.09873.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact