Document Open Access Logo

Private Estimation When Data and Privacy Demands Are Correlated

Authors Syomantak Chaudhuri, Thomas A. Courtade

PDF

File

Thumbnail PDF
PDF
LIPIcs.FORC.2025.3.pdf
  • Filesize: 0.89 MB
  • 20 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Security and privacy
Keywords
  • Differential Privacy
  • Personalized Privacy
  • Heterogeneous Privacy
  • Correlations in Privacy

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

Differential Privacy (DP) is the current gold-standard for ensuring privacy for statistical queries. Estimation problems under DP constraints appearing in the literature have largely focused on providing equal privacy to all users. We consider the problems of empirical mean estimation for univariate data and frequency estimation for categorical data, both subject to heterogeneous privacy constraints. Each user, contributing a sample to the dataset, is allowed to have a different privacy demand. The dataset itself is assumed to be worst-case and we study both problems under two different formulations - first, where privacy demands and data may be correlated, and second, where correlations are weakened by random permutation of the dataset. We establish theoretical performance guarantees for our proposed algorithms, under both PAC error and mean-squared error. These performance guarantees translate to minimax optimality in several instances, and experiments confirm superior performance of our algorithms over other baseline techniques.

Cite As Get BibTex

Syomantak Chaudhuri and Thomas A. Courtade. Private Estimation When Data and Privacy Demands Are Correlated. In 6th Symposium on Foundations of Responsible Computing (FORC 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 329, pp. 3:1-3:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.FORC.2025.3

Author Details

Syomantak Chaudhuri
  • UC Berkeley, CA, USA
Thomas A. Courtade
  • UC Berkeley, CA, USA

Funding

Authors were supported from NSF CCF-1750430 and CCF-2007669
  • Chaudhuri, Syomantak: Supported by AI Policy Hub, U.C. Berkeley.

Supplementary Materials

References

  1. John M. Abowd. The u.s. census bureau adopts differential privacy. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Association for Computing Machinery, 2018. URL: https://doi.org/10.1145/3219819.3226070.
  2. Krishna Acharya, Franziska Boenisch, Rakshit Naidu, and Juba Ziani. Personalized differential privacy for ridge regression. arXiv preprint arXiv:2401.17127, 2024. URL: https://doi.org/10.48550/arXiv.2401.17127.
  3. Mark S. Ackerman, Lorrie Faith Cranor, and Joseph Reagle. Privacy in e-commerce: examining user scenarios and privacy preferences. In Proceedings of the 1st ACM Conference on Electronic Commerce, EC '99, pages 1-8. Association for Computing Machinery, 1999. URL: https://doi.org/10.1145/336992.336995.
  4. Mohammad Alaggan, Sébastien Gambs, and Anne-Marie Kermarrec. Heterogeneous differential privacy. Journal of Privacy and Confidentiality, 2017. URL: https://doi.org/10.29012/jpc.v7i2.652.
  5. Noga Alon, Raef Bassily, and Shay Moran. Limits of private learning with access to public data. Advances in neural information processing systems, 2019. URL: https://doi.org/10.48550/arXiv.1910.11519.
  6. Ehsan Amid, Arun Ganesh, Rajiv Mathews, Swaroop Ramaswamy, Shuang Song, Thomas Steinke, Vinith M Suriyakumar, Om Thakkar, and Abhradeep Thakurta. Public data-assisted mirror descent for private model training. In International Conference on Machine Learning. PMLR, 2022. URL: https://proceedings.mlr.press/v162/amid22a.html.
  7. Ameya Anjarlekar, S. Rasoul Etesami, and R. Srikant. Striking a balance: An optimal mechanism design for heterogenous differentially private data acquisition for logistic regression. CoRR, abs/2309.10340, 2023. URL: https://doi.org/10.48550/arXiv.2309.10340.
  8. Apple. Learning with privacy at scale differential, 2017. URL: https://api.semanticscholar.org/CorpusID:43986173.
  9. Raef Bassily, Albert Cheu, Shay Moran, Aleksandar Nikolov, Jonathan Ullman, and Steven Wu. Private query release assisted by public data. In International Conference on Machine Learning. PMLR, 2020. URL: http://proceedings.mlr.press/v119/bassily20a.html.
  10. Raef Bassily, Shay Moran, and Anupama Nandi. Learning from mixtures of private and public populations. Advances in Neural Information Processing Systems, 2020. URL: https://doi.org/10.48550/arXiv.2008.00331.
  11. Kornelia M. Batko and Andrzej Slezak. The use of big data analytics in healthcare. J. Big Data, 9(1):3, 2022. URL: https://doi.org/10.1186/s40537-021-00553-4.
  12. Alex Bie, Gautam Kamath, and Vikrant Singhal. Private estimation with public data. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022. URL: http://papers.nips.cc/paper_files/paper/2022/hash/765ec49952dd0140ac754d6d3f9bc899-Abstract-Conference.html.
  13. Sourav Biswas, Yihe Dong, Gautam Kamath, and Jonathan Ullman. Coinpress: Practical private mean and covariance estimation. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, volume 33, pages 14475-14485, 2020. URL: https://proceedings.neurips.cc/paper/2020/hash/a684eceee76fc522773286a895bc8436-Abstract.html.
  14. CA State Controller. University of california salary data, 2022. Accessed: 2024-01-22. URL: https://publicpay.ca.gov/Reports/RawExport.aspx.
  15. Clément L. Canonne and Yucheng Sun. Private distribution testing with heterogeneous constraints: Your epsilon might not be mine. CoRR, abs/2309.06068, 2023. URL: https://doi.org/10.48550/arXiv.2309.06068.
  16. Syomantak Chaudhuri and Thomas A. Courtade. Mean estimation under heterogeneous privacy: Some privacy can be free. In IEEE International Symposium on Information Theory, ISIT 2023, Taipei, Taiwan, June 25-30, 2023, pages 1639-1644. IEEE, 2023. URL: https://doi.org/10.1109/ISIT54713.2023.10206746.
  17. Syomantak Chaudhuri and Thomas A. Courtade. Private estimation when data and privacy demands are correlated, 2024. URL: https://doi.org/10.48550/arXiv.2407.11274.
  18. Syomantak Chaudhuri, Konstantin Miagkov, and Thomas A. Courtade. Mean estimation under heterogeneous privacy demands. IEEE Trans. Inf. Theory, 71(2):1362-1375, 2025. URL: https://doi.org/10.1109/TIT.2024.3511498.
  19. Rachel Cummings, Hadi Elzayn, Emmanouil Pountourakis, Vasilis Gkatzelis, and Juba Ziani. Optimal data acquisition with privacy-aware agents. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning, SaTML 2023, Raleigh, NC, USA, February 8-10, 2023, pages 210-224. IEEE, IEEE, 2023. URL: https://doi.org/10.1109/SaTML54575.2023.00023.
  20. Steven Diamond and Stephen Boyd. CVXPY: A Python-embedded modeling language for convex optimization. Journal of Machine Learning Research, 2016. URL: https://jmlr.org/papers/v17/15-408.html.
  21. John C. Duchi, Martin J. Wainwright, and Michael I. Jordan. Local privacy and minimax bounds: Sharp rates for probability estimation. In Christopher J. C. Burges, Léon Bottou, Zoubin Ghahramani, and Kilian Q. Weinberger, editors, Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, pages 1529-1537, 2013. URL: https://proceedings.neurips.cc/paper/2013/hash/5807a685d1a9ab3b599035bc566ce2b9-Abstract.html.
  22. Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor. Our data, ourselves: Privacy via distributed noise generation. In Serge Vaudenay, editor, Advances in Cryptology - EUROCRYPT 2006, 25th Annual International Conference on the Theory and Applications of Cryptographic Techniques, St. Petersburg, Russia, May 28 - June 1, 2006, Proceedings, volume 4004 of Lecture Notes in Computer Science, pages 486-503. Springer, Springer, 2006. URL: https://doi.org/10.1007/11761679_29.
  23. Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam D. Smith. Calibrating noise to sensitivity in private data analysis. In Shai Halevi and Tal Rabin, editors, Theory of Cryptography, Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006, Proceedings, volume 3876 of Lecture Notes in Computer Science, pages 265-284. Springer, Springer, 2006. URL: https://doi.org/10.1007/11681878_14.
  24. Úlfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova. RAPPOR: randomized aggregatable privacy-preserving ordinal response. In Gail-Joon Ahn, Moti Yung, and Ninghui Li, editors, Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, Scottsdale, AZ, USA, November 3-7, 2014, pages 1054-1067, Scottsdale, Arizona, 2014. ACM. URL: https://doi.org/10.1145/2660267.2660348.
  25. Alireza Fallah, Ali Makhdoumi, Azarakhsh Malekian, and Asuman E. Ozdaglar. Optimal and differentially private data acquisition: Central and local mechanisms. Oper. Res., 72(3):1105-1123, 2024. URL: https://doi.org/10.1287/opre.2022.0014.
  26. Cecilia Ferrando, Jennifer Gillenwater, and Alex Kulesza. Combining public and private data. CoRR, abs/2111.00115, 2021. URL: https://doi.org/10.48550/arXiv.2111.00115.
  27. Arpita Ghosh and Aaron Roth. Selling privacy at auction. In Yoav Shoham, Yan Chen, and Tim Roughgarden, editors, Proceedings 12th ACM Conference on Electronic Commerce (EC-2011), San Jose, CA, USA, June 5-9, 2011, pages 199-208. ACM, 2011. URL: https://doi.org/10.1145/1993574.1993605.
  28. Zach Jorgensen, Ting Yu, and Graham Cormode. Conservative or liberal? personalized differential privacy. In Johannes Gehrke, Wolfgang Lehner, Kyuseok Shim, Sang Kyun Cha, and Guy M. Lohman, editors, 31st IEEE International Conference on Data Engineering, ICDE 2015, Seoul, South Korea, April 13-17, 2015, pages 1023-1034. IEEE Computer Society, 2015. URL: https://doi.org/10.1109/ICDE.2015.7113353.
  29. Peter Kairouz, Kallista A. Bonawitz, and Daniel Ramage. Discrete distribution estimation under local privacy. In Maria-Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, pages 2436-2444. PMLR, JMLR.org, 2016. URL: http://proceedings.mlr.press/v48/kairouz16.html.
  30. Peter Kairouz, Mónica Ribero Diaz, Keith Rush, and Abhradeep Thakurta. (nearly) dimension independent private ERM with adagrad ratesvia publicly estimated subspaces. In Mikhail Belkin and Samory Kpotufe, editors, Conference on Learning Theory, COLT 2021, 15-19 August 2021, Boulder, Colorado, USA, volume 134 of Proceedings of Machine Learning Research, pages 2717-2746. PMLR, PMLR, 2021. URL: http://proceedings.mlr.press/v134/kairouz21a.html.
  31. Justin Singh Kang, Ramtin Pedarsani, and Kannan Ramchandran. The fair value of data under heterogeneous privacy constraints in federated learning. Trans. Mach. Learn. Res., 2024, 2024. URL: https://openreview.net/forum?id=ynG5Ak7n7Q.
  32. Shiva Prasad Kasiviswanathan, Homin K. Lee, Kobbi Nissim, Sofya Raskhodnikova, and Adam D. Smith. What can we learn privately? SIAM J. Comput., 40(3):793-826, 2011. URL: https://doi.org/10.1137/090756090.
  33. Hong Ping Li, Haibo Hu, and Jianliang Xu. Nearby friend alert: Location anonymity in mobile geosocial networks. IEEE Pervasive Comput., 12(4):62-70, 2013. URL: https://doi.org/10.1109/MPRV.2012.82.
  34. Terrance Liu, Giuseppe Vietri, Thomas Steinke, Jonathan R. Ullman, and Zhiwei Steven Wu. Leveraging public data for practical private query release. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 6968-6977. PMLR, PMLR, 2021. URL: http://proceedings.mlr.press/v139/liu21w.html.
  35. Harry Markowitz. Portfolio selection. The Journal of Finance, 7(1):77-91, 1952. URL: http://www.jstor.org/stable/2975974.
  36. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Aarti Singh and Xiaojin (Jerry) Zhu, editors, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA, volume 54 of Proceedings of Machine Learning Research, pages 1273-1282. PMLR, PMLR, 2017. URL: http://proceedings.mlr.press/v54/mcmahan17a.html.
  37. Anupama Nandi and Raef Bassily. Privately answering classification queries in the agnostic PAC model. In Aryeh Kontorovich and Gergely Neu, editors, Algorithmic Learning Theory, ALT 2020, 8-11 February 2020, San Diego, CA, USA, volume 117 of Proceedings of Machine Learning Research, pages 687-703. PMLR, PMLR, 2020. URL: http://proceedings.mlr.press/v117/nandi20a.html.
  38. Koustuv Saha, Yozen Liu, Nicholas Vincent, Farhan Asif Chowdhury, Leonardo Neves, Neil Shah, and Maarten W. Bos. Advertiming matters: Examining user ad consumption for effective ad allocations on social media. In Yoshifumi Kitamura, Aaron Quigley, Katherine Isbister, Takeo Igarashi, Pernille Bjørn, and Steven Mark Drucker, editors, CHI '21: CHI Conference on Human Factors in Computing Systems, Virtual Event / Yokohama, Japan, May 8-13, 2021, pages 581:1-581:18. ACM, 2021. URL: https://doi.org/10.1145/3411764.3445394.
  39. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, and National Cancer Institute. U.s. cancer statistics data visualizations tool, based on 2022 submission data (1999-2020), 2020. Accessed: 2024-01-22. URL: https://gis.cdc.gov/Cancer/USCS/#/AtAGlance/.
  40. Salil P. Vadhan. The complexity of differential privacy. In Yehuda Lindell, editor, Tutorials on the Foundations of Cryptography, pages 347-450. Springer International Publishing, 2017. URL: https://doi.org/10.1007/978-3-319-57048-8_7.
  41. Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint. Cambridge university press, 2019. Google Scholar
  42. Di Wang, Huangyu Zhang, Marco Gaboardi, and Jinhui Xu. Estimating smooth GLM in non-interactive local differential privacy model with public unlabeled data. In Vitaly Feldman, Katrina Ligett, and Sivan Sabato, editors, Algorithmic Learning Theory, 16-19 March 2021, Virtual Conference, Worldwide, volume 132 of Proceedings of Machine Learning Research, pages 1207-1213. PMLR, PMLR, 2021. URL: http://proceedings.mlr.press/v132/wang21a.html.
  43. Teng Wang, Xuefeng Zhang, Jingyu Feng, and Xinyu Yang. A comprehensive survey on local differential privacy toward data statistics and analysis. Sensors, 20(24):7030, 2020. URL: https://doi.org/10.3390/s20247030.
  44. Tianhao Wang, Jeremiah Blocki, Ninghui Li, and Somesh Jha. Locally differentially private protocols for frequency estimation. In Engin Kirda and Thomas Ristenpart, editors, 26th USENIX Security Symposium, USENIX Security 2017, Vancouver, BC, Canada, August 16-18, 2017, pages 729-745. USENIX Association, 2017. URL: https://www.usenix.org/conference/usenixsecurity17/technical-sessions/presentation/wang-tianhao.
  45. Ian Waudby-Smith, Zhiwei Steven Wu, and Aaditya Ramdas. Nonparametric extensions of randomized response for private confidence sets. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 36748-36789. PMLR, 23-29 July 2023. URL: https://proceedings.mlr.press/v202/waudby-smith23a.html.
  46. Tong Yang, Jie Jiang, Peng Liu, Qun Huang, Junzhi Gong, Yang Zhou, Rui Miao, Xiaoming Li, and Steve Uhlig. Elastic sketch: adaptive and fast network-wide measurements. In Sergey Gorinsky and János Tapolcai, editors, Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication, SIGCOMM 2018, Budapest, Hungary, August 20-25, 2018, pages 561-575. ACM, 2018. URL: https://doi.org/10.1145/3230543.3230544.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact