Document Open Access Logo

Generative Models of Huge Objects

Authors Lunjia Hu, Inbal Rachel Livni Navon, Omer Reingold

PDF
Thumbnail PDF

File

LIPIcs.CCC.2023.5.pdf
  • Filesize: 0.66 MB
  • 20 pages

Document Identifiers

Author Details

Lunjia Hu
  • Department of Computer Science, Stanford University, CA, USA
Inbal Rachel Livni Navon
  • Department of Computer Science, Stanford University, CA, USA
Omer Reingold
  • Department of Computer Science, Stanford University, CA, USA

Funding

  • Hu, Lunjia: Supported by the Simons Foundation Collaboration on the Theory of Algorithmic Fairness, Omer Reingold’s NSF Award IIS-1908774, and Moses Charikar’s Simons Investigators award.
  • Livni Navon, Inbal Rachel: Supported by the Simons Foundation Collaboration on the Theory of Algorithmic Fairness, the Sloan Foundation Grant 2020-13941, and the Zuckerman STEM Leadership Program.
  • Reingold, Omer: Supported by the Simons Foundation Collaboration on the Theory of Algorithmic Fairness and the Simons Foundation Investigators award 689988.

Cite As Get BibTex

Lunjia Hu, Inbal Rachel Livni Navon, and Omer Reingold. Generative Models of Huge Objects. In 38th Computational Complexity Conference (CCC 2023). Leibniz International Proceedings in Informatics (LIPIcs), Volume 264, pp. 5:1-5:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2023) https://doi.org/10.4230/LIPIcs.CCC.2023.5

Abstract

This work initiates the systematic study of explicit distributions that are indistinguishable from a single exponential-size combinatorial object. In this we extend the work of Goldreich, Goldwasser and Nussboim (SICOMP 2010) that focused on the implementation of huge objects that are indistinguishable from the uniform distribution, satisfying some global properties (which they coined truthfulness). Indistinguishability from a single object is motivated by the study of generative models in learning theory and regularity lemmas in graph theory. Problems that are well understood in the setting of pseudorandomness present significant challenges and at times are impossible when considering generative models of huge objects. 
We demonstrate the versatility of this study by providing a learning algorithm for huge indistinguishable objects in several natural settings including: dense functions and graphs with a truthfulness requirement on the number of ones in the function or edges in the graphs, and a version of the weak regularity lemma for sparse graphs that satisfy some global properties. These and other results generalize basic pseudorandom objects as well as notions introduced in algorithmic fairness. The results rely on notions and techniques from a variety of areas including learning theory, complexity theory, cryptography, and game theory.

Subject Classification

ACM Subject Classification
  • Theory of computation → Pseudorandomness and derandomization
  • Theory of computation → Random network models
  • Theory of computation → Generating random combinatorial structures
Keywords
  • pseudorandomness
  • generative models
  • regularity lemma

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Related Versions

References

  1. Edoardo M Airoldi, David M Blei, Stephen E Fienberg, and Eric P Xing. Mixed membership stochastic blockmodels. Journal of machine learning research: JMLR, 9:1981-2014, 2008. Google Scholar
  2. Sanjeev Arora, Rong Ge, Yingyu Liang, Tengyu Ma, and Yi Zhang. Generalization and equilibrium in generative adversarial nets (GANs). In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 224-232. PMLR, 06-11 August 2017. URL: https://proceedings.mlr.press/v70/arora17a.html.
  3. Manuel Blum and Silvio Micali. How to generate cryptographically strong sequences of pseudorandom bits. SIAM Journal on Computing, 13(4):850-864, 1984. URL: https://doi.org/10.1137/0213053.
  4. Yi-Hsiu Chen, Kai-Min Chung, and Jyun-Jie Liao. On the complexity of simulating auxiliary input. In Advances in cryptology - EUROCRYPT 2018. Part III, volume 10822 of Lecture Notes in Comput. Sci., pages 371-390. Springer, Cham, 2018. URL: https://doi.org/10.1007/978-3-319-78372-7_12.
  5. Cynthia Dwork, Michael P Kim, Omer Reingold, Guy N Rothblum, and Gal Yona. Outcome indistinguishability. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, pages 1095-1108, 2021. Google Scholar
  6. Cynthia Dwork, Daniel Lee, Huijia Lin, and Pranay Tankala. New insights into multi-calibration. arXiv preprint, 2023. URL: https://arxiv.org/abs/2301.08837.
  7. Alan Frieze and Ravi Kannan. Quick approximation to matrices and applications. Combinatorica, 19(2):175-220, 1999. Google Scholar
  8. Oded Goldreich, Shafi Goldwasser, and Silvio Micali. How to construct random functions. Journal of the ACM, 33(4):792-807, October 1986. Google Scholar
  9. Oded Goldreich, Shafi Goldwasser, and Asaf Nussboim. On the implementation of huge random objects. SIAM Journal on Computing, 39(7):2761-2822, 2010. Google Scholar
  10. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014. URL: https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf.
  11. Parikshit Gopalan, Lunjia Hu, Michael P. Kim, Omer Reingold, and Udi Wieder. Loss Minimization Through the Lens Of Outcome Indistinguishability. In Yael Tauman Kalai, editor, 14th Innovations in Theoretical Computer Science Conference (ITCS 2023), volume 251 of Leibniz International Proceedings in Informatics (LIPIcs), pages 60:1-60:20, Dagstuhl, Germany, 2023. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.ITCS.2023.60.
  12. Parikshit Gopalan, Adam Tauman Kalai, Omer Reingold, Vatsal Sharan, and Udi Wieder. Omnipredictors. In Mark Braverman, editor, 13th Innovations in Theoretical Computer Science Conference, ITCS 2022, January 31 - February 3, 2022, Berkeley, CA, USA, volume 215 of LIPIcs, pages 79:1-79:21. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022. URL: https://doi.org/10.4230/LIPIcs.ITCS.2022.79.
  13. Parikshit Gopalan, Michael P Kim, Mihir A Singhal, and Shengjia Zhao. Low-degree multicalibration. In Po-Ling Loh and Maxim Raginsky, editors, Proceedings of Thirty Fifth Conference on Learning Theory, volume 178 of Proceedings of Machine Learning Research, pages 3193-3234. PMLR, 02-05 July 2022. URL: https://proceedings.mlr.press/v178/gopalan22a.html.
  14. Ursula Hébert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. Multicalibration: Calibration for the (computationally-identifiable) masses. In International Conference on Machine Learning, pages 1939-1948. PMLR, 2018. Google Scholar
  15. Paul W. Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt. Stochastic blockmodels: First steps. Social Networks, 5(2):109-137, 1983. URL: https://doi.org/10.1016/0378-8733(83)90021-7.
  16. Shlomo Hoory, Nathan Linial, and Avi Wigderson. Expander graphs and their applications. Bulletin of the AMS, 43(4):439-561, 2006. Google Scholar
  17. Lunjia Hu, Inbal Livni-Navon, and Omer Reingold. Generative models of huge objects, 2023. URL: https://arxiv.org/abs/2302.12823.
  18. Lunjia Hu, Inbal Livni-Navon, Omer Reingold, and Chutong Yang. Omnipredictors for constrained optimization. arXiv preprint, 2022. URL: https://arxiv.org/abs/2209.07463.
  19. Lunjia Hu and Charlotte Peale. Comparative Learning: A Sample Complexity Theory for Two Hypothesis Classes. In Yael Tauman Kalai, editor, 14th Innovations in Theoretical Computer Science Conference (ITCS 2023), volume 251 of Leibniz International Proceedings in Informatics (LIPIcs), pages 72:1-72:30, Dagstuhl, Germany, 2023. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.ITCS.2023.72.
  20. Lunjia Hu, Charlotte Peale, and Omer Reingold. Metric entropy duality and the sample complexity of outcome indistinguishability. In Sanjoy Dasgupta and Nika Haghtalab, editors, Proceedings of The 33rd International Conference on Algorithmic Learning Theory, volume 167 of Proceedings of Machine Learning Research, pages 515-552. PMLR, 29 March-01 April 2022. URL: https://proceedings.mlr.press/v167/hu22a.html.
  21. Russell Impagliazzo. Lecture on learning models: connections between boosting, hard-core distributions, dense models, GAN, and regularity I. https://www.ias.edu/video/csdm/2017/1113-RussellImpagliazzo, 2017.
  22. Dimitar Jetchev and Krzysztof Pietrzak. How to fake auxiliary input. In Theory of cryptography, volume 8349 of Lecture Notes in Comput. Sci., pages 566-590. Springer, Heidelberg, 2014. URL: https://doi.org/10.1007/978-3-642-54242-8_24.
  23. Michael P Kim, Amirata Ghorbani, and James Zou. Multiaccuracy: Black-box post-processing for fairness in classification. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 247-254, 2019. Google Scholar
  24. Michael P. Kim, Christoph Kern, Shafi Goldwasser, Frauke Kreuter, and Omer Reingold. Universal adaptability: Target-independent inference that competes with propensity scoring. Proceedings of the National Academy of Sciences, 119(4):e2108097119, 2022. URL: https://doi.org/10.1073/pnas.2108097119.
  25. Michael P. Kim and Juan C. Perdomo. Making Decisions Under Outcome Performativity. In Yael Tauman Kalai, editor, 14th Innovations in Theoretical Computer Science Conference (ITCS 2023), volume 251 of Leibniz International Proceedings in Informatics (LIPIcs), pages 79:1-79:15, Dagstuhl, Germany, 2023. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.ITCS.2023.79.
  26. Yoshiharu Kohayakawa and Vojtech Rödl. Szemerédi’s regularity lemma and quasi-randomness. Recent advances in algorithms and combinatorics, pages 289-351, 2003. Google Scholar
  27. Michael Luby and Charles Rackoff. How to construct pseudorandom permutations from pseudorandom functions. SIAM Journal on Computing, 17(2):373-386, 1988. Google Scholar
  28. Michael Mitzenmacher. A Brief History of Generative Models for Power Law and Lognormal Distributions. Internet Mathematics, 1(2):226-251, 2003. Google Scholar
  29. Joseph Naor and Moni Naor. Small-bias probability spaces: Efficient constructions and applications. SIAM Journal on Computing, 22(4):838-856, August 1993. Google Scholar
  30. Moni Naor and Asaf Nussboim. Implementing huge sparse random graphs. In Moses Charikar, Klaus Jansen, Omer Reingold, and José D. P. Rolim, editors, Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques, 10th International Workshop, APPROX 2007, and 11th International Workshop, RANDOM 2007, Princeton, NJ, USA, August 20-22, 2007, Proceedings, volume 4627 of Lecture Notes in Computer Science, pages 596-608. Springer, 2007. URL: https://doi.org/10.1007/978-3-540-74208-1_43.
  31. Moni Naor, Asaf Nussboim, and Eran Tromer. Efficiently constructible huge graphs that preserve first order properties of random graphs. In Joe Kilian, editor, Theory of Cryptography, Second Theory of Cryptography Conference, TCC 2005, Cambridge, MA, USA, February 10-12, 2005, Proceedings, volume 3378 of Lecture Notes in Computer Science, pages 66-85. Springer, 2005. URL: https://doi.org/10.1007/978-3-540-30576-7_5.
  32. Moni Naor and Omer Reingold. Constructing pseudo-random permutations with a prescribed structure. J. Cryptol., 15(2):97-102, January 2002. URL: https://doi.org/10.1007/s00145-001-0008-5.
  33. Alexander Scott. Szemerédi’s regularity lemma for matrices and sparse graphs. Combinatorics, Probability and Computing, 20(3):455-466, 2011. Google Scholar
  34. Maciej Skórski. Simulating auxiliary inputs, revisited. In Theory of cryptography. Part I, volume 9985 of Lecture Notes in Comput. Sci., pages 159-179. Springer, Berlin, 2016. URL: https://doi.org/10.1007/978-3-662-53641-4_7.
  35. Maciej Skórski. A subgradient algorithm for computational distances and applications to cryptography. Cryptology ePrint Archive, 2016. Google Scholar
  36. Maciej Skórski. A cryptographic view of regularity lemmas: simpler unified proofs and refined bounds. In Theory and applications of models of computation, volume 10185 of Lecture Notes in Comput. Sci., pages 586-599. Springer, Cham, 2017. URL: https://doi.org/10.1007/978-3-319-55911-7.
  37. Luca Trevisan, Madhur Tulsiani, and Salil P. Vadhan. Regularity, boosting, and efficiently simulating every high-entropy distribution. In Proceedings of the 24th Annual IEEE Conference on Computational Complexity, CCC 2009, Paris, France, 15-18 July 2009, pages 126-136. IEEE Computer Society, 2009. URL: https://doi.org/10.1109/CCC.2009.41.
  38. Salil Vadhan and Colin Jia Zheng. A uniform min-max theorem with applications in cryptography. In Advances in Cryptology-CRYPTO 2013: 33rd Annual Cryptology Conference, Santa Barbara, CA, USA, August 18-22, 2013. Proceedings, Part I, pages 93-110. Springer, 2013. Google Scholar
  39. Salil P. Vadhan. Pseudorandomness. Foundations and Trends® in Theoretical Computer Science, 7(1–3):1-336, 2012. URL: https://doi.org/10.1561/0400000010.
  40. Mark N. Wegman and J. Lawrence Carter. New hash functions and their use in authentication and set equality. Journal of Computer and System Sciences, 1981. Google Scholar
  41. Andrew C. Yao. Theory and applications of trapdoor functions. In 23rd annual symposium on foundations of computer science (Chicago, Ill., 1982), pages 80-91. IEEE, New York, 1982. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact