Document Open Access Logo

Pseudo-Derandomizing Learning and Approximation

Authors Igor Carboni Oliveira, Rahul Santhanam

PDF

File

Thumbnail PDF
PDF
LIPIcs.APPROX-RANDOM.2018.55.pdf
  • Filesize: 0.5 MB
  • 19 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Pseudorandomness and derandomization
Keywords
  • derandomization
  • learning
  • approximation
  • boolean circuits

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

We continue the study of pseudo-deterministic algorithms initiated by Gat and Goldwasser [Eran Gat and Shafi Goldwasser, 2011]. A pseudo-deterministic algorithm is a probabilistic algorithm which produces a fixed output with high probability. We explore pseudo-determinism in the settings of learning and approximation. Our goal is to simulate known randomized algorithms in these settings by pseudo-deterministic algorithms in a generic fashion - a goal we succinctly term pseudo-derandomization. Learning. In the setting of learning with membership queries, we first show that randomized learning algorithms can be derandomized (resp. pseudo-derandomized) under the standard hardness assumption that E (resp. BPE) requires large Boolean circuits. Thus, despite the fact that learning is an algorithmic task that requires interaction with an oracle, standard hardness assumptions suffice to (pseudo-)derandomize it. We also unconditionally pseudo-derandomize any {quasi-polynomial} time learning algorithm for polynomial size circuits on infinitely many input lengths in sub-exponential time.
Next, we establish a generic connection between learning and derandomization in the reverse direction, by showing that deterministic (resp. pseudo-deterministic) learning algorithms for a concept class C imply hitting sets against C that are computable deterministically (resp. pseudo-deterministically). In particular, this suggests a new approach to constructing hitting set generators against AC^0[p] circuits by giving a deterministic learning algorithm for AC^0[p]. Approximation. Turning to approximation, we unconditionally pseudo-derandomize any poly-time randomized approximation scheme for integer-valued functions infinitely often in subexponential time over any samplable distribution on inputs. As a corollary, we get that the (0,1)-Permanent has a fully pseudo-deterministic approximation scheme running in sub-exponential time infinitely often over any samplable distribution on inputs.
Finally, we {investigate} the notion of approximate canonization of Boolean circuits. We use a connection between pseudodeterministic learning and approximate canonization to show that if BPE does not have sub-exponential size circuits infinitely often, then there is a pseudo-deterministic approximate canonizer for AC^0[p] computable in quasi-polynomial time.

Cite As Get BibTex

Igor Carboni Oliveira and Rahul Santhanam. Pseudo-Derandomizing Learning and Approximation. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2018). Leibniz International Proceedings in Informatics (LIPIcs), Volume 116, pp. 55:1-55:19, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2018) https://doi.org/10.4230/LIPIcs.APPROX-RANDOM.2018.55

Author Details

Igor Carboni Oliveira
  • Department of Computer Science, University of Oxford, United Kingdom.
Rahul Santhanam
  • Department of Computer Science, University of Oxford, United Kingdom.

Funding

This work was supported by the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007-2013)/ERC Grant No. 615075.

References

  1. Boaz Barak, Oded Goldreich, Russell Impagliazzo, Steven Rudich, Amit Sahai, Salil P. Vadhan, and Ke Yang. On the (im)possibility of obfuscating programs. Journal of the ACM, 59(2):6:1-6:48, 2012. Google Scholar
  2. Dan Boneh and Richard J. Lipton. Amplification of weak learning under the uniform distribution. In Conference on Computational Learning Theory (COLT), pages 347-351, 1993. Google Scholar
  3. Zvika Brakerski, Christina Brzuska, and Nils Fleischhacker. On statistically secure obfuscation with approximate correctness. In International Cryptology Conference (CRYPTO), pages 551-578, 2016. Google Scholar
  4. Marco L. Carmosino, Russell Impagliazzo, Valentine Kabanets, and Antonina Kolokolova. Learning algorithms from natural proofs. In Conference on Computational Complexity (CCC), pages 10:1-10:24, 2016. Google Scholar
  5. Bill Fefferman, Ronen Shaltiel, Christopher Umans, and Emanuele Viola. On beating the hybrid argument. Theory of Computing, 9:809-843, 2013. Google Scholar
  6. Lance Fortnow and Adam R. Klivans. Efficient learning algorithms yield circuit lower bounds. Journal of Computer and System Sciences, 75(1):27-36, 2009. URL: http://dx.doi.org/10.1016/j.jcss.2008.07.006.
  7. Eran Gat and Shafi Goldwasser. Probabilistic search algorithms with unique answers and their cryptographic applications. Electronic Colloquium on Computational Complexity (ECCC), 18:136, 2011. Google Scholar
  8. Oded Goldreich, Noam Nisan, and Avi Wigderson. On Yao’s XOR lemma. In Studies in Complexity and Cryptography, pages 273-301. Springer, 2011. URL: http://dx.doi.org/10.1007/978-3-642-22670-0_23.
  9. Shafi Goldwasser and Ofer Grossman. Bipartite perfect matching in pseudo-deterministic NC. In International Colloquium on Automata, Languages, and Programming (ICALP), pages 87:1-87:13, 2017. Google Scholar
  10. Shafi Goldwasser, Ofer Grossman, and Dhiraj Holden. Pseudo-deterministic proofs. Electronic Colloquium on Computational Complexity (ECCC), 24:105, 2017. Google Scholar
  11. Shafi Goldwasser, Dan Gutfreund, Alexander Healy, Tali Kaufman, and Guy N. Rothblum. Verifying and decoding in constant depth. In Symposium on Theory of Computing (STOC), pages 440-449, 2007. Google Scholar
  12. Parikshit Gopalan and Rocco A. Servedio. Learning and lower bounds for AC⁰ with threshold gates. In International Workshop on Approximation, Randomization, and Combinatorial Optimization (RANDOM-APPROX), pages 588-601, 2010. Google Scholar
  13. Ofer Grossman. Finding primitive roots pseudo-deterministically. Electronic Colloquium on Computational Complexity (ECCC), 22:207, 2015. Google Scholar
  14. Dan Gutfreund and Guy N. Rothblum. The complexity of local list decoding. In International Workshop on Approximation, Randomization and Combinatorial Optimization (RANDOM-APPROX), pages 455-468, 2008. Google Scholar
  15. Ryan C. Harkins and John M. Hitchcock. Exact learning algorithms, betting games, and circuit lower bounds. Transactions on Computation Theory (TOCT), 5(4):18:1-18:11, 2013. URL: http://dx.doi.org/10.1145/2539126.2539130.
  16. Dhiraj Holden. A note on unconditional subexponential-time pseudo-deterministic algorithms for BPP search problems. arXiv, 2017. URL: http://arxiv.org/abs/1707.05808.
  17. Russell Impagliazzo, Valentine Kabanets, and Avi Wigderson. In search of an easy witness: exponential time vs. probabilistic polynomial time. Journal of Computer and System Sciences, 65(4):672-694, 2002. URL: http://dx.doi.org/10.1016/S0022-0000(02)00024-7.
  18. Russell Impagliazzo and Avi Wigderson. P = BPP if E requires exponential circuits: Derandomizing the XOR lemma. In Symposium on the Theory of Computing (STOC), pages 220-229, 1997. URL: http://dx.doi.org/10.1145/258533.258590.
  19. Jeffrey C. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. Journal of Computer and System Sciences, 55(3):414-440, 1997. Google Scholar
  20. Mark Jerrum, Alistair Sinclair, and Eric Vigoda. A polynomial-time approximation algorithm for the permanent of a matrix with nonnegative entries. Journal of the ACM, 51(4):671-697, 2004. Google Scholar
  21. Valentine Kabanets and Russell Impagliazzo. Derandomizing polynomial identity tests means proving circuit lower bounds. Computational Complexity, 13(1-2):1-46, 2004. URL: http://dx.doi.org/10.1007/s00037-004-0182-6.
  22. Michael Kearns and Umesh Vazirani. An Introduction to Computational Learning Theory. MIT Press, 1994. Google Scholar
  23. Adam Klivans, Pravesh Kothari, and Igor C. Oliveira. Constructing hard functions using learning algorithms. In Conference on Computational Complexity (CCC), pages 86-97, 2013. Google Scholar
  24. Nathan Linial, Yishay Mansour, and Noam Nisan. Constant depth circuits, fourier transform, and learnability. Journal of the ACM, 40(3):607-620, 1993. Google Scholar
  25. Moni Naor and Omer Reingold. Number-theoretic constructions of efficient pseudo-random functions. Journal of the ACM, 51(2):231-262, 2004. Google Scholar
  26. Noam Nisan and Avi Wigderson. Hardness vs randomness. Journal of Computer and System Sciences, 49(2):149-167, 1994. Google Scholar
  27. Ryan O'Donnell. Analysis of Boolean Functions. Cambridge University Press, 2014. Google Scholar
  28. Igor C. Oliveira and Rahul Santhanam. Conspiracies between learning algorithms, circuit lower bounds, and pseudorandomness. In Computational Complexity Conference (CCC), pages 18:1-18:49, 2017. Google Scholar
  29. Igor C. Oliveira and Rahul Santhanam. Pseudodeterministic constructions in subexponential time. In Symposium on Theory of Computing (STOC), pages 665-677, 2017. Google Scholar
  30. Alexander A. Razborov and Steven Rudich. Natural proofs. Journal of Computer and System Sciences, 55(1):24-35, 1997. Google Scholar
  31. Benjamin Rossman, Rocco A. Servedio, and Li-Yang Tan. An average-case depth hierarchy theorem for boolean circuits. CoRR, abs/1504.03398, 2015. Google Scholar
  32. Meera Sitharam. Pseudorandom generators and learning algorithms for AC⁰. Computational Complexity, 5(3/4):248-266, 1995. Google Scholar
  33. Meera Sitharam and Timothy Straney. Derandomized learning of boolean functions. In International Conference on Algorithmic Learning Theory (ALT), pages 100-115, 1997. Google Scholar
  34. Christopher Umans. Pseudo-random generators for all hardnesses. Journal of Computer and System Sciences, 67(2):419-440, 2003. Google Scholar
  35. Emanuele Viola. Randomness buys depth for approximate counting. Computational Complexity, 23(3):479-508, 2014. Google Scholar
  36. Ilya Volkovich. On learning, lower bounds and (un)keeping promises. In International Colloquium on Automata, Languages, and Programming (ICALP), pages 1027-1038, 2014. URL: http://dx.doi.org/10.1007/978-3-662-43948-7_85.
  37. Ryan Williams. Improving exhaustive search implies superpolynomial lower bounds. SIAM Journal on Computing, 42(3):1218-1244, 2013. URL: http://dx.doi.org/10.1137/10080703X.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact