Document Open Access Logo

Conspiracies Between Learning Algorithms, Circuit Lower Bounds, and Pseudorandomness

Authors Igor C. Carboni Oliveira, Rahul Santhanam

PDF

File

Thumbnail PDF
PDF
LIPIcs.CCC.2017.18.pdf
  • Filesize: 0.9 MB
  • 49 pages

Document Identifiers

Subject Classification

Keywords
  • boolean circuits
  • learning theory
  • pseudorandomness

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

We prove several results giving new and stronger connections between learning theory, circuit complexity and pseudorandomness. Let C be any typical class of Boolean circuits, and C[s(n)] denote n-variable C-circuits of size <= s(n). We show:

Learning Speedups:  If C[s(n)] admits a randomized weak learning algorithm under the uniform distribution with membership queries that runs in time 2^n/n^{\omega(1)}, then for every k >= 1 and epsilon > 0 the class C[n^k] can be learned to high accuracy in time O(2^{n^epsilon}). There is epsilon > 0 such that C[2^{n^{epsilon}}] can be learned in time 2^n/n^{omega(1)} if and only if C[poly(n)] can be learned in time 2^{(log(n))^{O(1)}}.

Equivalences between Learning Models: We use learning speedups to obtain equivalences between various randomized learning and compression models, including sub-exponential time learning with membership queries, sub-exponential time learning with membership and equivalence queries, probabilistic function compression and probabilistic average-case function compression.

A Dichotomy between Learnability and Pseudorandomness: In the non-uniform setting, there is non-trivial learning for C[poly(n)] if and only if there are no exponentially secure pseudorandom functions computable in C[poly(n)].

Lower Bounds from Nontrivial Learning: If for each k >= 1, (depth-d)-C[n^k] admits a randomized weak learning algorithm with membership queries under the uniform distribution that runs in time 2^n/n^{\omega(1)}, then for each k >= 1, BPE is not contained in (depth-d)-C[n^k]. If for some epsilon > 0 there are P-natural proofs useful against C[2^{n^{epsilon}}], then ZPEXP is not contained in C[poly(n)].

Karp-Lipton Theorems for Probabilistic Classes: If there is a k > 0 such that BPE is contained in i.o.Circuit[n^k], then BPEXP is contained in i.o.EXP/O(log(n)). If ZPEXP is contained in i.o.Circuit[2^{n/3}], then ZPEXP is contained in i.o.ESUBEXP.

Hardness Results for MCSP: All functions in non-uniform NC^1 reduce to the Minimum Circuit Size Problem via truth-table reductions computable by TC^0 circuits. In particular, if MCSP is in TC^0 then NC^1 = TC^0.

Cite As Get BibTex

Igor C. Carboni Oliveira and Rahul Santhanam. Conspiracies Between Learning Algorithms, Circuit Lower Bounds, and Pseudorandomness. In 32nd Computational Complexity Conference (CCC 2017). Leibniz International Proceedings in Informatics (LIPIcs), Volume 79, pp. 18:1-18:49, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2017) https://doi.org/10.4230/LIPIcs.CCC.2017.18

Author Details

Igor C. Carboni Oliveira
Rahul Santhanam

References

  1. Manindra Agrawal and V. Vinay. Arithmetic circuits: A chasm at depth four. In Symposium on Foundations of Computer Science (FOCS), pages 67-75, 2008. URL: http://dx.doi.org/10.1109/FOCS.2008.32.
  2. Eric Allender, Vikraman Arvind, and Fengming Wang. Uniform derandomization from pathetic lower bounds. In International Workshop on Randomization and Computation (RANDOM), pages 380-393, 2010. URL: http://dx.doi.org/10.1007/978-3-642-15369-3_29.
  3. Eric Allender, Harry Buhrman, Michal Koucký, Dieter van Melkebeek, and Detlef Ronneburger. Power from random strings. SIAM J. Comput., 35(6):1467-1493, 2006. URL: http://dx.doi.org/10.1137/050628994.
  4. Eric Allender and Bireswar Das. Zero knowledge and circuit minimization. In Symposium on Mathematical Foundations of Computer Science (MFCS), pages 25-32, 2014. URL: http://dx.doi.org/10.1007/978-3-662-44465-8_3.
  5. Eric Allender, Joshua A. Grochow, and Cristopher Moore. Graph isomorphism and circuit size. CoRR, abs/1511.08189, 2015. URL: http://arxiv.org/abs/1511.08189.
  6. Eric Allender, Lisa Hellerstein, Paul McCabe, Toniann Pitassi, and Michael E. Saks. Minimizing disjunctive normal form formulas and AC⁰ circuits given a truth table. SIAM J. Comput., 38(1):63-84, 2008. URL: http://dx.doi.org/10.1137/060664537.
  7. Eric Allender, Dhiraj Holden, and Valentine Kabanets. The minimum oracle circuit size problem. In Symposium on Theoretical Aspects of Computer Science (STACS), volume 30 of LIPIcs, pages 21-33, 2015. URL: http://dx.doi.org/10.4230/LIPIcs.STACS.2015.21.
  8. Eric Allender and Michal Koucký. Amplifying lower bounds by means of self-reducibility. J. ACM, 57(3), 2010. URL: http://dx.doi.org/10.1145/1706591.1706594.
  9. Ingo Althöfer. On sparse approximations to randomized strategies and convex combinations. Linear Algebra Appl., 199:339-355, 1994. URL: http://dx.doi.org/10.1016/0024-3795(94)90357-3.
  10. Dana Angluin. Queries and concept learning. Machine Learning, 2(4):319-342, 1987. URL: http://dx.doi.org/10.1007/BF00116828.
  11. Benny Applebaum, Yuval Ishai, and Eyal Kushilevitz. Cryptography in NC⁰. SIAM J. Comput., 36(4):845-888, 2006. URL: http://dx.doi.org/10.1137/S0097539705446950.
  12. László Babai, Lance Fortnow, Noam Nisan, and Avi Wigderson. BPP has subexponential time simulations unless EXPTIME has publishable proofs. Computational Complexity, 3:307-318, 1993. URL: http://dx.doi.org/10.1007/BF01275486.
  13. László Babai and Shlomo Moran. Arthur-Merlin games: A randomized proof system, and a hierarchy of complexity classes. J. Comput. Syst. Sci., 36(2):254-276, 1988. URL: http://dx.doi.org/10.1016/0022-0000(88)90028-1.
  14. Avrim Blum, Merrick L. Furst, Michael J. Kearns, and Richard J. Lipton. Cryptographic primitives based on hard learning problems. In Advances in Cryptology (CRYPTO), pages 278-291, 1993. URL: http://dx.doi.org/10.1007/3-540-48329-2_24.
  15. Manuel Blum. A machine-independent theory of the complexity of recursive functions. J. ACM, 14(2):322-336, 1967. URL: http://dx.doi.org/10.1145/321386.321395.
  16. Dan Boneh and Richard J. Lipton. Amplification of weak learning under the uniform distribution. In Conference on Computational Learning Theory (COLT), pages 347-351, 1993. URL: http://dx.doi.org/10.1145/168304.168372.
  17. Harry Buhrman, Lance Fortnow, and Thomas Thierauf. Nonrelativizing separations. In Conference on Computational Complexity (CCC), pages 8-12, 1998. URL: http://dx.doi.org/10.1109/CCC.1998.694585.
  18. Harry Buhrman and Steven Homer. Superpolynomial circuits, almost sparse oracles and the exponential hierarchy. In Conference on Foundations of Software Technology and Theoretical Computer Science (FSTTCS), pages 116-127, 1992. URL: http://dx.doi.org/10.1007/3-540-56287-7_99.
  19. Marco L. Carmosino, Russell Impagliazzo, Valentine Kabanets, and Antonina Kolokolova. Learning algorithms from natural proofs. In Conference on Computational Complexity (CCC), volume 50 of LIPIcs, pages 10:1-10:24, 2016. URL: http://dx.doi.org/10.4230/LIPIcs.CCC.2016.10.
  20. Ruiwen Chen, Valentine Kabanets, Antonina Kolokolova, Ronen Shaltiel, and David Zuckerman. Mining circuit lower bound proofs for meta-algorithms. Computational Complexity, 24(2):333-392, 2015. URL: http://dx.doi.org/10.1007/s00037-015-0100-0.
  21. Lance Fortnow and Adam R. Klivans. Efficient learning algorithms yield circuit lower bounds. J. Comput. Syst. Sci., 75(1):27-36, 2009. URL: http://dx.doi.org/10.1016/j.jcss.2008.07.006.
  22. Lance Fortnow and Rahul Santhanam. Infeasibility of instance compression and succinct PCPs for NP. J. Comput. Syst. Sci., 77(1):91-106, 2011. URL: http://dx.doi.org/10.1016/j.jcss.2010.06.007.
  23. Oded Goldreich. The Foundations of Cryptography - Volume 1, Basic Techniques. Cambridge University Press, 2001. Google Scholar
  24. Oded Goldreich, Noam Nisan, and Avi Wigderson. On Yao’s XOR-Lemma. In Studies in Complexity and Cryptography, pages 273-301. Springer, 2011. URL: http://dx.doi.org/10.1007/978-3-642-22670-0_23.
  25. Shafi Goldwasser and Silvio Micali. Probabilistic encryption and how to play mental poker keeping secret all partial information. In Symposium on Theory of Computing (STOC), pages 365-377, 1982. URL: http://dx.doi.org/10.1145/800070.802212.
  26. Shafi Goldwasser and Silvio Micali. Probabilistic encryption. J. Comput. Syst. Sci., 28(2):270-299, 1984. URL: http://dx.doi.org/10.1016/0022-0000(84)90070-9.
  27. Ryan C. Harkins and John M. Hitchcock. Exact learning algorithms, betting games, and circuit lower bounds. Transactions on Computation Theory (TOCT), 5(4):18, 2013. URL: http://dx.doi.org/10.1145/2539126.2539130.
  28. Shuichi Hirahara and Osamu Watanabe. Limits of minimum circuit size problem as oracle. In Conference on Computational Complexity (CCC), volume 50 of LIPIcs, pages 18:1-18:20, 2016. URL: http://dx.doi.org/10.4230/LIPIcs.CCC.2016.18.
  29. John M. Hitchcock and Aduri Pavan. On the NP-completeness of the minimum circuit size problem. In Conference on Foundation of Software Technology and Theoretical Computer Science (FSTTCS), volume 45 of LIPIcs, pages 236-245, 2015. URL: http://dx.doi.org/10.4230/LIPIcs.FSTTCS.2015.236.
  30. Russell Impagliazzo, Valentine Kabanets, and Avi Wigderson. In search of an easy witness: exponential time vs. probabilistic polynomial time. J. Comput. Syst. Sci., 65(4):672-694, 2002. URL: http://dx.doi.org/10.1016/S0022-0000(02)00024-7.
  31. Russell Impagliazzo and Leonid A. Levin. No better ways to generate hard NP instances than picking uniformly at random. In Symposium on Foundations of Computer Science (FOCS), pages 812-821, 1990. URL: http://dx.doi.org/10.1109/FSCS.1990.89604.
  32. Russell Impagliazzo and Avi Wigderson. P = BPP if E requires exponential circuits: Derandomizing the XOR lemma. In Symposium on Theory of Computing (STOC), pages 220-229, 1997. URL: http://dx.doi.org/10.1145/258533.258590.
  33. Russell Impagliazzo and Avi Wigderson. Randomness vs time: Derandomization under a uniform assumption. J. Comput. Syst. Sci., 63(4):672-688, 2001. URL: http://dx.doi.org/10.1006/jcss.2001.1780.
  34. Jeffrey C. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. J. Comput. Syst. Sci., 55(3):414-440, 1997. URL: http://dx.doi.org/10.1006/jcss.1997.1533.
  35. Hamid Jahanjou, Eric Miles, and Emanuele Viola. Local reductions. In International Colloquium on Automata, Languages, and Programming (ICALP), pages 749-760, 2015. URL: http://dx.doi.org/10.1007/978-3-662-47672-7_61.
  36. Svante Janson, Tomasz Łuczak, and Andrzej Ruciński. Random graphs. Wiley-Interscience, New York, 2000. Google Scholar
  37. Valentine Kabanets. Easiness assumptions and hardness tests: Trading time for zero error. J. Comput. Syst. Sci., 63(2):236-252, 2001. URL: http://dx.doi.org/10.1006/jcss.2001.1763.
  38. Valentine Kabanets and Jin-Yi Cai. Circuit minimization problem. In Symposium on Theory of Computing (STOC), pages 73-79, 2000. URL: http://dx.doi.org/10.1145/335305.335314.
  39. Richard M. Karp and Richard J. Lipton. Some connections between nonuniform and uniform complexity classes. In Symposium on Theory of Computing (STOC), pages 302-309, 1980. URL: http://dx.doi.org/10.1145/800141.804678.
  40. Jonathan Katz and Yehuda Lindell. Introduction to Modern Cryptography. Chapman and Hall/CRC Press, 2007. Google Scholar
  41. Michael Kearns and Umesh Vazirani. An Introduction to Computational Learning Theory. MIT Press, 1994. Google Scholar
  42. Michael J. Kearns and Leslie G. Valiant. Cryptographic limitations on learning boolean formulae and finite automata. J. ACM, 41(1):67-95, 1994. URL: http://dx.doi.org/10.1145/174644.174647.
  43. Subhash Khot and Rishi Saket. Hardness of minimizing and learning DNF expressions. In Symposium on Foundations of Computer Science (FOCS), pages 231-240, 2008. URL: http://dx.doi.org/10.1109/FOCS.2008.37.
  44. Adam Klivans, Pravesh Kothari, and Igor C. Oliveira. Constructing hard functions using learning algorithms. In Conference on Computational Complexity (CCC), pages 86-97, 2013. URL: http://dx.doi.org/10.1109/CCC.2013.18.
  45. Jan Krajícek. Forcing with Random Variables and Proof Complexity, volume 382 of London Mathematical Society Lecture Note Series. Cambridge University Press, 2011. URL: http://www.cambridge.org/de/academic/subjects/mathematics/logic-categories-and-sets/forcing-random-variables-and-proof-complexity?format=PB.
  46. Nathan Linial, Yishay Mansour, and Noam Nisan. Constant depth circuits, fourier transform, and learnability. J. ACM, 40(3):607-620, 1993. URL: http://dx.doi.org/10.1145/174130.174138.
  47. Richard J. Lipton and Neal E. Young. Simple strategies for large zero-sum games with applications to complexity theory. In Symposium on Theory of Computing (STOC), pages 734-740, 1994. URL: http://dx.doi.org/10.1145/195058.195447.
  48. William J. Masek. Some NP-complete set covering problems. Unpublished Manuscript, 1979. Google Scholar
  49. Cody D. Murray and Richard Ryan Williams. On the (non) NP-hardness of computing circuit complexity. In Conference on Computational Complexity (CCC), volume 33 of LIPIcs, pages 365-380, 2015. URL: http://dx.doi.org/10.4230/LIPIcs.CCC.2015.365.
  50. Moni Naor and Omer Reingold. Number-theoretic constructions of efficient pseudo-random functions. J. ACM, 51(2):231-262, 2004. URL: http://dx.doi.org/10.1145/972639.972643.
  51. Noam Nisan and Avi Wigderson. Hardness vs randomness. J. Comput. Syst. Sci., 49(2):149-167, 1994. URL: http://dx.doi.org/10.1016/S0022-0000(05)80043-1.
  52. Noam Nisan and David Zuckerman. Randomness is linear in space. J. Comput. Syst. Sci., 52(1):43-52, 1996. URL: http://dx.doi.org/10.1006/jcss.1996.0004.
  53. Ryan O'Donnell. Analysis of Boolean Functions. Cambridge University Press, 2014. URL: http://www.cambridge.org/de/academic/subjects/computer-science/algorithmics-complexity-computer-algebra-and-computational-g/analysis-boolean-functions.
  54. Igor C. Oliveira. Unconditional Lower Bounds in Complexity Theory. PhD thesis, Columbia University, 2015. Google Scholar
  55. Christos H. Papadimitriou. Computational complexity. Addison-Wesley, 1994. Google Scholar
  56. Ramamohan Paturi and Pavel Pudlák. On the complexity of circuit satisfiability. In Symposium on Theory of Computing (STOC), pages 241-250, 2010. URL: http://dx.doi.org/10.1145/1806689.1806724.
  57. Ján Pich. Circuit lower bounds in bounded arithmetics. Ann. Pure Appl. Logic, 166(1):29-45, 2015. URL: http://dx.doi.org/10.1016/j.apal.2014.08.004.
  58. Alexander A. Razborov. Pseudorandom generators hard for k-DNF resolution and polynomial calculus resolution. Ann. of Math. (2), 181(2):415-472, 2015. URL: http://dx.doi.org/10.4007/annals.2015.181.2.1.
  59. Alexander A. Razborov and Steven Rudich. Natural proofs. J. Comput. Syst. Sci., 55(1):24-35, 1997. URL: http://dx.doi.org/10.1006/jcss.1997.1494.
  60. Oded Regev. On lattices, learning with errors, random linear codes, and cryptography. J. ACM, 56(6), 2009. URL: http://dx.doi.org/10.1145/1568318.1568324.
  61. Rahul Santhanam. Circuit lower bounds for Merlin-Arthur classes. SIAM J. Comput., 39(3):1038-1061, 2009. URL: http://dx.doi.org/10.1137/070702680.
  62. Rahul Santhanam and Ryan Williams. On uniformity and circuit lower bounds. Computational Complexity, 23(2):177-205, 2014. URL: http://dx.doi.org/10.1007/s00037-014-0087-y.
  63. Ronen Shaltiel and Emanuele Viola. Hardness amplification proofs require majority. SIAM J. Comput., 39(7):3122-3154, 2010. URL: http://dx.doi.org/10.1137/080735096.
  64. Srikanth Srinivasan. A compression algorithm for AC⁰[⊕] circuits using certifying polynomials. Electronic Colloquium on Computational Complexity (ECCC), 22:142, 2015. URL: https://eccc.weizmann.ac.il/report/2015/142/.
  65. Luca Trevisan and Salil P. Vadhan. Pseudorandomness and average-case complexity via uniform reductions. Computational Complexity, 16(4):331-364, 2007. URL: http://dx.doi.org/10.1007/s00037-007-0233-x.
  66. John v. Neumann. Zur Theorie der Gesellschaftsspiele. Math. Ann., 100(1):295-320, 1928. URL: http://dx.doi.org/10.1007/BF01448847.
  67. Salil P. Vadhan and Colin J. Zheng. A uniform min-max theorem with applications in cryptography. In Cryptology Conference (CRYPTO), pages 93-110, 2013. URL: http://dx.doi.org/10.1007/978-3-642-40041-4_6.
  68. Leslie Valiant. A theory of the learnable. Communications of the ACM, pages 1134-1142, 1984. Google Scholar
  69. Ilya Volkovich. On learning, lower bounds and (un)keeping promises. In International Colloquium on Automata, Languages, and Programming (ICALP), pages 1027-1038, 2014. URL: http://dx.doi.org/10.1007/978-3-662-43948-7_85.
  70. Ryan Williams. Improving exhaustive search implies superpolynomial lower bounds. SIAM J. Comput., 42(3):1218-1244, 2013. URL: http://dx.doi.org/10.1137/10080703X.
  71. Ryan Williams. Algorithms for circuits and circuits for algorithms: Connecting the tractable and intractable. Proceedings of the International Congress of Mathematicians (ICM), Volume IV:659-682, 2014. Google Scholar
  72. Ryan Williams. New algorithms and lower bounds for circuits with linear threshold gates. In Symposium on Theory of Computing (STOC), pages 194-202, 2014. URL: http://dx.doi.org/10.1145/2591796.2591858.
  73. Ryan Williams. Nonuniform ACC circuit lower bounds. J. ACM, 61(1):2:1-2:32, 2014. URL: http://dx.doi.org/10.1145/2559903.
  74. Ryan Williams. Natural proofs versus derandomization. SIAM J. Comput., 45(2):497-529, 2016. URL: http://dx.doi.org/10.1137/130938219.
  75. Chee-Keng Yap. Some consequences of non-uniform conditions on uniform classes. Theor. Comput. Sci., 26:287-300, 1983. URL: http://dx.doi.org/10.1016/0304-3975(83)90020-8.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact