Document Open Access Logo

Learning Algorithms from Natural Proofs

Authors Marco L. Carmosino, Russell Impagliazzo, Valentine Kabanets, Antonina Kolokolova

PDF

File

Thumbnail PDF
PDF
LIPIcs.CCC.2016.10.pdf
  • Filesize: 0.69 MB
  • 24 pages

Document Identifiers

Subject Classification

Keywords
  • natural proofs
  • circuit complexity
  • lower bounds
  • learning
  • compression

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

Based on Hastad's (1986) circuit lower bounds, Linial, Mansour, and Nisan (1993) gave a quasipolytime learning algorithm for AC^0 (constant-depth circuits with AND, OR, and NOT gates), in the PAC model over the uniform distribution. It was an open question to get a learning algorithm (of any kind) for the class of AC^0[p] circuits (constant-depth, with AND, OR, NOT, and MOD_p gates for a prime p). 

Our main result is a quasipolytime learning algorithm for AC^0[p] in the PAC model over the uniform distribution with membership queries. This algorithm is an application of a general connection we show to hold between natural proofs (in the sense of Razborov and Rudich (1997)) and learning algorithms. We argue that a natural proof of a circuit lower bound against any (sufficiently powerful) circuit class yields a learning algorithm for the same circuit class. As the lower bounds against AC^0[p] by Razborov (1987) and Smolensky (1987) are natural, we obtain our learning algorithm for AC^0[p].

Cite As Get BibTex

Marco L. Carmosino, Russell Impagliazzo, Valentine Kabanets, and Antonina Kolokolova. Learning Algorithms from Natural Proofs. In 31st Conference on Computational Complexity (CCC 2016). Leibniz International Proceedings in Informatics (LIPIcs), Volume 50, pp. 10:1-10:24, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2016) https://doi.org/10.4230/LIPIcs.CCC.2016.10

Author Details

Marco L. Carmosino
Russell Impagliazzo
Valentine Kabanets
Antonina Kolokolova

References

  1. Baris Aydinlioglu and Dieter van Melkebeek. Nondeterministic circuit lower bounds from mildly de-randomizing Arthur-Merlin games. In Proceedings of the 27th Conference on Computational Complexity, CCC 2012, Porto, Portugal, June 26-29, 2012, pages 269-279, 2012. Google Scholar
  2. László Babai, Lance Fortnow, Noam Nisan, and Avi Wigderson. BPP has subexponential time simulations unless EXPTIME has publishable proofs. Computational Complexity, 3:307-318, 1993. URL: http://dx.doi.org/10.1007/BF01275486.
  3. Paul Beame, Russell Impagliazzo, and Srikanth Srinivasan. Approximating AC⁰ by small height decision trees and a deterministic algorithm for #AC⁰SAT. In Proceedings of the 27th Conference on Computational Complexity, CCC 2012, Porto, Portugal, June 26-29, 2012, pages 117-125, 2012. Google Scholar
  4. Mark Braverman. Polylogarithmic independence fools AC⁰ circuits. Journal of the ACM, 57:28:1-28:10, 2010. Google Scholar
  5. Marco Carmosino, Russell Impagliazzo, Valentine Kabanets, and Antonina Kolokolova. Tighter connections between derandomization and circuit lower bounds. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques, APPROX/RANDOM 2015, August 24-26, 2015, Princeton, NJ, USA, pages 645-658, 2015. Google Scholar
  6. Ruiwen Chen and Valentine Kabanets. Correlation bounds and #SAT algorithms for small linear-size circuits. In Dachuan Xu, Donglei Du, and Dingzhu Du, editors, Computing and Combinatorics - 21st International Conference, COCOON 2015, Beijing, China, August 4-6, 2015, Proceedings, volume 9198 of Lecture Notes in Computer Science, pages 211-222. Springer, 2015. URL: http://dx.doi.org/10.1007/978-3-319-21398-9_17.
  7. Ruiwen Chen, Valentine Kabanets, Antonina Kolokolova, Ronen Shaltiel, and David Zuckerman. Mining circuit lower bound proofs for meta-algorithms. Computational Complexity, 24(2):333-392, 2015. URL: http://dx.doi.org/10.1007/s00037-015-0100-0.
  8. Ruiwen Chen, Valentine Kabanets, and Nitin Saurabh. An improved deterministic #SAT algorithm for small de Morgan formulas. In Mathematical Foundations of Computer Science 2014 - 39th International Symposium, MFCS 2014, Budapest, Hungary, August 25-29, 2014. Proceedings, Part II, pages 165-176, 2014. Google Scholar
  9. Ruiwen Chen and Rahul Santhanam. Improved algorithms for sparse MAX-SAT and MAX-k-CSP. In Theory and Applications of Satisfiability Testing - SAT 2015 - 18th International Conference, Austin, TX, USA, September 24-27, 2015, Proceedings, pages 33-45, 2015. URL: http://dx.doi.org/10.1007/978-3-319-24318-4_4.
  10. Lance Fortnow and Adam R. Klivans. Efficient learning algorithms yield circuit lower bounds. J. Comput. Syst. Sci., 75(1):27-36, 2009. URL: http://dx.doi.org/10.1016/j.jcss.2008.07.006.
  11. Oded Goldreich, Shafi Goldwasser, and Silvio Micali. How to construct random functions. J. ACM, 33(4):792-807, 1986. Google Scholar
  12. Oded Goldreich and Leonid A. Levin. A hard-core predicate for all one-way functions. In David S. Johnson, editor, Proceedings of the 21st Annual ACM Symposium on Theory of Computing, May 14-17, 1989, Seattle, Washigton, USA, pages 25-32. ACM, 1989. URL: http://dx.doi.org/10.1145/73007.73010.
  13. Oded Goldreich, Noam Nisan, and Avi Wigderson. On Yao’s XOR-lemma. In Studies in Complexity and Cryptography. Miscellanea on the Interplay between Randomness and Computation, pages 273-301. Springer, 2011. Google Scholar
  14. Oded Goldreich, Ronitt Rubinfeld, and Madhu Sudan. Learning polynomials with queries: The highly noisy case. SIAM J. Discrete Math., 13(4):535-570, 2000. URL: http://dx.doi.org/10.1137/S0895480198344540.
  15. Johan Håstad. Almost optimal lower bounds for small depth circuits. In S. Micali, editor, Randomness and Computation, pages 143-170, Greenwich, Connecticut, 1989. Advances in Computing Research, vol. 5, JAI Press. Google Scholar
  16. Alexander Healy, Salil Vadhan, and Emanuele Viola. Using nondeterminism to amplify hardness. SIAM Journal on Computing, 35(4):903-931, 2006. Google Scholar
  17. Russell Impagliazzo, Ragesh Jaiswal, Valentine Kabanets, and Avi Wigderson. Uniform direct product theorems: Simplified, optimized, and derandomized. SIAM J. Comput., 39(4):1637-1665, 2010. URL: http://dx.doi.org/10.1137/080734030.
  18. Russell Impagliazzo, Valentine Kabanets, and Avi Wigderson. In search of an easy witness: Exponential time vs. probabilistic polynomial time. J. Comput. Syst. Sci., 65(4):672-694, 2002. URL: http://dx.doi.org/10.1016/S0022-0000(02)00024-7.
  19. Russell Impagliazzo, William Matthews, and Ramamohan Paturi. A satisfiability algorithm for ∿⁰. In Yuval Rabani, editor, Proceedings of the Twenty-Third Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2012, Kyoto, Japan, January 17-19, 2012, pages 961-972. SIAM, 2012. URL: http://dx.doi.org/10.1137/1.9781611973099.
  20. Russell Impagliazzo, Raghu Meka, and David Zuckerman. Pseudorandomness from shrinkage. In 53rd Annual IEEE Symposium on Foundations of Computer Science, FOCS 2012, New Brunswick, NJ, USA, October 20-23, 2012, pages 111-119, 2012. URL: http://dx.doi.org/10.1109/FOCS.2012.78.
  21. Russell Impagliazzo and Avi Wigderson. P = BPP if E requires exponential circuits: Derandomizing the XOR lemma. In Frank Thomson Leighton and Peter W. Shor, editors, Proceedings of the Twenty-Ninth Annual ACM Symposium on the Theory of Computing, El Paso, Texas, USA, May 4-6, 1997, pages 220-229. ACM, 1997. URL: http://dx.doi.org/10.1145/258533.258590.
  22. Russell Impagliazzo and Avi Wigderson. Randomness vs time: Derandomization under a uniform assumption. J. Comput. Syst. Sci., 63(4):672-688, 2001. URL: http://dx.doi.org/10.1006/jcss.2001.1780.
  23. Valentine Kabanets and Russell Impagliazzo. Derandomizing polynomial identity tests means proving circuit lower bounds. Computational Complexity, 13(1-2):1-46, 2004. URL: http://dx.doi.org/10.1007/s00037-004-0182-6.
  24. Adam Klivans, Pravesh Kothari, and Igor Carboni Oliveira. Constructing hard functions using learning algorithms. In Proceedings of the 28th Conference on Computational Complexity, CCC 2013, Palo Alto, California, USA, 5-7 June, 2013, pages 86-97. IEEE, 2013. URL: http://dx.doi.org/10.1109/CCC.2013.18.
  25. Nathan Linial, Yishay Mansour, and Noam Nisan. Constant depth circuits, Fourier transform, and learnability. J. ACM, 40(3):607-620, 1993. URL: http://dx.doi.org/10.1145/174130.174138.
  26. Oleg B. Lupanov. On the synthesis of switching circuits. Soviet Mathematics, 119(1):23-26, 1958. English translation in Soviet Mathematics Doklady. Google Scholar
  27. Oleg B. Lupanov. A method of circuit synthesis. Izvestiya VUZ, Radiofizika, 1(1):120-140, 1959. (in Russian). Google Scholar
  28. Noam Nisan and Avi Wigderson. Hardness vs randomness. J. Comput. Syst. Sci., 49(2):149-167, 1994. URL: http://dx.doi.org/10.1016/S0022-0000(05)80043-1.
  29. Ryan O'Donnell. Hardness amplification within NP. J. Comput. Syst. Sci., 69(1):68-94, 2004. Google Scholar
  30. Alexander A. Razborov. Lower bounds on the size of bounded depth circuits over a complete basis with logical addition. Mathematical Notes, 41(4):333-338, 1987. Google Scholar
  31. Alexander A. Razborov and Steven Rudich. Natural proofs. J. Comput. Syst. Sci., 55(1):24-35, 1997. URL: http://dx.doi.org/10.1006/jcss.1997.1494.
  32. Rahul Santhanam. Fighting perebor: New and improved algorithms for formula and QBF satisfiability. In 51th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2010, October 23-26, 2010, Las Vegas, Nevada, USA, pages 183-192. IEEE Computer Society, 2010. URL: http://dx.doi.org/10.1109/FOCS.2010.25.
  33. Rahul Santhanam and Richard Ryan Williams. Beating exhaustive search for quantified boolean formulas and connections to circuit complexity. In Proceedings of the Twenty-Sixth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2015, San Diego, CA, USA, January 4-6, 2015, pages 231-241, 2015. URL: http://dx.doi.org/10.1137/1.9781611973730.18.
  34. Kazuhisa Seto and Suguru Tamaki. A satisfiability algorithm and average-case hardness for formulas over the full binary basis. In Proceedings of the Twenty-Seventh Annual IEEE Conference on Computational Complexity, pages 107-116, 2012. Google Scholar
  35. Roman Smolensky. Algebraic methods in the theory of lower bounds for boolean circuit complexity. In Proceedings of the nineteenth annual ACM symposium on Theory of computing, pages 77-82. ACM, 1987. Google Scholar
  36. Srikanth Srinivasan. A compression algorithm for AC⁰[⊕] circuits using certifying polynomials. Electronic Colloquium on Computational Complexity (ECCC), 22:142, 2015. URL: http://eccc.hpi-web.de/report/2015/142.
  37. Avishay Tal. #SAT algorithms from shrinkage. Electronic Colloquium on Computational Complexity (ECCC), 22:114, 2015. URL: http://eccc.hpi-web.de/report/2015/114.
  38. Luca Trevisan. On uniform amplification of hardness in NP. In Proceedings of the thirty-seventh annual ACM symposium on Theory of computing, pages 31-38. ACM, 2005. Google Scholar
  39. Christopher Umans. Pseudo-random generators for all hardnesses. J. Comput. Syst. Sci., 67(2):419-440, 2003. URL: http://dx.doi.org/10.1016/S0022-0000(03)00046-1.
  40. John von Neumann. Various techniques used in connection with random digits. J. Research Nat. Bur. Stand., Appl. Math. Series, 12:36-38, 1951. Google Scholar
  41. Ryan Williams. Improving exhaustive search implies superpolynomial lower bounds. SIAM J. Comput., 42(3):1218-1244, 2013. URL: http://dx.doi.org/10.1137/10080703X.
  42. Ryan Williams. New algorithms and lower bounds for circuits with linear threshold gates. In Symposium on Theory of Computing, STOC 2014, New York, NY, USA, May 31 - June 03, 2014, pages 194-202, 2014. URL: http://dx.doi.org/10.1145/2591796.2591858.
  43. Ryan Williams. Nonuniform ACC circuit lower bounds. J. ACM, 61(1):2:1-2:32, 2014. URL: http://dx.doi.org/10.1145/2559903.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact