Document Open Access Logo

Polynomial Threshold Functions for Decision Lists

Authors Vladimir Podolskii , Nikolay V. Proskurin

PDF

File

Thumbnail PDF
PDF
LIPIcs.ISAAC.2022.52.pdf
  • Filesize: 0.69 MB
  • 12 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Theory of computation → Models of computation
Keywords
  • Threshold function
  • decision list
  • Hamming ball

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

For S ⊆ {0,1}ⁿ a Boolean function f : S → {-1,1} is a polynomial threshold function (PTF) of degree d and weight W if there is a polynomial p with integer coefficients of degree d and with sum of absolute coefficients W such that f(x) = sign p(x) for all x ∈ S. We study a representation of decision lists as PTFs over Boolean cubes {0,1}ⁿ and over Hamming balls {0,1}ⁿ_{≤ k}.
As our first result, we show that for all d = O((n/(log n))^{1/3}) any decision list over {0,1}ⁿ can be represented by a PTF of degree d and weight 2^O(n/d²). This improves the result by Klivans and Servedio [Adam R. Klivans and Rocco A. Servedio, 2006] by a log² d factor in the exponent of the weight. Our bound is tight for all d = O((n/(log n))^{1/3}) due to the matching lower bound by Beigel [Richard Beigel, 1994].
For decision lists over a Hamming ball {0,1}ⁿ_{≤ k} we show that the upper bound on weight above can be drastically improved to n^O(√k) for d = Θ(√k). We also show that similar improvement is not possible for smaller degrees by proving the lower bound W = 2^Ω(n/d²) for all d = O(√k).

Cite As Get BibTex

Vladimir Podolskii and Nikolay V. Proskurin. Polynomial Threshold Functions for Decision Lists. In 33rd International Symposium on Algorithms and Computation (ISAAC 2022). Leibniz International Proceedings in Informatics (LIPIcs), Volume 248, pp. 52:1-52:12, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2022) https://doi.org/10.4230/LIPIcs.ISAAC.2022.52

Author Details

Vladimir Podolskii
  • Courant Institute of Mathematical Sciences, New York University, NY, USA
  • Steklov Mathematical Institute of Russian Academy of Sciences, Moscow, Russian Federation
Nikolay V. Proskurin
  • HSE University, Moscow, Russian Federation

Funding

  • Proskurin, Nikolay V.: This paper was prepared within the framework of the HSE University Basic Research Program.

References

  1. Martin Anthony, Yves Crama, and Peter L. Hammer. Decision lists and related classes of boolean functions. In Yves Crama and Peter L. Hammer, editors, Boolean Models and Methods in Mathematics, Computer Science, and Engineering, pages 577-596. Cambridge University Press, 2010. URL: https://doi.org/10.1017/cbo9780511780448.017.
  2. Richard Beigel. The polynomial method in circuit complexity. In Proceedings of the Eighth Annual Structure in Complexity Theory Conference, pages 82-95. IEEE Computer Society Press, 1993. Google Scholar
  3. Richard Beigel. Perceptrons, PP, and the polynomial hierarchy. Comput. Complex., 4:339-349, 1994. URL: https://doi.org/10.1007/BF01263422.
  4. Arnab Bhattacharyya, Suprovat Ghoshal, and Rishi Saket. Hardness of learning noisy halfspaces using polynomial thresholds. In Sébastien Bubeck, Vianney Perchet, and Philippe Rigollet, editors, Conference On Learning Theory, COLT 2018, Stockholm, Sweden, 6-9 July 2018, volume 75 of Proceedings of Machine Learning Research, pages 876-917. PMLR, 2018. URL: http://proceedings.mlr.press/v75/bhattacharyya18a.html.
  5. Avrim Blum. Rank-r decision trees are a subclass of r-decision lists. Inf. Process. Lett., 42(4):183-185, 1992. URL: https://doi.org/10.1016/0020-0190(92)90237-P.
  6. Andrej Bogdanov and Christopher Williamson. Approximate bounded indistinguishability. In Ioannis Chatzigiannakis, Piotr Indyk, Fabian Kuhn, and Anca Muscholl, editors, 44th International Colloquium on Automata, Languages, and Programming, ICALP 2017, July 10-14, 2017, Warsaw, Poland, volume 80 of LIPIcs, pages 53:1-53:11. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2017. URL: https://doi.org/10.4230/LIPIcs.ICALP.2017.53.
  7. Jehoshua Bruck and Roman Smolensky. Polynomial threshold functions, AC⁰ functions, and spectral norms. SIAM Journal on Computing, 21(1):33-42, 1992. Google Scholar
  8. Harry Buhrman, Nikolai K. Vereshchagin, and Ronald de Wolf. On computation and communication with small bias. In 22nd Annual IEEE Conference on Computational Complexity (CCC 2007), 13-16 June 2007, San Diego, California, USA, pages 24-32. IEEE Computer Society, 2007. URL: https://doi.org/10.1109/CCC.2007.18.
  9. Mark Bun, Robin Kothari, and Justin Thaler. The polynomial method strikes back: Tight quantum query bounds via dual polynomials. Theory Comput., 16:1-71, 2020. URL: https://doi.org/10.4086/toc.2020.v016a010.
  10. Mark Bun and Justin Thaler. A nearly optimal lower bound on the approximate degree of AC^0. SIAM J. Comput., 49(4), 2020. URL: https://doi.org/10.1137/17M1161737.
  11. Mark Bun and Justin Thaler. The large-error approximate degree of AC^0. Theory Comput., 17:1-46, 2021. URL: https://theoryofcomputing.org/articles/v017a007/.
  12. Arkadev Chattopadhyay, Meena Mahajan, Nikhil S. Mande, and Nitin Saurabh. Lower bounds for linear decision lists. Chic. J. Theor. Comput. Sci., 2020, 2020. URL: http://cjtcs.cs.uchicago.edu/articles/2020/1/contents.html.
  13. Arkadev Chattopadhyay and Nikhil S. Mande. A short list of equalities induces large sign rank. In Mikkel Thorup, editor, 59th IEEE Annual Symposium on Foundations of Computer Science, FOCS 2018, Paris, France, October 7-9, 2018, pages 47-58. IEEE Computer Society, 2018. URL: https://doi.org/10.1109/FOCS.2018.00014.
  14. Ronald de Wolf. A note on quantum algorithms and the minimal degree of ε-error polynomials for symmetric functions. Quantum Inf. Comput., 8(10):943-950, 2008. URL: https://doi.org/10.26421/QIC8.10-4.
  15. Ilias Diakonikolas, Daniel M. Kane, and Alistair Stewart. Learning geometric concepts with nasty noise. In Ilias Diakonikolas, David Kempe, and Monika Henzinger, editors, Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2018, Los Angeles, CA, USA, June 25-29, 2018, pages 1061-1073. ACM, 2018. URL: https://doi.org/10.1145/3188745.3188754.
  16. Ilias Diakonikolas, Ryan O'Donnell, Rocco A. Servedio, and Yi Wu. Hardness results for agnostically learning low-degree polynomial threshold functions. In Dana Randall, editor, Proceedings of the Twenty-Second Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2011, San Francisco, California, USA, January 23-25, 2011, pages 1590-1606. SIAM, 2011. URL: https://doi.org/10.1137/1.9781611973082.123.
  17. Mikael Goldmann, Johan Håstad, and Alexander A. Razborov. Majority gates vs. general weighted threshold gates. Computational Complexity, 2(4):277-300, 1992. Google Scholar
  18. Lisa Hellerstein and Rocco A. Servedio. On PAC learning algorithms for rich boolean function classes. Theor. Comput. Sci., 384(1):66-76, 2007. URL: https://doi.org/10.1016/j.tcs.2007.05.018.
  19. Xuangui Huang and Emanuele Viola. Approximate degree, weight, and indistinguishability. ACM Trans. Comput. Theory, 14(1):3:1-3:26, 2022. URL: https://doi.org/10.1145/3492338.
  20. Adam R. Klivans, Ryan O'Donnell, and Rocco A. Servedio. Learning intersections and thresholds of halfspaces. Journal of Computer and System Sciences, 68(4):808-840, 2004. Google Scholar
  21. Adam R. Klivans and Rocco A. Servedio. Learning DNF in time 2^Õ(n^1/3). Journal of Computer and System Sciences, 68(2):303-318, 2004. Google Scholar
  22. Adam R. Klivans and Rocco A. Servedio. Toward attribute efficient learning of decision lists and parities. J. Mach. Learn. Res., 7:587-602, 2006. URL: http://jmlr.org/papers/v7/klivans06a.html.
  23. Matthias Krause and Pavel Pudlák. Computing boolean functions by polynomials and threshold circuits. Computational Complexity, 7(4):346-370, 1998. Google Scholar
  24. Nick Littlestone. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Mach. Learn., 2(4):285-318, 1987. URL: https://doi.org/10.1007/BF00116827.
  25. Philip M. Long and Rocco A. Servedio. On the weight of halfspaces over hamming balls. SIAM J. Discret. Math., 28(3):1035-1061, 2014. URL: https://doi.org/10.1137/120868402.
  26. Noam Nisan and Mario Szegedy. On the degree of boolean functions as real polynomials. Comput. Complex., 4:301-313, 1994. URL: https://doi.org/10.1007/BF01263419.
  27. Ryan O'Donnell and Rocco A. Servedio. New degree bounds for polynomial threshold functions. Comb., 30(3):327-358, 2010. URL: https://doi.org/10.1007/s00493-010-2173-3.
  28. Micheal E. Saks. Slicing the hypercube. In Keith Walker, editor, Surveys in Combinatorics, number 187 in London Mathematical Society Lecture Note Series, pages 211-256. Cambridge University Press, 1993. Google Scholar
  29. Rocco A. Servedio, Li-Yang Tan, and Justin Thaler. Attribute-efficient learning and weight-degree tradeoffs for polynomial threshold functions. In Shie Mannor, Nathan Srebro, and Robert C. Williamson, editors, COLT 2012 - The 25th Annual Conference on Learning Theory, June 25-27, 2012, Edinburgh, Scotland, volume 23 of JMLR Proceedings, pages 14.1-14.19. JMLR.org, 2012. URL: http://proceedings.mlr.press/v23/servedio12/servedio12.pdf.
  30. Alexander A. Sherstov. Communication lower bounds using dual polynomials. Bulletin of the EATCS, 95:59-93, 2008. Google Scholar
  31. Alexander A. Sherstov. Algorithmic polynomials. SIAM J. Comput., 49(6):1173-1231, 2020. URL: https://doi.org/10.1137/19M1278831.
  32. Alexander A. Sherstov. The approximate degree of DNF and CNF formulas. In Stefano Leonardi and Anupam Gupta, editors, STOC '22: 54th Annual ACM SIGACT Symposium on Theory of Computing, Rome, Italy, June 20 - 24, 2022, pages 1194-1207. ACM, 2022. URL: https://doi.org/10.1145/3519935.3520000.
  33. Alexander A. Sherstov and Pei Wu. Near-optimal lower bounds on the threshold degree and sign-rank of AC^0. In Moses Charikar and Edith Cohen, editors, Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, STOC 2019, Phoenix, AZ, USA, June 23-26, 2019, pages 401-412. ACM, 2019. URL: https://doi.org/10.1145/3313276.3316408.
  34. Kei Uchizawa and Eiji Takimoto. Lower bounds for linear decision trees with bounded weights. In Giuseppe F. Italiano, Tiziana Margaria-Steffen, Jaroslav Pokorný, Jean-Jacques Quisquater, and Roger Wattenhofer, editors, SOFSEM 2015: Theory and Practice of Computer Science - 41st International Conference on Current Trends in Theory and Practice of Computer Science, Pec pod Sněžkou, Czech Republic, January 24-29, 2015. Proceedings, volume 8939 of Lecture Notes in Computer Science, pages 412-422. Springer, 2015. URL: https://doi.org/10.1007/978-3-662-46078-8_34.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact