Document Open Access Logo

New Statistical and Computational Results for Learning Junta Distributions

Author Lorenzo Beretta

PDF

File

Thumbnail PDF
PDF
LIPIcs.APPROX-RANDOM.2025.31.pdf
  • Filesize: 0.98 MB
  • 23 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Boolean function learning
Keywords
  • Junta Distributions
  • Learning Parities with Noise

Metrics

Abstract

We study the problem of learning junta distributions on {±1}ⁿ, where a distribution is a k-junta if its probability mass function depends on a subset of at most k variables. We make two main contributions:  
- We show that learning k-junta distributions is computationally equivalent to learning k-parity functions with noise (LPN), a landmark problem in computational learning theory.
- We design an algorithm for learning junta distributions whose statistical complexity is optimal, up to polylogarithmic factors. Computationally, our algorithm matches the complexity of previous (non-sample-optimal) algorithms.  Combined, our two contributions imply that our algorithm cannot be significantly improved, statistically or computationally, barring a breakthrough for LPN.

Cite As Get BibTex

Lorenzo Beretta. New Statistical and Computational Results for Learning Junta Distributions. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 353, pp. 31:1-31:23, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.APPROX/RANDOM.2025.31

Author Details

Lorenzo Beretta
  • University of California, Santa Cruz, CA, USA

Acknowledgements

I would like to thank Caleb Koch for the many insightful conversations on juntas, and for introducing me to learning theory in the first place.

References

  1. Maryam Aliakbarpour, Eric Blais, and Ronitt Rubinfeld. Learning and testing junta distributions. In Conference on Learning Theory, pages 19-46. PMLR, 2016. URL: http://proceedings.mlr.press/v49/aliakbarpour16.html.
  2. Josh Alman. An illuminating algorithm for the light bulb problem. arXiv preprint arXiv:1810.06740, 2018. URL: https://arxiv.org/abs/1810.06740.
  3. Jan Arpe and Rüdiger Reischuk. Learning juntas in the presence of noise. Theoretical computer science, 384(1):2-21, 2007. URL: https://doi.org/10.1016/J.TCS.2007.05.014.
  4. Vikraman Arvind, Johannes Köbler, and Wolfgang Lindner. Parameterized learnability of juntas. Theoretical computer science, 410(47-49):4928-4936, 2009. URL: https://doi.org/10.1016/J.TCS.2009.07.003.
  5. Zongbo Bao and Penghui Yao. On testing and learning quantum junta channels. In The Thirty Sixth Annual Conference on Learning Theory, pages 1064-1094. PMLR, 2023. URL: https://proceedings.mlr.press/v195/bao23b.html.
  6. Boaz Barak, Benjamin Edelman, Surbhi Goel, Sham Kakade, Eran Malach, and Cyril Zhang. Hidden progress in deep learning: Sgd learns parities near the computational limit. Advances in Neural Information Processing Systems, 35:21750-21764, 2022. Google Scholar
  7. Eric Blais. Improved bounds for testing juntas. In International Workshop on Approximation Algorithms for Combinatorial Optimization, pages 317-330. Springer, 2008. URL: https://doi.org/10.1007/978-3-540-85363-3_26.
  8. Avrim Blum. Relevant examples and relevant features: Thoughts from computational learning theory. In AAAI Fall Symposium on ‘Relevance, volume 5, page 1, 1994. Google Scholar
  9. Avrim Blum. Open problems-learning a function of r relevant variables. Lecture Notes in Computer Science, 2777:731-733, 2003. Google Scholar
  10. Avrim Blum, Adam Kalai, and Hal Wasserman. Noise-tolerant learning, the parity problem, and the statistical query model. Journal of the ACM (JACM), 50(4):506-519, 2003. URL: https://doi.org/10.1145/792538.792543.
  11. Avrim L Blum and Pat Langley. Selection of relevant features and examples in machine learning. Artificial intelligence, 97(1-2):245-271, 1997. URL: https://doi.org/10.1016/S0004-3702(97)00063-5.
  12. Nader H Bshouty and Areej Costa. Exact learning of juntas from membership queries. Theoretical Computer Science, 742:82-97, 2018. URL: https://doi.org/10.1016/J.TCS.2017.12.032.
  13. Girish Chandrashekar and Ferat Sahin. A survey on feature selection methods. Computers & electrical engineering, 40(1):16-28, 2014. URL: https://doi.org/10.1016/J.COMPELECENG.2013.11.024.
  14. Thomas Chen, Shivam Nadimpalli, and Henry Yuen. Testing and learning quantum juntas nearly optimally. In Proceedings of the 2023 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 1163-1185. SIAM, 2023. URL: https://doi.org/10.1137/1.9781611977554.CH43.
  15. Xi Chen, Rajesh Jayaram, Amit Levi, and Erik Waingarten. Learning and testing junta distributions with sub cube conditioning. In Conference on Learning Theory, pages 1060-1113. PMLR, 2021. URL: http://proceedings.mlr.press/v134/chen21b.html.
  16. Anindya De, Huan Li, Shivam Nadimpalli, and Rocco A Servedio. Detecting low-degree truncation. In Proceedings of the 56th Annual ACM Symposium on Theory of Computing, pages 1027-1038, 2024. URL: https://doi.org/10.1145/3618260.3649633.
  17. Ilias Diakonikolas. Learning structured distributions. Handbook of Big Data, 267:10-1201, 2016. Google Scholar
  18. Ilias Diakonikolas, Themis Gouleakis, John Peebles, and Eric Price. Sample-optimal identity testing with high probability. In 45th International Colloquium on Automata, Languages, and Programming (ICALP 2018). Schloss Dagstuhl - Leibniz Zentrum für Informatik, 2018. URL: https://doi.org/10.4230/LIPIcs.ICALP.2018.41.
  19. Francisco Escudero-Gutiérrez. Learning junta distributions and quantum junta states, and qac 0 circuits. arXiv preprint arXiv:2410.15822, 2024. URL: https://doi.org/10.48550/arXiv.2410.15822.
  20. Vitaly Feldman, Parikshit Gopalan, Subhash Khot, and Ashok Kumar Ponnuswami. New results for learning noisy parities and halfspaces. In 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06), pages 563-574. IEEE, 2006. URL: https://doi.org/10.1109/FOCS.2006.51.
  21. Vitaly Feldman, Parikshit Gopalan, Subhash Khot, and Ashok Kumar Ponnuswami. On agnostic learning of parities, monomials, and halfspaces. SIAM Journal on Computing, 39(2):606-645, 2009. URL: https://doi.org/10.1137/070684914.
  22. Vitaly Feldman, Pravesh Kothari, and Jan Vondrák. Representation, approximation and learning of submodular functions using low-rank decision trees. In Conference on Learning Theory, pages 711-740. PMLR, 2013. URL: http://proceedings.mlr.press/v30/Feldman13.html.
  23. Surbhi Goel, Sushrut Karmalkar, and Adam Klivans. Time/accuracy tradeoffs for learning a relu with respect to gaussian marginals. Advances in neural information processing systems, 32, 2019. Google Scholar
  24. William He and Shivam Nadimpalli. Testing junta truncation. arXiv preprint arXiv:2308.13992, 2023. URL: https://doi.org/10.48550/arXiv.2308.13992.
  25. Adam Tauman Kalai, Adam R Klivans, Yishay Mansour, and Rocco A Servedio. Agnostically learning halfspaces. SIAM Journal on Computing, 37(6):1777-1805, 2008. URL: https://doi.org/10.1137/060649057.
  26. Michael Kearns. Efficient noise-tolerant learning from statistical queries. Journal of the ACM (JACM), 45(6):983-1006, 1998. URL: https://doi.org/10.1145/293347.293351.
  27. Michael Kearns, Yishay Mansour, Dana Ron, Ronitt Rubinfeld, Robert E Schapire, and Linda Sellie. On the learnability of discrete distributions. In Proceedings of the twenty-sixth annual ACM symposium on Theory of computing, pages 273-282, 1994. URL: https://doi.org/10.1145/195058.195155.
  28. Eike Kiltz, Krzysztof Pietrzak, Daniele Venturi, David Cash, and Abhishek Jain. Efficient authentication from hard learning problems. Journal of Cryptology, 30(4):1238-1275, 2017. URL: https://doi.org/10.1007/S00145-016-9247-3.
  29. Adam Klivans and Pravesh Kothari. Embedding hard learning problems into gaussian space. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2014). Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2014. URL: https://doi.org/10.4230/LIPIcs.APPROX-RANDOM.2014.793.
  30. Caleb Koch, Carmen Strassle, and Li-Yang Tan. Superpolynomial lower bounds for decision tree learning and testing. In Proceedings of the 2023 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 1962-1994. SIAM, 2023. URL: https://doi.org/10.1137/1.9781611977554.CH75.
  31. Nathan Linial, Yishay Mansour, and Noam Nisan. Constant depth circuits, fourier transform, and learnability. Journal of the ACM (JACM), 40(3):607-620, 1993. URL: https://doi.org/10.1145/174130.174138.
  32. Elchanan Mossel, Ryan O'Donnell, and Rocco A Servedio. Learning functions of k relevant variables. Journal of Computer and System Sciences, 69(3):421-434, 2004. URL: https://doi.org/10.1016/J.JCSS.2004.04.002.
  33. Oded Regev. On lattices, learning with errors, random linear codes, and cryptography. Journal of the ACM (JACM), 56(6):1-40, 2009. URL: https://doi.org/10.1145/1568318.1568324.
  34. Mert Sağlam. Near log-convexity of measured heat in (discrete) time and consequences. In 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS), pages 967-978. IEEE, 2018. Google Scholar
  35. Rocco A Servedio, Li-Yang Tan, and John Wright. Adaptivity helps for testing juntas. In 30th Conference on Computational Complexity (CCC 2015). Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2015. URL: https://doi.org/10.4230/LIPIcs.CCC.2015.264.
  36. Gregory Valiant. Finding correlations in subquadratic time, with applications to learning parities and the closest pair problem. Journal of the ACM (JACM), 62(2):1-45, 2015. URL: https://doi.org/10.1145/2728167.
  37. Gregory Valiant and Paul Valiant. Estimating the unseen: improved estimators for entropy and other properties. Journal of the ACM (JACM), 64(6):1-41, 2017. URL: https://doi.org/10.1145/3125643.
  38. Yiming Ying. Mcdiarmid’s inequalities of bernstein and bennett forms. City University of Hong Kong, page 318, 2004. Google Scholar
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact