Simple and Efficient Pseudorandom Generators from Gaussian Processes
We show that a very simple pseudorandom generator fools intersections of k linear threshold functions (LTFs) and arbitrary functions of k LTFs over n-dimensional Gaussian space. The two analyses of our PRG (for intersections versus arbitrary functions of LTFs) are quite different from each other and from previous analyses of PRGs for functions of halfspaces. Our analysis for arbitrary functions of LTFs establishes bounds on the Wasserstein distance between Gaussian random vectors with similar covariance matrices, and combines these bounds with a conversion from Wasserstein distance to "union-of-orthants" distance from [Xi Chen et al., 2014]. Our analysis for intersections of LTFs uses extensions of the classical Sudakov-Fernique type inequalities, which give bounds on the difference between the expectations of the maxima of two Gaussian random vectors with similar covariance matrices.
For all values of k, our generator has seed length O(log n) + poly(k) for arbitrary functions of k LTFs and O(log n) + poly(log k) for intersections of k LTFs. The best previous result, due to [Gopalan et al., 2010], only gave such PRGs for arbitrary functions of k LTFs when k=O(log log n) and for intersections of k LTFs when k=O((log n)/(log log n)). Thus our PRG achieves an O(log n) seed length for values of k that are exponentially larger than previous work could achieve.
By combining our PRG over Gaussian space with an invariance principle for arbitrary functions of LTFs and with a regularity lemma, we obtain a deterministic algorithm that approximately counts satisfying assignments of arbitrary functions of k general LTFs over {0,1}^n in time poly(n) * 2^{poly(k,1/epsilon)} for all values of k. This algorithm has a poly(n) runtime for k =(log n)^c for some absolute constant c>0, while the previous best poly(n)-time algorithms could only handle k = O(log log n). For intersections of LTFs, by combining these tools with a recent PRG due to [R. O'Donnell et al., 2018], we obtain a deterministic algorithm that can approximately count satisfying assignments of intersections of k general LTFs over {0,1}^n in time poly(n) * 2^{poly(log k, 1/epsilon)}. This algorithm has a poly(n) runtime for k =2^{(log n)^c} for some absolute constant c>0, while the previous best poly(n)-time algorithms for intersections of k LTFs, due to [Gopalan et al., 2010], could only handle k=O((log n)/(log log n)).
Polynomial threshold functions
Gaussian processes
Johnson-Lindenstrauss
pseudorandom generators
Theory of computation~Pseudorandomness and derandomization
4:1-4:33
Regular Paper
A full version of this paper is available at https://eccc.weizmann.ac.il/report/2018/100/.
The authors thank Oded Regev and Li-Yang Tan for helpful discussions.
Eshan
Chattopadhyay
Eshan Chattopadhyay
Cornell University, Ithaca, NY, USA
Supported by NSF grants CCF-1412958, CCF-1849899, and the Simons foundation. Part of the work done while the author was a postdoctoral researcher at the Institute for Advanced Study, Princeton.
Anindya
De
Anindya De
University of Pennsylvania, Philadelphia, PA, USA
Supported by NSF CCF 1926872 (transferred from CCF 1814706). Work done while the author was at Northwestern University supported by a start-up grant.
Rocco A.
Servedio
Rocco A. Servedio
Columbia University, New York, NY, USA
Supported by NSF CCF 1814873, NSF CCF 1563155, and by the Simons Collaboration on Algorithms and Geometry.
10.4230/LIPIcs.CCC.2019.4
Andrew C. Berry. The accuracy of the Gaussian approximation to the sum of independent variates. Transactions of the American Mathematical Society, 49(1):122-136, 1941.
R. Bhatia. Matrix analysis, volume 169. Springer, 2013.
Thomas Bonis. Rates in the Central Limit Theorem and diffusion approximation via Stein’s Method. arXiv preprint, 2015. URL: http://arxiv.org/abs/1506.06966.
http://arxiv.org/abs/1506.06966
Sourav Chatterjee. An error bound in the Sudakov-Fernique inequality. arXiv preprint, 2005. URL: http://arxiv.org/abs/math/0510424.
http://arxiv.org/abs/math/0510424
Xi Chen, Rocco Servedio, and Li-Yang Tan. New algorithms and lower bounds for testing monotonicity. In Proceedings of the 55th Annual Symposium on Foundations of Computer Science (FOCS 2014), pages 286-295, 2014.
Victor Chernozhukov, Denis Chetverikov, and Kengo Kato. Comparison and anti-concentration bounds for maxima of Gaussian random vectors. Probability Theory and Related Fields, 162(1-2):47-70, 2015.
Anindya De, Ilias Diakonikolas, and Rocco A. Servedio. Deterministic approximate counting for juntas of degree-2 polynomial threshold functions. In Proceedings of the 29th Annual Conference on Computational Complexity (CCC), pages 229-240. IEEE, 2014.
Ilias Diakonikolas, Parikshit Gopalan, Rajesh Jaiswal, Rocco A. Servedio, and Emanuele Viola. Bounded Independence Fools Halfspaces. SIAM Journal on Computing, 39(8):3441-3462, 2010.
Ilias Diakonikolas, Daniel M. Kane, and Jelani Nelson. Bounded independence fools degree-2 threshold functions. In Foundations of Computer Science (FOCS), 2010 51st Annual IEEE Symposium on, pages 11-20. IEEE, 2010.
Carl-Gustav Esseen. On the Liapunoff limit of error in the theory of probability. Arkiv för matematik, astronomi och fysik, A:1-19, 1942.
X. Fernique. Regularité des trajectoires des fonctions aléatoires gaussiennes. In Ecole d’Eté de Probabilités de Saint-Flour IV—1974, pages 1-96. Springer, 1975.
C. Givens and R. Shortt. A class of Wasserstein metrics for probability distributions. The Michigan Mathematical Journal, 31(2):231-240, 1984.
Parikshit Gopalan, Daniel Kane, and Raghu Meka. Pseudorandomness via the discrete fourier transform. In Foundations of Computer Science (FOCS), 2015 IEEE 56th Annual Symposium on, pages 903-922. IEEE, 2015.
Parikshit Gopalan, Ryan O'Donnell, Yi Wu, and David Zuckerman. Fooling functions of halfspaces under product distributions. In Computational Complexity (CCC), 2010 IEEE 25th Annual Conference on, pages 223-234. IEEE, 2010.
Parikshit Gopalan, Ryan O'Donnell, Yi Wu, and David Zuckerman. Fooling Functions of Halfspaces under Product Distributions, 2010. URL: http://arxiv.org/abs/1001.1593.
http://arxiv.org/abs/1001.1593
Prahladh Harsha, Adam Klivans, and Raghu Meka. An invariance principle for polytopes. Journal of the ACM (JACM), 59(6):29, 2012.
Daniel Kane. A Small PRG for Polynomial Threshold Functions of Gaussians. In FOCS, pages 257-266, 2011.
Daniel Kane. k-independent Gaussians fool polynomial threshold functions. In IEEE Conference on Computational Complexity, pages 252-261, 2011.
Daniel Kane. A pseudorandom generator for polynomial threshold functions of Gaussian with subpolynomial seed length. In Proceedings of the 29th Annual Conference on Computational Complexity (CCC), pages 217-228, 2014.
Daniel Kane, Raghu Meka, and Jelani Nelson. Almost optimal explicit Johnson-Lindenstrauss families. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques, pages 628-639. Springer, 2011.
Daniel M. Kane. A Polylogarithmic PRG for Degree 2 Threshold Functions in the Gaussian Setting. In 30th Conference on Computational Complexity, CCC 2015, pages 567-581, 2015.
Daniel M. Kane. A polylogarithmic PRG for degree 2 threshold functions in the Gaussian setting. In LIPIcs-Leibniz International Proceedings in Informatics, volume 33. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2015.
Pravesh K. Kothari and Raghu Meka. Almost Optimal Pseudorandom Generators for Spherical Caps. In Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing, pages 247-256, 2015.
H. Landau and L. Shepp. On the supremum of a Gaussian process. Sankhyā: The Indian Journal of Statistics, Series A, pages 369-378, 1970.
R. Meka. A polynomial time approximation scheme for computing the supremum of Gaussian processes. Ann. Appl. Probab., 25(2):465-476, April 2015.
Raghu Meka and David Zuckerman. Pseudorandom generators for polynomial threshold functions. SIAM Journal on Computing, 42(3):1275-1301, 2013.
Fedor Nazarov. On the Maximal Perimeter of a Convex Set in Rⁿ with Respect to a Gaussian Measure. In Geometric Aspects of Functional Analysis: Israel Seminar 2001-2002, pages 169-187, 2003.
R. O'Donnell, R.A. Servedio, and L.-Y. Tan. Fooling Polytopes. Available at https://arxiv.org/abs/1808.04035, to appear in STOC 2019, 2018.
https://arxiv.org/abs/1808.04035
Rocco A. Servedio and Li-Yang Tan. Deterministic Search for CNF Satisfying Assignments in Almost Polynomial Time. In 58th IEEE Annual Symposium on Foundations of Computer Science, FOCS 2017, Berkeley, CA, USA, October 15-17, 2017, pages 813-823. IEEE, 2017.
V. Sudakov. Geometric problems in the theory of infinite-dimensional probability distributions, volume 141. American Mathematical Soc., 1979.
M. Talagrand. Majorizing measures: the generic chaining. The Annals of Probability, pages 1049-1103, 1996.
Gregory Valiant and Paul Valiant. Estimating the unseen: an n/log(n)-sample estimator for entropy and support size, shown optimal via new CLTs. In Proceedings of the 43rd Symposium on Theory of Computing (STOC), pages 685-694, 2011.
Alex Zhai. A high-dimensional CLT in W2 distance with near optimal convergence rate. Probability Theory and Related Fields, 170(3):821-845, April 2018.
Eshan Chattopadhyay, Anindya De, and Rocco A. Servedio
Creative Commons Attribution 3.0 Unported license
https://creativecommons.org/licenses/by/3.0/legalcode