Document Open Access Logo

On Sketching the q to p Norms

Authors Aditya Krishnan, Sidhanth Mohanty, David P. Woodruff

PDF
Thumbnail PDF

File

LIPIcs.APPROX-RANDOM.2018.15.pdf
  • Filesize: 0.6 MB
  • 20 pages

Document Identifiers

Author Details

Aditya Krishnan
  • Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, USA
Sidhanth Mohanty
  • Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, USA
David P. Woodruff
  • Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, USA

Funding

D. Woodruff would like to acknowledge the support by the National Science Foundation under Grant No. CCF-1815840.

Cite AsGet BibTex

Aditya Krishnan, Sidhanth Mohanty, and David P. Woodruff. On Sketching the q to p Norms. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2018). Leibniz International Proceedings in Informatics (LIPIcs), Volume 116, pp. 15:1-15:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2018)
https://doi.org/10.4230/LIPIcs.APPROX-RANDOM.2018.15

Abstract

We initiate the study of data dimensionality reduction, or sketching, for the q -> p norms. Given an n x d matrix A, the q -> p norm, denoted |A |_{q -> p} = sup_{x in R^d \ 0} |Ax |_p / |x |_q, is a natural generalization of several matrix and vector norms studied in the data stream and sketching models, with applications to datamining, hardness of approximation, and oblivious routing. We say a distribution S on random matrices L in R^{nd} - > R^k is a (k,alpha)-sketching family if from L(A), one can approximate |A |_{q -> p} up to a factor alpha with constant probability. We provide upper and lower bounds on the sketching dimension k for every p, q in [1, infty], and in a number of cases our bounds are tight. While we mostly focus on constant alpha, we also consider large approximation factors alpha, as well as other variants of the problem such as when A has low rank.

Subject Classification

ACM Subject Classification
  • Theory of computation → Numeric approximation algorithms
Keywords
  • Dimensionality Reduction
  • Norms
  • Sketching
  • Streaming

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Related Versions

References

  1. Noga Alon, Yossi Matias, and Mario Szegedy. The space complexity of approximating the frequency moments. In Proceedings of the twenty-eighth annual ACM symposium on Theory of computing, pages 20-29. ACM, 1996. Google Scholar
  2. Alexandr Andoni. Nearest neighbor search in high-dimensional spaces. In the workshop: Barriers in Computational Complexity II, 2010. URL: http://www.mit.edu/~andoni/nns-barriers.pdf.
  3. Alexandr Andoni. High frequency moments via max-stability. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2017, New Orleans, LA, USA, March 5-9, 2017, pages 6364-6368, 2017. Google Scholar
  4. Alexandr Andoni et al. Eigenvalues of a matrix in the streaming model. In Proceedings of the twenty-fourth annual ACM-SIAM symposium on Discrete algorithms, pages 1729-1737. Society for Industrial and Applied Mathematics, 2013. Google Scholar
  5. Alexandr Andoni, T. S. Jayram, and Mihai Patrascu. Lower bounds for edit distance and product metrics via poincaré-type inequalities. In Proceedings of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2010, Austin, Texas, USA, January 17-19, 2010, pages 184-192, 2010. Google Scholar
  6. Alexandr Andoni, Robert Krauthgamer, and Krzysztof Onak. Streaming algorithms via precision sampling. In Foundations of Computer Science (FOCS), 2011 IEEE 52nd Annual Symposium on, pages 363-372. IEEE, 2011. Google Scholar
  7. Alexandr Andoni, Robert Krauthgamer, and Ilya P. Razenshteyn. Sketching and embedding are equivalent for norms. In Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing, STOC 2015, Portland, OR, USA, June 14-17, 2015, pages 479-488, 2015. Google Scholar
  8. Alexandr Andoni, Huy L Nguyen, Aleksandar Nikolov, Ilya Razenshteyn, and Erik Waingarten. Approximate near neighbors for general symmetric norms. In Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing, pages 902-913. ACM, 2017. Google Scholar
  9. Alexandr Andoni, Huy L Nguyên, Yury Polyanskiy, and Yihong Wu. Tight lower bound for linear sketches of moments. In International Colloquium on Automata, Languages, and Programming, pages 25-32. Springer, 2013. Google Scholar
  10. Ziv Bar-Yossef, Thathachar S Jayram, Ravi Kumar, and D Sivakumar. An information statistics approach to data stream and communication complexity. In Foundations of Computer Science, 2002. Proceedings. The 43rd Annual IEEE Symposium on, pages 209-218. IEEE, 2002. Google Scholar
  11. Boaz Barak, Fernando GSL Brandao, Aram W Harrow, Jonathan Kelner, David Steurer, and Yuan Zhou. Hypercontractivity, sum-of-squares proofs, and their applications. In Proceedings of the forty-fourth annual ACM symposium on Theory of computing, pages 307-326. ACM, 2012. Google Scholar
  12. Aditya Bhaskara and Aravindan Vijayaraghavan. Approximating matrix p-norms. In Proceedings of the twenty-second annual ACM-SIAM symposium on Discrete Algorithms, pages 497-511. SIAM, 2011. Google Scholar
  13. Jaroslaw Blasiok, Vladimir Braverman, Stephen R. Chestnut, Robert Krauthgamer, and Lin F. Yang. Streaming symmetric norms via measure concentration. In Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2017, Montreal, QC, Canada, June 19-23, 2017, pages 716-729, 2017. Google Scholar
  14. Fernando GSL Brandão and Aram W Harrow. Estimating operator norms using covering nets. arXiv preprint arXiv:1509.05065, 2015. Google Scholar
  15. V. Braverman, S. R. Chestnut, R. Krauthgamer, Y. Li, D. P. Woodruff, and L. F. Yang. Matrix Norms in Data Streams: Faster, Multi-Pass and Row-Order. ArXiv e-prints, 2016. URL: http://arxiv.org/abs/1609.05885.
  16. Jop Briët, Fernando Mário de Oliveira Filho, and Frank Vallentin. The positive semidefinite grothendieck problem with rank constraint. In Automata, Languages and Programming, 37th International Colloquium, ICALP 2010, Bordeaux, France, July 6-10, 2010, Proceedings, Part I, pages 31-42, 2010. Google Scholar
  17. Jop Briët, Oded Regev, and Rishi Saket. Tight hardness of the non-commutative grothendieck problem. Theory of Computing, 13(1):1-24, 2017. Google Scholar
  18. Kenneth L Clarkson and David P Woodruff. Low rank approximation and regression in input sparsity time. In Proceedings of the forty-fifth annual ACM symposium on Theory of computing, pages 81-90. ACM, 2013. Google Scholar
  19. Graham Cormode and S Muthukrishnan. Space efficient mining of multigraph streams. In Proceedings of the twenty-fourth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pages 271-282. ACM, 2005. Google Scholar
  20. Uffe Haagerup. The best constants in the khintchine inequality. Studia Mathematica, 70(3):231-283, 1981. Google Scholar
  21. Moritz Hardt and Eric Price. Tight bounds for learning a mixture of two gaussians. In Proceedings of the forty-seventh annual ACM symposium on Theory of computing, pages 753-760. ACM, 2015. Google Scholar
  22. Aram W Harrow, Ashley Montanaro, and Anthony J Short. Limitations on quantum dimensionality reduction. In International Colloquium on Automata, Languages, and Programming, pages 86-97. Springer, 2011. Google Scholar
  23. Piotr Indyk. Stable distributions, pseudorandom generators, embeddings, and data stream computation. Journal of the ACM (JACM), 53(3):307-323, 2006. Google Scholar
  24. Piotr Indyk and David Woodruff. Optimal approximations of the frequency moments of data streams. In Proceedings of the thirty-seventh annual ACM symposium on Theory of computing, pages 202-208. ACM, 2005. Google Scholar
  25. T. S. Jayram. On the information complexity of cascaded norms with small domains. In 2013 IEEE Information Theory Workshop, ITW 2013, Sevilla, Spain, September 9-13, 2013, pages 1-5, 2013. Google Scholar
  26. Thathachar S Jayram and David P Woodruff. The data stream space complexity of cascaded norms. In Foundations of Computer Science, 2009. FOCS'09. 50th Annual IEEE Symposium on, pages 765-774. IEEE, 2009. Google Scholar
  27. Daniel M Kane, Jelani Nelson, Ely Porat, and David P Woodruff. Fast moment estimation in data streams in optimal space. In Proceedings of the forty-third annual ACM symposium on Theory of computing, pages 745-754. ACM, 2011. Google Scholar
  28. Daniel M Kane, Jelani Nelson, and David P Woodruff. On the exact space complexity of sketching and streaming small norms. In Proceedings of the twenty-first annual ACM-SIAM symposium on Discrete Algorithms, pages 1161-1178. SIAM, 2010. Google Scholar
  29. Ashish Khetan and Sewoong Oh. Matrix norm estimation from a few entries. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pages 6427-6436, 2017. Google Scholar
  30. Subhash A Khot and Nisheeth K Vishnoi. The unique games conjecture, integrality gap for cut problems and embeddability of negative-type metrics into 𝓁₁. Journal of the ACM (JACM), 62(1):8, 2015. Google Scholar
  31. Weihao Kong and Gregory Valiant. Spectrum estimation from samples. CoRR, abs/1602.00061, 2016. Google Scholar
  32. Yi Li, Huy L Nguyên, and David P Woodruff. On sketching matrix norms and the top singular vector. In Proceedings of the twenty-fifth annual ACM-SIAM symposium on Discrete algorithms, pages 1562-1581. Society for Industrial and Applied Mathematics, 2014. Google Scholar
  33. Yi Li and David P. Woodruff. A tight lower bound for high frequency moment estimation with small error. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques - 16th International Workshop, APPROX 2013, and 17th International Workshop, RANDOM 2013, Berkeley, CA, USA, August 21-23, 2013. Proceedings, pages 623-638, 2013. Google Scholar
  34. Yi Li and David P. Woodruff. On approximating functions of the singular values in a stream. In Proceedings of the 48th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2016, Cambridge, MA, USA, June 18-21, 2016, pages 726-739, 2016. Google Scholar
  35. Yi Li and David P Woodruff. Tight bounds for sketching the operator norm, schatten norms, and subspace embeddings. In LIPIcs-Leibniz International Proceedings in Informatics, volume 60. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2016. Google Scholar
  36. Yi Li and David P. Woodruff. Embeddings of schatten norms with applications to data streams. In 44th International Colloquium on Automata, Languages, and Programming, ICALP 2017, July 10-14, 2017, Warsaw, Poland, pages 60:1-60:14, 2017. Google Scholar
  37. Jirı Matoušek. Lecture notes on metric embeddings. Technical report, Technical report, ETH Zürich, 2013. Google Scholar
  38. Cameron Musco, Praneeth Netrapalli, Aaron Sidford, Shashanka Ubaru, and David P. Woodruff. Spectrum approximation beyond fast matrix multiplication: Algorithms and hardness. In 9th Innovations in Theoretical Computer Science Conference, ITCS 2018, January 11-14, 2018, Cambridge, MA, USA, pages 8:1-8:21, 2018. Google Scholar
  39. Assaf Naor, Oded Regev, and Thomas Vidick. Efficient rounding for the noncommutative grothendieck inequality. Theory of Computing, 10:257-295, 2014. Google Scholar
  40. Eric Price and David P. Woodruff. Applications of the shannon-hartley theorem to data streams and sparse recovery. In Proceedings of the 2012 IEEE International Symposium on Information Theory, ISIT 2012, Cambridge, MA, USA, July 1-6, 2012, pages 2446-2450, 2012. Google Scholar
  41. Tamas Sarlos. Improved approximation algorithms for large matrices via random projections. In Foundations of Computer Science, 2006. FOCS'06. 47th Annual IEEE Symposium on, pages 143-152. IEEE, 2006. Google Scholar
  42. Zhao Song, David P. Woodruff, and Peilin Zhong. Low rank approximation with entrywise l_1-norm error. In Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2017, Montreal, QC, Canada, June 19-23, 2017, pages 688-701, 2017. Google Scholar
  43. Terence Tao. Topics in random matrix theory, volume 132. American Mathematical Society Providence, RI, 2012. Google Scholar
  44. Shashanka Ubaru, Jie Chen, and Yousef Saad. Fast estimation of tr(f(a)) via stochastic lanczos quadrature, 2016. URL: http://www-users.cs.umn.edu/~saad/PDF/ys-2016-04.pdf.
  45. Roman Vershynin. Introduction to the non-asymptotic analysis of random matrices. arXiv preprint arXiv:1011.3027, 2010. Google Scholar
  46. Andreas J. Winter. Quantum and classical message identification via quantum channels. Quantum Information & Computation, 5(7):605-606, 2005. Google Scholar
  47. David P. Woodruff. Sketching as a tool for numerical linear algebra. Foundations and Trends in Theoretical Computer Science, 10(1-2):1-157, 2014. Google Scholar
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact