Document Open Access Logo

Vector-Matrix-Vector Queries for Solving Linear Algebra, Statistics, and Graph Problems

Authors Cyrus Rashtchian, David P. Woodruff, Hanlin Zhu

PDF

File

Thumbnail PDF
PDF
LIPIcs.APPROX-RANDOM.2020.26.pdf
  • Filesize: 0.58 MB
  • 20 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Theory of computation → Stochastic approximation
Keywords
  • Query complexity
  • property testing
  • vector-matrix-vector
  • linear algebra
  • statistics
  • graph parameter estimation

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

We consider the general problem of learning about a matrix through vector-matrix-vector queries. These queries provide the value of u^{T}Mv over a fixed field 𝔽 for a specified pair of vectors u,v ∈ 𝔽ⁿ. To motivate these queries, we observe that they generalize many previously studied models, such as independent set queries, cut queries, and standard graph queries. They also specialize the recently studied matrix-vector query model. Our work is exploratory and broad, and we provide new upper and lower bounds for a wide variety of problems, spanning linear algebra, statistics, and graphs. Many of our results are nearly tight, and we use diverse techniques from linear algebra, randomized algorithms, and communication complexity.

Cite As Get BibTex

Cyrus Rashtchian, David P. Woodruff, and Hanlin Zhu. Vector-Matrix-Vector Queries for Solving Linear Algebra, Statistics, and Graph Problems. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2020). Leibniz International Proceedings in Informatics (LIPIcs), Volume 176, pp. 26:1-26:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2020) https://doi.org/10.4230/LIPIcs.APPROX/RANDOM.2020.26

Author Details

Cyrus Rashtchian
  • Department of Computer Science & Engineering, UC San Diego, CA, USA
David P. Woodruff
  • Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, USA
Hanlin Zhu
  • Institute for Interdisciplinary Information Sciences, Tsinghua University, Beijing, China

Funding

  • Woodruff, David P.: D. Woodruff would like to thank support in part by the Office of Naval Research (ONR) grant N00014-18-1-2562.

References

  1. Hasan Abasi and Bshouty Nader. On Learning Graphs with Edge-Detecting Queries. In Algorithmic Learning Theory, pages 3-30, 2019. Google Scholar
  2. Maryam Aliakbarpour, Amartya Shankha Biswas, Themis Gouleakis, John Peebles, Ronitt Rubinfeld, and Anak Yodpinyanee. Sublinear-time algorithms for counting star subgraphs via edge sampling. Algorithmica, 80(2):668-697, 2018. URL: https://doi.org/10.1007/s00453-017-0287-3.
  3. Noga Alon and Vera Asodi. Learning a Hidden Subgraph. SIAM Journal on Discrete Mathematics, 18(4):697-712, 2005. Google Scholar
  4. Noga Alon, Richard Beigel, Simon Kasif, Steven Rudich, and Benny Sudakov. Learning a hidden matching. SIAM Journal on Computing, 33(2):487-501, 2004. Google Scholar
  5. Sepehr Assadi, Michael Kapralov, and Sanjeev Khanna. A Simple Sublinear-Time Algorithm for Counting Arbitrary Subgraphs via Edge Sampling. In Proc. 10th Innovations in Theoretical Computer Science Conference (ITCS), 2019. Google Scholar
  6. Sepehr Assadi, Sanjeev Khanna, and Yang Li. On estimating maximum matching size in graph streams. In Proceedings of the Twenty-Eighth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1723-1742. SIAM, 2017. Google Scholar
  7. Ziv Bar-Yossef, Thathachar S Jayram, Ravi Kumar, and D Sivakumar. An information statistics approach to data stream and communication complexity. Journal of Computer and System Sciences, 68(4):702-732, 2004. Google Scholar
  8. Ziv Bar-Yossef, Ravi Kumar, and D Sivakumar. Reductions in streaming algorithms, with an application to counting triangles in graphs. In Proceedings of the thirteenth annual ACM-SIAM symposium on Discrete algorithms, pages 623-632. Society for Industrial and Applied Mathematics, 2002. Google Scholar
  9. P. Beame, S. Har-Peled, S. Natarajan Ramamoorthy, C. Rashtchian, and M. Sinha. Edge estimation with independent set oracles. In Anna R. Karlin, editor, Proc. Innov. Theo. Comp. Sci. (ITCS), volume 94 of LIPIcs, pages 38:1-38:21, 2018. Google Scholar
  10. I. Ben-Eliezer, T. Kaufman, M. Krivelevich, and D. Ron. Comparing the strength of query types in property testing: The case of k-colorability. Computational Complexity, 22(1):89-135, 2013. URL: https://doi.org/10.1007/s00037-012-0048-2.
  11. Anup Bhattacharya, Arijit Bishnu, Arijit Ghosh, and Gopinath Mishra. Hyperedge estimation using polylogarithmic subset queries. CoRR, abs/1908.04196, 2019. URL: http://arxiv.org/abs/1908.04196.
  12. Anup Bhattacharya, Arijit Bishnu, Arijit Ghosh, and Gopinath Mishra. Triangle estimation using tripartite independent set queries. In Pinyan Lu and Guochuan Zhang, editors, Proc. 30th Annu. Internat. Sympos. Algorithms Comput. (ISAAC), volume 149 of LIPIcs, pages 19:1-19:17. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2019. URL: https://doi.org/10.4230/LIPIcs.ISAAC.2019.19.
  13. Mark Braverman, Elad Hazan, Max Simchowitz, and Blake E. Woodworth. The gradient complexity of linear regression. CoRR, abs/1911.02212, 2019. URL: http://arxiv.org/abs/1911.02212.
  14. Clément L Canonne. A survey on distribution testing: your data is big. but is it blue? In Electronic Colloquium on Computational Complexity (ECCC), volume 22:63, pages 1-1, 2015. Google Scholar
  15. Diptarka Chakraborty, Lior Kamma, and Kasper Green Larsen. Tight Cell Probe Bounds for Succinct Boolean Matrix-Vector Multiplication. In Proc. 50th Annual ACM SIGACT Symposium on Theory of Computing (STOC), pages 1297-1306, 2018. Google Scholar
  16. Arkadev Chattopadhyay, Michal Koucký, Bruno Loff, and Sagnik Mukhopadhyay. Simulation Beats Richness: New Data-Structure Lower Bounds. In Proc. 50th Annual ACM SIGACT Symposium on Theory of Computing (STOC), pages 1013-1020, 2018. Google Scholar
  17. Xi Chen, Amit Levi, and Erik Waingarten. Nearly optimal edge estimation with independent set queries. In Shuchi Chawla, editor, Proc. 31st ACM-SIAM Sympos. Discrete Algs. (SODA), pages 2916-2935. SIAM, 2020. URL: https://doi.org/10.1137/1.9781611975994.177.
  18. Holger Dell and John Lapinskas. Fine-grained reductions from approximate counting to decision. In Ilias Diakonikolas, David Kempe, and Monika Henzinger, editors, Proc. 50th Annu. ACM Sympos. Theory Comput. (STOC), pages 281-288. ACM, 2018. URL: https://doi.org/10.1145/3188745.3188920.
  19. Holger Dell, John Lapinskas, and Kitty Meeks. Approximately counting and sampling small witnesses using a colourful decision oracle. In Shuchi Chawla, editor, Proc. 31st ACM-SIAM Sympos. Discrete Algs. (SODA), pages 2201-2211. SIAM, 2020. URL: https://doi.org/10.1137/1.9781611975994.135.
  20. Zeev Dvir, Alexander Golovnev, and Omri Weinstein. Static Data Structure Lower Bounds Imply Rigidity. In Proc. 51st Annual ACM SIGACT Symp. on Theory of Computing (STOC), pages 967-978, 2019. Google Scholar
  21. T. Eden, A. Levi, D. Ron, and C. Seshadhri. Approximately counting triangles in sublinear time. SIAM J. Comput., 46(5):1603-1646, 2017. URL: https://doi.org/10.1137/15M1054389.
  22. T. Eden, D. Ron, and C. Seshadhri. On Approximating the Number of k-cliques in Sublinear Time. CoRR, abs/1707.04858, July 2017. URL: http://arxiv.org/abs/1707.04858.
  23. Yonina C Eldar and Gitta Kutyniok. Compressed sensing: theory and applications. Cambridge university press, 2012. Google Scholar
  24. U. Feige. On sums of independent random variables with unbounded variance and estimating the average degree in a graph. SIAM J. Comput., 35(4):964-984, 2006. URL: https://doi.org/10.1137/s0097539704447304.
  25. O. Goldreich and D. Ron. Approximating average parameters of graphs. Random Struct. Algo., 32(4):473-493, 2008. URL: https://doi.org/10.1002/rsa.v32:4.
  26. Oded Goldreich. Introduction to property testing. Cambridge University Press, 2017. Google Scholar
  27. M. Gonen, D. Ron, and Y. Shavitt. Counting stars and other small subgraphs in sublinear-time. SIAM J. Discrete Math, 25(3):1365-1411, 2011. URL: https://doi.org/10.1137/100783066.
  28. Johan Håstad and Avi Wigderson. The randomized communication complexity of set disjointness. Theory of Computing, 3(1):211-219, 2007. Google Scholar
  29. Bala Kalyanasundaram and Georg Schintger. The probabilistic communication complexity of set intersection. SIAM Journal on Discrete Mathematics, 5(4):545-557, 1992. Google Scholar
  30. Kasper Green Larsen and Ryan Williams. Faster Online Matrix-Vector Multiplication. In Proc. 28th ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 2182-2189, 2017. Google Scholar
  31. Yi Li, Huy L Nguyen, and David P Woodruff. On approximating matrix norms in data streams. SIAM Journal on Computing, 48(6):1643-1697, 2019. Google Scholar
  32. Yi Li, Xiaoming Sun, Chengu Wang, and David P Woodruff. On the communication complexity of linear algebraic problems in the message passing model. In International Symposium on Distributed Computing, pages 499-513. Springer, 2014. Google Scholar
  33. Yi Li and David P. Woodruff. Tight bounds for sketching the operator norm, schatten norms, and subspace embeddings. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques, APPROX/RANDOM 2016, September 7-9, 2016, Paris, France, pages 39:1-39:11, 2016. Google Scholar
  34. Marco Molinaro, David P Woodruff, and Grigory Yaroslavtsev. Beating the direct sum theorem in communication complexity with implications for sketching. In Proceedings of the twenty-fourth annual ACM-SIAM symposium on Discrete algorithms, pages 1738-1756. SIAM, 2013. Google Scholar
  35. Sagnik Mukhopadhyay and Danupon Nanongkai. Weighted Min-Cut: Sequential, Cut-Query and Streaming Algorithms. In Proc. 52nd Annual ACM Symposium on Theory of Computing (STOC), 2020. Google Scholar
  36. Sivaramakrishnan Natarajan Ramamoorthy and Cyrus Rashtchian. Equivalence of Systematic Linear Data Structures and Matrix Rigidity. In Proc. 11th Innovations in Theoretical Computer Science Conference (ITCS), 2020. Google Scholar
  37. K. Onak, D. Ron, M. Rosen, and R. Rubinfeld. A near-optimal sublinear-time algorithm for approximating the minimum vertex cover size. In Proc. 23rd ACM-SIAM Sympos. Discrete Algs. (SODA), pages 1123-1131, 2012. URL: http://dl.acm.org/citation.cfm?id=2095116.2095204.
  38. AA Razborov. On the distributional complexity of disjointness. Theoretical Computer Science, 106:385-390, 1992. Google Scholar
  39. Aviad Rubinstein, Tselil Schramm, and S Matthew Weinberg. Computing exact minimum cuts without knowing the graph. In Proc. 9th Innovations in Theoretical Computer Science Conference (ITCS), 2018. Google Scholar
  40. C. Seshadhri. A simpler sublinear algorithm for approximating the triangle count. CoRR, abs/1505.01927, May 2015. URL: http://arxiv.org/abs/1505.01927.
  41. Max Simchowitz, Ahmed El Alaoui, and Benjamin Recht. Tight query complexity lower bounds for PCA via finite sample deformed wigner law. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2018, Los Angeles, CA, USA, June 25-29, 2018, pages 1249-1259, 2018. Google Scholar
  42. Xiaoming Sun, David P Woodruff, Guang Yang, and Jialin Zhang. Querying a matrix through matrix-vector products. In 46th International Colloquium on Automata, Languages, and Programming (ICALP 2019), 2019. Google Scholar
  43. Dan Wang and Zhu Han. Sublinear algorithms for big data applications. Springer, 2015. Google Scholar
  44. J. Wang, E. Lo, and M. L. Yiu. Identifying the most connected vertices in hidden bipartite graphs using group testing. IEEE Tran. Knowl. Data Eng., 25(10):2245-2256, 2013. URL: https://doi.org/10.1109/TKDE.2012.178.
  45. David P. Woodruff. Sketching as a tool for numerical linear algebra. Foundations and Trendsregistered in Theoretical Computer Science, 10(1-2):1-157, 2014. Google Scholar
  46. Andrew Chi-Chin Yao. Probabilistic computations: Toward a unified measure of complexity. In 18th Annual Symposium on Foundations of Computer Science, pages 222-227. IEEE, 1977. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact