On Solving Linear Systems in Sublinear Time

Authors Alexandr Andoni, Robert Krauthgamer, Yosef Pogrow

Thumbnail PDF


  • Filesize: 0.57 MB
  • 19 pages

Document Identifiers

Author Details

Alexandr Andoni
  • Columbia University, New York, NY, USA
Robert Krauthgamer
  • Weizmann Institute of Science, Rehovot, Israel
Yosef Pogrow
  • Weizmann Institute of Science, Rehovot, Israel

Cite AsGet BibTex

Alexandr Andoni, Robert Krauthgamer, and Yosef Pogrow. On Solving Linear Systems in Sublinear Time. In 10th Innovations in Theoretical Computer Science Conference (ITCS 2019). Leibniz International Proceedings in Informatics (LIPIcs), Volume 124, pp. 3:1-3:19, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2019)


We study sublinear algorithms that solve linear systems locally. In the classical version of this problem the input is a matrix S in R^{n x n} and a vector b in R^n in the range of S, and the goal is to output x in R^n satisfying Sx=b. For the case when the matrix S is symmetric diagonally dominant (SDD), the breakthrough algorithm of Spielman and Teng [STOC 2004] approximately solves this problem in near-linear time (in the input size which is the number of non-zeros in S), and subsequent papers have further simplified, improved, and generalized the algorithms for this setting. Here we focus on computing one (or a few) coordinates of x, which potentially allows for sublinear algorithms. Formally, given an index u in [n] together with S and b as above, the goal is to output an approximation x^_u for x^*_u, where x^* is a fixed solution to Sx=b. Our results show that there is a qualitative gap between SDD matrices and the more general class of positive semidefinite (PSD) matrices. For SDD matrices, we develop an algorithm that approximates a single coordinate x_{u} in time that is polylogarithmic in n, provided that S is sparse and has a small condition number (e.g., Laplacian of an expander graph). The approximation guarantee is additive | x^_u-x^*_u | <=epsilon | x^* |_infty for accuracy parameter epsilon>0. We further prove that the condition-number assumption is necessary and tight. In contrast to the SDD matrices, we prove that for certain PSD matrices S, the running time must be at least polynomial in n (for the same additive approximation), even if S has bounded sparsity and condition number.

Subject Classification

ACM Subject Classification
  • Theory of computation → Streaming, sublinear and near linear time algorithms
  • Linear systems
  • Laplacian solver
  • Sublinear time
  • Randomized linear algebra


  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    PDF Downloads


  1. José A. Adell and P. Jodrá. Exact Kolmogorov and total variation distances between some familiar discrete distributions. Journal of Inequalities and Applications, 2006(1):64307, 2006. URL: http://dx.doi.org/10.1155/JIA/2006/64307.
  2. Andris Ambainis. Variable time amplitude amplification and quantum algorithms for linear algebra problems. In 29th Symposium on Theoretical Aspects of Computer Science (STACS'12), volume 14, pages 636-647. LIPIcs, 2012. URL: http://dx.doi.org/10.4230/LIPIcs.STACS.2012.636.
  3. Reid Andersen, Christian Borgs, Jennifer T. Chayes, John E. Hopcroft, Vahab S. Mirrokni, and Shang-Hua Teng. Local computation of PageRank contributions. Internet Mathematics, 5(1):23-45, 2008. URL: http://dx.doi.org/10.1080/15427951.2008.10129302.
  4. Reid Andersen, Fan R. K. Chung, and Kevin J. Lang. Using PageRank to locally partition a graph. Internet Mathematics, 4(1):35-64, 2007. URL: http://dx.doi.org/10.1080/15427951.2007.10129139.
  5. Reid Andersen and Yuval Peres. Finding sparse cuts locally using evolving sets. In Proceedings of the Forty-first Annual ACM Symposium on Theory of Computing, STOC '09, pages 235-244. ACM, 2009. URL: http://dx.doi.org/10.1145/1536414.1536449.
  6. Christian Borgs, Michael Brautbar, Jennifer T. Chayes, and Shang-Hua Teng. Multiscale matrix sampling and sublinear-time PageRank computation. Internet Mathematics, 10(1-2):20-48, 2014. URL: http://dx.doi.org/10.1080/15427951.2013.802752.
  7. Marco Bressan, Enoch Peserico, and Luca Pretto. Brief announcement: On approximating PageRank locally with sublinear query complexity. In 30th on Symposium on Parallelism in Algorithms and Architectures, SPAA '18, pages 87-89. ACM, 2018. http://arxiv.org/abs/1404.1864, URL: http://dx.doi.org/10.1145/3210377.3210664.
  8. Sergey Brin and Lawrence Page. The anatomy of a large-scale hypertextual web search engine. Computer Networks, 30(1-7):107-117, 1998. URL: http://dx.doi.org/10.1016/S0169-7552(98)00110-X.
  9. Clément L. Canonne. A survey on distribution testing: Your data is big. but is it blue? Electronic Colloquium on Computational Complexity (ECCC), 22:63, 2015. URL: http://eccc.hpi-web.de/report/2015/063.
  10. Yen-Yu Chen, Qingqing Gan, and Torsten Suel. Local methods for estimating PageRank values. In Proceedings of the Thirteenth ACM International Conference on Information and Knowledge Management, CIKM '04, pages 381-389. ACM, 2004. URL: http://dx.doi.org/10.1145/1031171.1031248.
  11. A. Childs, R. Kothari, and R. Somma. Quantum algorithm for systems of linear equations with exponentially improved dependence on precision. SIAM Journal on Computing, 46(6):1920-1950, 2017. URL: http://dx.doi.org/10.1137/16M1087072.
  12. Fan Chung and Olivia Simpson. Solving local linear systems with boundary conditions using heat kernel PageRank. Internet Mathematics, 11(4-5):449-471, 2015. URL: http://dx.doi.org/10.1080/15427951.2015.1009522.
  13. Fan Chung and Wenbo Zhao. PageRank and Random Walks on Graphs, pages 43-62. Springer, 2010. URL: http://dx.doi.org/10.1007/978-3-642-13580-4_3.
  14. Michael B. Cohen, Rasmus Kyng, Gary L. Miller, Jakub W. Pachocki, Richard Peng, Anup B. Rao, and Shen Chen Xu. Solving SDD linear systems in nearly m log^1/2 n time. In Proceedings of the 46th Annual ACM Symposium on Theory of Computing, pages 343-352, 2014. URL: http://dx.doi.org/10.1145/2591796.2591833.
  15. Danial Dervovic, Mark Herbster, Peter Mountney, Simone Severini, Naïri Usher, and Leonard Wossnig. Quantum linear systems algorithms: a primer. CoRR, abs/1802.08227, 2018. URL: http://arxiv.org/abs/1802.08227.
  16. Dean Doron, François Le Gall, and Amnon Ta-Shma. Probabilistic logarithmic-space algorithms for Laplacian solvers. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques, APPROX/RANDOM 2017, pages 41:1-41:20, 2017. URL: http://dx.doi.org/10.4230/LIPIcs.APPROX-RANDOM.2017.41.
  17. Dean Doron, Amir Sarid, and Amnon Ta-Shma. On approximating the eigenvalues of stochastic matrices in probabilistic logspace. Comput. Complex., 26(2):393-420, June 2017. URL: http://dx.doi.org/10.1007/s00037-016-0150-y.
  18. George E Forsythe and Richard A Leibler. Matrix inversion by a Monte Carlo method. Mathematics of Computation, 4(31):127-129, 1950. URL: http://dx.doi.org/10.1090/S0025-5718-1950-0038138-X.
  19. François Le Gall. Powers of tensors and fast matrix multiplication. In International Symposium on Symbolic and Algebraic Computation, ISSAC '14, pages 296-303, 2014. URL: http://dx.doi.org/10.1145/2608628.2608664.
  20. Aram W. Harrow, Avinatan Hassidim, and Seth Lloyd. Quantum algorithm for linear systems of equations. Phys. Rev. Lett., 103:150502, Oct 2009. URL: http://dx.doi.org/10.1103/PhysRevLett.103.150502.
  21. Shlomo Hoory, Nathan Linial, and Avi Wigderson. Expander graphs and their applications. Bull. Amer. Math. Soc., 43(4):439-561, 2006. URL: http://dx.doi.org/10.1090/S0273-0979-06-01126-8.
  22. Iordanis Kerenidis and Anupam Prakash. Quantum recommendation systems. In 8th Innovations in Theoretical Computer Science Conference (ITCS' 17), volume 67 of Leibniz International Proceedings in Informatics (LIPIcs), pages 49:1-49:21. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2017. URL: http://dx.doi.org/10.4230/LIPIcs.ITCS.2017.49.
  23. Bo'az Klartag and Oded Regev. Quantum one-way communication can be exponentially stronger than classical communication. In 43rd Annual ACM Symposium on Theory of Computing, STOC '11, pages 31-40, 2011. URL: http://dx.doi.org/10.1145/1993636.1993642.
  24. Rasmus Kyng, Yin Tat Lee, Richard Peng, Sushant Sachdeva, and Daniel A Spielman. Sparsified Cholesky and multigrid solvers for connection Laplacians. In 48th Annual ACM Symposium on Theory of Computing, pages 842-850. ACM, 2016. Google Scholar
  25. Yin Tat Lee. Probabilistic spectral sparsification in sublinear time. CoRR, abs/1401.0085, 2014. URL: http://arxiv.org/abs/1401.0085.
  26. Alexander Lubotzky, Ralph Phillips, and Peter Sarnak. Ramanujan graphs. Combinatorica, 8(3):261-277, 1988. URL: http://dx.doi.org/10.1007/BF02126799.
  27. G. A. Margulis. Explicit group-theoretic constructions of combinatorial schemes and their applications in the construction of expanders and concentrators. Problemy Peredachi Informatsii, 24(1):51-60, 1988. Google Scholar
  28. Ran Raz. Exponential separation of quantum and classical communication complexity. In 31st Annual ACM Symposium on Theory of Computing, STOC '99, pages 358-367, 1999. URL: http://dx.doi.org/10.1145/301250.301343.
  29. Ronitt Rubinfeld, Gil Tamir, Shai Vardi, and Ning Xie. Fast local computation algorithms. In Innovations in Computer Science - ICS 2010, pages 223-238, 2011. URL: http://conference.iiis.tsinghua.edu.cn/ICS2011/content/papers/36.html.
  30. Nitin Shyamkumar, Siddhartha Banerjee, and Peter Lofgren. Sublinear estimation of a single element in sparse linear systems. In 54th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2016, pages 856-860, 2016. URL: http://dx.doi.org/10.1109/ALLERTON.2016.7852323.
  31. D. A. Spielman and S.-H. Teng. Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems. In 36th Annual ACM Symposium on Theory of Computing, pages 81-90. ACM, 2004. URL: http://dx.doi.org/10.1145/1007352.1007372.
  32. Daniel A. Spielman. Algorithms, graph theory, and linear equations in Laplacian matrices. In Proceedings of the International Congress of Mathematicians, volume 4, pages 2698-2722, 2010. URL: http://dx.doi.org/10.1142/9789814324359_0164.
  33. Daniel A. Spielman and Nikhil Srivastava. Graph sparsification by effective resistances. SIAM J. Comput., 40(6):1913-1926, December 2011. URL: http://dx.doi.org/10.1137/080734029.
  34. Jukka Suomela. Survey of local algorithms. ACM Comput. Surv., 45(2):24:1-24:40, March 2013. URL: http://dx.doi.org/10.1145/2431211.2431223.
  35. Ewin Tang. A quantum-inspired classical algorithm for recommendation systems. CoRR, abs/1807.04271, 2018. URL: http://arxiv.org/abs/1807.04271.
  36. Nisheeth K. Vishnoi. Lx = b. Foundations and Trends in Theoretical Computer Science, 8(1-2):1-141, 2013. URL: http://dx.doi.org/10.1561/0400000054.
  37. Alastair J. Walker. An efficient method for generating discrete random variables with general distributions. ACM Trans. Math. Softw., 3(3):253-256, 1977. URL: http://dx.doi.org/10.1145/355744.355749.
  38. Wolfgang R Wasow. A note on the inversion of matrices by random walks. Mathematical Tables and Other Aids to Computation, 6(38):78-81, 1952. URL: http://dx.doi.org/10.2307/2002546.
Questions / Remarks / Feedback

Feedback for Dagstuhl Publishing

Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail