Document Open Access Logo

Deterministic Complexity Analysis of Hermitian Eigenproblems

Author Aleksandros Sobczyk

PDF

File

Thumbnail PDF
PDF
LIPIcs.ICALP.2025.131.pdf
  • Filesize: 0.93 MB
  • 21 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Divide and conquer
  • Theory of computation → Numeric approximation algorithms
Keywords
  • Hermitian eigenproblem
  • eigenvalues
  • SVD
  • tridiagonal reduction
  • matrix multiplication time
  • diagonalization
  • bit complexity

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

In this work we revisit the arithmetic and bit complexity of Hermitian eigenproblems. Recently, [BGVKS, FOCS 2020] proved that a (non-Hermitian) matrix A can be diagonalized with a randomized algorithm in O(n^{ω}log²(n/ε)) arithmetic operations, where ω≲ 2.371 is the square matrix multiplication exponent, and [Shah, SODA 2025] significantly improved the bit complexity for the Hermitian case. Our main goal is to obtain similar deterministic complexity bounds for various Hermitian eigenproblems. In the Real RAM model, we show that a Hermitian matrix can be diagonalized deterministically in O(n^{ω}log(n)+n²polylog(n/ε)) arithmetic operations, improving the classic deterministic Õ(n³) algorithms, and derandomizing the aforementioned state-of-the-art. The main technical step is a complete, detailed analysis of a well-known divide-and-conquer tridiagonal eigensolver of Gu and Eisenstat [GE95], when accelerated with the Fast Multipole Method, asserting that it can accurately diagonalize a symmetric tridiagonal matrix in nearly-O(n²) operations. In finite precision, we show that an algorithm by Schönhage [Sch72] to reduce a Hermitian matrix to tridiagonal form is stable in the floating point model, using O(log(n/ε)) bits of precision. This leads to a deterministic algorithm to compute all the eigenvalues of a Hermitian matrix in O(n^{ω}ℱ(log(n/ε)) + n²polylog(n/ε)) bit operations, where ℱ(b) ∈ Õ(b) is the bit complexity of a single floating point operation on b bits. This improves the best known Õ(n³) deterministic and O(n^{ω}log²(n/ε)ℱ(log(n/ε))) randomized complexities. We conclude with some other useful subroutines such as computing spectral gaps, condition numbers, and spectral projectors, and with some open problems.

Cite As Get BibTex

Aleksandros Sobczyk. Deterministic Complexity Analysis of Hermitian Eigenproblems. In 52nd International Colloquium on Automata, Languages, and Programming (ICALP 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 334, pp. 131:1-131:21, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.ICALP.2025.131

Author Details

Aleksandros Sobczyk
  • IBM Research Europe, Rüschlikon, Switzerland
  • ETH Zurich, Zurich, Switzerland

Acknowledgements

I am grateful to Efstratios Gallopoulos, Daniel Kressner, and David Woodruff for helpful discussions.

References

  1. Josh Alman, Ran Duan, Virginia Vassilevska Williams, Yinzhan Xu, Zixuan Xu, and Renfei Zhou. More asymmetry yields faster matrix multiplication. In Proc. 2025 ACM-SIAM Symposium on Discrete Algorithms, pages 2005-2039. SIAM, 2025. URL: https://doi.org/10.1137/1.9781611978322.63.
  2. Alexandr Andoni and Huy L Nguyen. Eigenvalues of a matrix in the streaming model. In Proc. 24th ACM-SIAM Symposium on Discrete Algorithms, pages 1729-1737. SIAM, 2013. URL: https://doi.org/10.1137/1.9781611973105.124.
  3. Diego Armentano, Carlos Beltrán, Peter Bürgisser, Felipe Cucker, and Michael Shub. A stable, polynomial-time algorithm for the eigenpair problem. Journal of the European Mathematical Society, 20(6):1375-1437, 2018. Google Scholar
  4. Grey Ballard, James Demmel, and Nicholas Knight. Communication avoiding successive band reduction. ACM SIGPLAN Notices, 47(8):35-44, 2012. URL: https://doi.org/10.1145/2145816.2145822.
  5. Grey Ballard, James Demmel, and Nicholas Knight. Avoiding communication in successive band reduction. ACM Transactions on Parallel Computing, 1(2):1-37, 2015. URL: https://doi.org/10.1145/2686877.
  6. Jess Banks, Jorge Garza-Vargas, Archit Kulkarni, and Nikhil Srivastava. Pseudospectral Shattering, the Sign Function, and Diagonalization in Nearly Matrix Multiplication Time. Foundations of Computational Mathematics, pages 1-89, 2022. Google Scholar
  7. Jess Banks, Jorge Garza-Vargas, and Nikhil Srivastava. Global Convergence of Hessenberg Shifted QR II: Numerical Stability. arXiv, 2022. URL: https://doi.org/10.48550/arXiv.2205.06810.
  8. Jess Banks, Jorge Garza-Vargas, and Nikhil Srivastava. Global convergence of Hessenberg Shifted QR I: Exact Arithmetic. Foundations of Computational Mathematics, pages 1-34, 2024. Google Scholar
  9. Jess Banks, Jorge Garza-Vargas, and Nikhil Srivastava. Global Convergence of Hessenberg Shifted QR III: Approximate Ritz Values via Shifted Inverse Iteration. SIAM Journal on Matrix Analysis and Applications, 46(2):1212-1246, 2025. Google Scholar
  10. Jesse L Barlow. Error analysis of update methods for the symmetric eigenvalue problem. SIAM Journal on Matrix Analysis and Applications, 14(2):598-618, 1993. URL: https://doi.org/10.1137/0614042.
  11. Michael Ben-Or and Lior Eldar. A Quasi-Random Approach to Matrix Spectral Analysis. In Proc. 9th Innovations in Theoretical Computer Science Conference, pages 6:1-6:22. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2018. URL: https://doi.org/10.4230/LIPICS.ITCS.2018.6.
  12. Rajendra Bhatia. Perturbation bounds for matrix eigenvalues. SIAM, 2007. Google Scholar
  13. Dario Bini and Victor Pan. Parallel complexity of tridiagonal symmetric eigenvalue problem. In Proc. 2nd ACM-SIAM Symposium on Discrete Algorithms, pages 384-393, 1991. URL: http://dl.acm.org/citation.cfm?id=127787.127855.
  14. Dario Bini and Victor Pan. Practical improvement of the divide-and-conquer eigenvalue algorithms. Computing, 48(1):109-123, 1992. URL: https://doi.org/10.1007/BF02241709.
  15. Dario Bini and Victor Y Pan. Computing matrix eigenvalues and polynomial zeros where the output is real. SIAM Journal on Computing, 27(4):1099-1115, 1998. URL: https://doi.org/10.1137/S0097539790182482.
  16. Christian H Bischof, Bruno Lang, and Xiaobai Sun. A framework for symmetric band reduction. ACM Transactions on Mathematical Software, 26(4):581-601, 2000. URL: https://doi.org/10.1145/365723.365735.
  17. D Boley and Gene Howard Golub. Inverse eigenvalue problems for band matrices. In Numerical Analysis: Proceedings of the Biennial Conference Held at Dundee, June 28-July 1, 1977, pages 23-31. Springer, 1977. Google Scholar
  18. Christos Boutsidis and David P Woodruff. Optimal CUR matrix decompositions. In Proc. 46th ACM Symposium on Theory of Computing, pages 353-362, 2014. URL: https://doi.org/10.1145/2591796.2591819.
  19. Christos Boutsidis, David P Woodruff, and Peilin Zhong. Optimal principal component analysis in distributed and streaming models. In Proc. 48th ACM Symposium on Theory of Computing, pages 236-249, 2016. URL: https://doi.org/10.1145/2897518.2897646.
  20. Christos Boutsidis, Anastasios Zouzias, Michael W Mahoney, and Petros Drineas. Randomized dimensionality reduction for k-means clustering. IEEE Transactions on Information Theory, 61(2):1045-1062, 2014. URL: https://doi.org/10.1109/TIT.2014.2375327.
  21. Difeng Cai and Jianlin Xia. A stable matrix version of the fast multipole method: stabilization strategies and examples. ETNA-Electronic Transactions on Numerical Analysis, 54, 2020. Google Scholar
  22. Kenneth L Clarkson and David P Woodruff. Low-rank approximation and regression in input sparsity time. Journal of the ACM, 63(6):1-45, 2017. Google Scholar
  23. Michael B Cohen, Sam Elder, Cameron Musco, Christopher Musco, and Madalina Persu. Dimensionality reduction for k-means clustering and low rank approximation. In Proc. 47th ACM Symposium on Theory of Computing, pages 163-172, 2015. URL: https://doi.org/10.1145/2746539.2746569.
  24. Jan JM Cuppen. A divide and conquer method for the symmetric tridiagonal eigenproblem. Numerische Mathematik, 36:177-195, 1980. Google Scholar
  25. TJ Dekker and JF Traub. The shifted QR algorithm for Hermitian matrices. Linear Algebra and its Applications, 4(3):137-154, 1971. Google Scholar
  26. James Demmel, Ioana Dumitriu, and Olga Holtz. Fast linear algebra is stable. Numerische Mathematik, 108(1):59-91, 2007. URL: https://doi.org/10.1007/S00211-007-0114-X.
  27. James Demmel, Ioana Dumitriu, Olga Holtz, and Robert Kleinberg. Fast matrix multiplication is stable. Numerische Mathematik, 106(2):199-224, 2007. URL: https://doi.org/10.1007/S00211-007-0061-6.
  28. James Demmel, Ioana Dumitriu, and Ryan Schneider. Fast and inverse-free algorithms for deflating subspaces. arXiv, 2024. URL: https://arxiv.org/abs/2310.00193.
  29. James Demmel, Ioana Dumitriu, and Ryan Schneider. Generalized Pseudospectral Shattering and Inverse-Free Matrix Pencil Diagonalization. Foundations of Computational Mathematics, pages 1-77, 2024. Google Scholar
  30. James Demmel and Krešimir Veselić. Jacobi’s method is more accurate than QR. SIAM Journal on Matrix Analysis and Applications, 13(4):1204-1245, 1992. URL: https://doi.org/10.1137/0613074.
  31. James W Demmel. Applied numerical linear algebra. SIAM, 1997. URL: https://doi.org/10.1137/1.9781611971446.
  32. Inderjit Singh Dhillon. A new O(N²) algorithm for the symmetric tridiagonal eigenvalue/eigenvector problem. University of California, Berkeley, 1997. Google Scholar
  33. Jack Dongarra and Francis Sullivan. Guest Editors Introduction to the top 10 algorithms. Computing in Science & Engineering, 2(01):22-23, 2000. URL: https://doi.org/10.1109/MCISE.2000.814652.
  34. Jack J Dongarra and Danny C Sorensen. A fully parallel algorithm for the symmetric eigenvalue problem. SIAM Journal on Scientific and Statistical Computing, 8(2):s139-s154, 1987. Google Scholar
  35. Petros Drineas, Michael W Mahoney, and Shan Muthukrishnan. Relative-error CUR matrix decompositions. SIAM Journal on Matrix Analysis and Applications, 30(2):844-881, 2008. URL: https://doi.org/10.1137/07070471X.
  36. Jeff Erickson, Ivor Van Der Hoog, and Tillmann Miltzow. Smoothing the gap between NP and ER. SIAM Journal on Computing, 0(0):FOCS20-102-FOCS20-138, 2022. Google Scholar
  37. John GF Francis. The QR transformation a unitary analogue to the LR transformation—Part 1. The Computer Journal, 4(3):265-271, 1961. URL: https://doi.org/10.1093/COMJNL/4.3.265.
  38. John GF Francis. The QR transformation—Part 2. The Computer Journal, 4(4):332-345, 1962. Google Scholar
  39. Alan Frieze, Ravi Kannan, and Santosh Vempala. Fast Monte-Carlo algorithms for finding low-rank approximations. Journal of the ACM, 51(6):1025-1041, 2004. URL: https://doi.org/10.1145/1039488.1039494.
  40. Martin Fürer. Faster integer multiplication. In Proc. 39th ACM Symposium on Theory of Computing, pages 57-66, 2007. URL: https://doi.org/10.1145/1250790.1250800.
  41. Doron Gill and Eitan Tadmor. An O(N²) Method for Computing the Eigensystem of N× N Symmetric Tridiagonal Matrices by the Divide and Conquer Approach. SIAM Journal on Scientific and Statistical Computing, 11(1):161-173, 1990. URL: https://doi.org/10.1137/0911010.
  42. Gene Golub and William Kahan. Calculating the singular values and pseudo-inverse of a matrix. Journal of the Society for Industrial and Applied Mathematics, Series B: Numerical Analysis, 2(2):205-224, 1965. Google Scholar
  43. Gene H Golub and Henk A Van der Vorst. Eigenvalue computation in the 20th century. Journal of Computational and Applied Mathematics, 123(1-2):35-65, 2000. Google Scholar
  44. Gene H Golub and Charles F Van Loan. Matrix Computations. Johns Hopkins University Press, 2013. Google Scholar
  45. Ming Gu and Stanley C Eisenstat. A stable and fast algorithm for updating the singular value decomposition, 1993. Google Scholar
  46. Ming Gu and Stanley C Eisenstat. A divide-and-conquer algorithm for the symmetric tridiagonal eigenproblem. SIAM Journal on Matrix Analysis and Applications, 16(1):172-191, 1995. URL: https://doi.org/10.1137/S0895479892241287.
  47. Ming Gu and Stanley C Eisenstat. Efficient algorithms for computing a strong rank-revealing qr factorization. SIAM Journal on Scientific Computing, 17(4):848-869, 1996. URL: https://doi.org/10.1137/0917055.
  48. Juris Hartmanis and Janos Simon. On the power of multiplication in random access machines. In 15th Annual Symposium on Switching and Automata Theory, pages 13-23. IEEE, 1974. URL: https://doi.org/10.1109/SWAT.1974.20.
  49. David Harvey and Joris Van Der Hoeven. Integer multiplication in time O(nlog n). Annals of Mathematics, 193(2):563-617, 2021. Google Scholar
  50. Nicholas J Higham. Accuracy and stability of numerical algorithms. SIAM, 2002. Google Scholar
  51. Walter Hoffmann and Beresford N Parlett. A new proof of global convergence for the tridiagonal QL algorithm. SIAM Journal on Numerical Analysis, 15(5):929-937, 1978. Google Scholar
  52. Alston S Householder. Unitary triangularization of a nonsymmetric matrix. Journal of the ACM, 5(4):339-342, 1958. URL: https://doi.org/10.1145/320941.320947.
  53. C.G.J. Jacobi. Über ein leichtes Verfahren die in der Theorie der Säcularstörungen vorkommenden Gleichungen numerisch aufzulösen. Journal für die reine und angewandte Mathematik, 30:51-94, 1846. URL: http://eudml.org/doc/147275.
  54. Ian T Jolliffe. Principal component analysis for special types of data. Springer, 2002. Google Scholar
  55. Praneeth Kacham and David P Woodruff. Faster Algorithms for Schatten-p Low Rank Approximation. In Proc. Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2024. Google Scholar
  56. Vera N Kublanovskaya. On some algorithms for the solution of the complete eigenvalue problem. USSR Computational Mathematics and Mathematical Physics, 1(3):637-657, 1962. Google Scholar
  57. Cornelius Lanczos. An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. Journal of Research of the National Bureau of Standards, 45(4), 1950. Google Scholar
  58. Oren E Livne and Achi Brandt. N roots of the secular equation in O(N) operations. SIAM Journal on Matrix Analysis and Applications, 24(2):439-453, 2002. URL: https://doi.org/10.1137/S0895479801383695.
  59. Anand Louis and Santosh S Vempala. Accelerated newton iteration for roots of black box polynomials. In Proc. 57th IEEE Symposium on Foundations of Computer Science, pages 732-740. IEEE, 2016. URL: https://doi.org/10.1109/FOCS.2016.83.
  60. RV Mises and Hilda Pollaczek-Geiringer. Praktische Verfahren der Gleichungsauflösung. ZAMM-Journal of Applied Mathematics and Mechanics/Zeitschrift für Angewandte Mathematik und Mechanik, 9(1):58-77, 1929. Google Scholar
  61. Cameron Musco, Christopher Musco, and Aaron Sidford. Stability of the Lanczos method for matrix function approximation. In Proc. 29th ACM-SIAM Symposium on Discrete Algorithms, pages 1605-1624. SIAM, 2018. URL: https://doi.org/10.1137/1.9781611975031.105.
  62. Yuji Nakatsukasa, Zhaojun Bai, and François Gygi. Optimizing Halley’s iteration for computing the matrix polar decomposition. SIAM Journal on Matrix Analysis and Applications, 31(5):2700-2720, 2010. URL: https://doi.org/10.1137/090774999.
  63. Yuji Nakatsukasa and Nicholas J Higham. Stable and efficient spectral divide and conquer algorithms for the symmetric eigenvalue decomposition and the SVD. SIAM Journal on Scientific Computing, 35(3):A1325-A1349, 2013. URL: https://doi.org/10.1137/120876605.
  64. Deanna Needell, William Swartworth, and David P Woodruff. Testing positive semidefiniteness using linear measurements. In Proc. 2022 IEEE Symposium on Foundations of Computer Science, pages 87-97. IEEE, 2022. URL: https://doi.org/10.1109/FOCS54457.2022.00016.
  65. Dianne P O'Leary and Gilbert W Stewart. Computing the eigenvalues and eigenvectors of symmetric arrowhead matrices. Journal of Computational Physics, 90(2):497-505, 1990. Google Scholar
  66. Xiaofeng Ou and Jianlin Xia. Superdc: superfast divide-and-conquer eigenvalue decomposition with improved stability for rank-structured matrices. SIAM Journal on Scientific Computing, 44(5):A3041-A3066, 2022. Google Scholar
  67. Chris C Paige. Accuracy and effectiveness of the Lanczos algorithm for the symmetric eigenproblem. Linear algebra and its Applications, 34:235-258, 1980. Google Scholar
  68. Victor Y Pan and Zhao Q Chen. The complexity of the matrix eigenproblem. In Proc. 31st ACM Symposium on Theory of Computing, pages 507-516, 1999. URL: https://doi.org/10.1145/301250.301389.
  69. Christos H Papadimitriou, Hisao Tamaki, Prabhakar Raghavan, and Santosh Vempala. Latent semantic indexing: A probabilistic analysis. In Proc. 17th ACM Symposium on Principles of Database Systems, pages 159-168, 1998. URL: https://doi.org/10.1145/275487.275505.
  70. Beresford N Parlett. The Symmetric Eigenvalue Problem. SIAM, 1998. Google Scholar
  71. Vladimir Rokhlin. Rapid solution of integral equations of classical potential theory. Journal of computational physics, 60(2):187-207, 1985. Google Scholar
  72. H Rutishauser. On Jacobi rotation patterns. In Proceedings of Symposia in Applied Mathematics, volume 15, pages 219-239, 1963. Google Scholar
  73. Yousef Saad. Numerical methods for large eigenvalue problems: revised edition. SIAM, 2011. Google Scholar
  74. Ryan Schneider. Pseudospectral divide-and-conquer for the generalized eigenvalue problem. University of California, San Diego, 2024. Google Scholar
  75. Arnold Schönhage. Unitäre transformationen großer matrizen. Numerische Mathematik, 20:409-417, 1972. Google Scholar
  76. Arnold Schönhage. On the power of random access machines. In Proc. International Colloquium on Automata, Languages, and Programming, pages 520-529. Springer, 1979. URL: https://doi.org/10.1007/3-540-09510-1_42.
  77. Arnold Schönhage and Volker Strassen. Fast multiplication of large numbers. Computing, 7:281-292, 1971. Google Scholar
  78. Rikhav Shah. Hermitian Diagonalization in Linear Precision. In Proc. 2025 ACM-SIAM Symposium on Discrete Algorithms, pages 5599-5615. SIAM, 2025. URL: https://doi.org/10.1137/1.9781611978322.192.
  79. Aleksandros Sobczyk. Deterministic complexity analysis of Hermitian eigenproblems. arXiv, 2025. URL: https://doi.org/10.48550/arXiv.2410.21550.
  80. Aleksandros Sobczyk, Marko Mladenovic, and Mathieu Luisier. Invariant subspaces and PCA in nearly matrix multiplication time. Advances in Neural Information Processing Systems, 37:19013-19086, 2024. Google Scholar
  81. Nevena Jakovčević Stor, Ivan Slapničar, and Jesse L Barlow. Accurate eigenvalue decomposition of real symmetric arrowhead matrices and applications. Linear Algebra and its Applications, 464:62-89, 2015. Google Scholar
  82. William Swartworth and David P Woodruff. Optimal eigenvalue approximation via sketching. In Proc. 55th ACM Symposium on Theory of Computing, pages 145-155, 2023. URL: https://doi.org/10.1145/3564246.3585102.
  83. James Vogel, Jianlin Xia, Stephen Cauley, and Venkataramanan Balakrishnan. Superfast divide-and-conquer method and perturbation analysis for structured eigenvalue solutions. SIAM Journal on Scientific Computing, 38(3):A1358-A1382, 2016. URL: https://doi.org/10.1137/15M1018812.
  84. James Hardy Wilkinson. Global convergene of tridiagonal QR algorithm with origin shifts. Linear Algebra and its Applications, 1(3):409-420, 1968. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact