Document Open Access Logo

On the I/O Complexity of the Cocke-Younger-Kasami Algorithm and of a Family of Related Dynamic Programming Algorithms

Authors Lorenzo De Stefani , Vedant Gupta

PDF

File

Thumbnail PDF
PDF
LIPIcs.WADS.2025.49.pdf
  • Filesize: 1 MB
  • 24 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Design and analysis of algorithms
Keywords
  • I/O complexity
  • Dynamic Programming Algorithms
  • Lower Bounds
  • Recomputation
  • Cocke-Younger-Kasami

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

Asymptotically tight lower bounds are derived for the Input/Output (I/O) complexity of a class of dynamic programming algorithms, including matrix chain multiplication, optimal polygon triangulation, and the construction of optimal binary search trees. Assuming no recomputation of intermediate values, we establish an Ω(n³/(√M B)) I/O lower bound, where n denotes the size of the input and M denotes the size of the available fast memory (cache). When recomputation is allowed, we show that the same bound holds for M < cn, where c is a positive constant. In the case where M ≥ 2n, we show an Ω(n/B) I/O lower bound. We also discuss algorithms for which the number of executed I/O operations matches asymptotically each of the presented lower bounds, which are thus asymptotically tight.
Additionally, we refine our general method to obtain a lower bound for the I/O complexity of the Cocke-Younger-Kasami algorithm, where the size of the grammar impacts the I/O complexity. An upper bound with asymptotically matching performance in many cases is also provided.

Cite As Get BibTex

Lorenzo De Stefani and Vedant Gupta. On the I/O Complexity of the Cocke-Younger-Kasami Algorithm and of a Family of Related Dynamic Programming Algorithms. In 19th International Symposium on Algorithms and Data Structures (WADS 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 349, pp. 49:1-49:24, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.WADS.2025.49

Author Details

Lorenzo De Stefani
  • Department of Computer Science, Brown University, Providence, RI, USA
Vedant Gupta
  • Department of Computer Science, Brown University, Providence, RI, USA

References

  1. Alok Aggarwal and S Vitter, Jeffrey. The input/output complexity of sorting and related problems. Communications of the ACM, 31(9):1116-1127, 1988. URL: https://doi.org/10.1145/48529.48535.
  2. Grey Ballard, James Demmel, Olga Holtz, and Oded Schwartz. Communication-optimal parallel and sequential Cholesky decomposition. In Proceedings of the twenty-first annual symposium on Parallelism in algorithms and architectures, pages 245-252, 2009. Google Scholar
  3. Grey Ballard, James Demmel, Olga Holtz, and Oded Schwartz. Graph expansion and communication costs of fast matrix multiplication. Journal of the ACM (JACM), 59(6):1-23, 2013. URL: https://doi.org/10.1145/2395116.2395121.
  4. Richard Bellman. The theory of dynamic programming. Bulletin of the American Mathematical Society, 60(6):503-515, 1954. Google Scholar
  5. Gianfranco Bilardi and Lorenzo De Stefani. The I/O complexity of Strassen’s matrix multiplication with recomputation. In Workshop on Algorithms and Data Structures, pages 181-192. Springer, 2017. URL: https://doi.org/10.1007/978-3-319-62127-2_16.
  6. Gianfranco Bilardi and Lorenzo De Stefani. The I/O complexity of Toom-Cook integer multiplication. In Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 2034-2052. SIAM, 2019. URL: https://doi.org/10.1137/1.9781611975482.123.
  7. Gianfranco Bilardi and Lorenzo De Stefani. The DAG Visit Approach for Pebbling and I/O Lower Bounds. In In Proc. FSTTCS 2022, volume 250, pages 7:1-7:23, Dagstuhl, Germany, 2022. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.FSTTCS.2022.7.
  8. Gianfranco Bilardi and Enoch Peserico. A characterization of temporal locality and its portability across memory hierarchies. In International Colloquium on Automata, Languages, and Programming, pages 128-139. Springer, 2001. URL: https://doi.org/10.1007/3-540-48224-5_11.
  9. Gianfranco Bilardi, Andrea Pietracaprina, and Paolo D’Alberto. On the space and access complexity of computation DAGs. In Graph-Theoretic Concepts in Computer Science: 26th International Workshop, WG 2000 Konstanz, Germany, June 15-17, 2000 Proceedings 26, pages 47-58. Springer, 2000. URL: https://doi.org/10.1007/3-540-40064-8_6.
  10. Gianfranco Bilardi and Franco P Preparata. Horizons of parallel computation. Journal of Parallel and Distributed Computing, 27(2):172-182, 1995. URL: https://doi.org/10.1006/JPDC.1995.1080.
  11. Gianfranco Bilardi and Franco P Preparata. Processor—time tradeoffs under bounded-speed message propagation: Part II, lower bounds. Theory of Computing Systems, 32(5):531-559, 1999. URL: https://doi.org/10.1007/S002240000131.
  12. Guy E Blleloch and Yan Gu. Improved parallel cache-oblivious algorithms for dynamic programming and linear algebra. arXiv preprint, 2018. URL: https://arxiv.org/abs/1809.09330.
  13. Cary Cherng and Richard E Ladner. Cache efficient simple dynamic programming. Discrete Mathematics & Theoretical Computer Science, 2005. Google Scholar
  14. Rezaul Chowdhury, Pramod Ganapathi, Yuan Tang, and Jesmin Jahan Tithi. Provably efficient scheduling of cache-oblivious wavefront algorithms. In Proceedings of the 29th ACM Symposium on Parallelism in Algorithms and Architectures, pages 339-350, 2017. URL: https://doi.org/10.1145/3087556.3087586.
  15. Rezaul Chowdhury, Pramod Ganapathi, Stephen Tschudi, Jesmin Jahan Tithi, Charles Bachmeier, Charles E Leiserson, Armando Solar-Lezama, Bradley C Kuszmaul, and Yuan Tang. Autogen: Automatic discovery of efficient recursive divide-8-conquer algorithms for solving dynamic programming problems. ACM Transactions on Parallel Computing (TOPC), 4(1):1-30, 2017. URL: https://doi.org/10.1145/3125632.
  16. Rezaul Alam Chowdhury and Vijaya Ramachandran. Cache-oblivious dynamic programming. In Proceedings of the seventeenth annual ACM-SIAM symposium on Discrete algorithm, pages 591-600. Citeseer, 2006. URL: http://dl.acm.org/citation.cfm?id=1109557.1109622.
  17. Rezaul Alam Chowdhury and Vijaya Ramachandran. The cache-oblivious gaussian elimination paradigm: theoretical framework, parallelization and experimental evaluation. In Proceedings of the nineteenth annual ACM Symposium on Parallel Algorithms and Architectures, pages 71-80, 2007. URL: https://doi.org/10.1145/1248377.1248392.
  18. Rezaul Alam Chowdhury and Vijaya Ramachandran. Cache-efficient dynamic programming algorithms for multicores. In Proceedings of the twentieth annual symposium on Parallelism in algorithms and architectures, pages 207-216, 2008. URL: https://doi.org/10.1145/1378533.1378574.
  19. Rezaul Alan Chowdhury, Hai-Son Le, and Vijaya Ramachandran. Cache-oblivious dynamic programming for bioinformatics. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 7(3):495-510, 2008. Google Scholar
  20. John Cocke. Programming languages and their compilers: Preliminary notes. New York University, 1969. Google Scholar
  21. Lorenzo De Stefani and Vedant Gupta. On the I/O complexity of the CYK algorithm and of a family of related DP algorithms. arXiv preprint arXiv:2410.20337, 2024. URL: https://doi.org/10.48550/arXiv.2410.20337.
  22. Erik D Demaine, Andrea Lincoln, Quanquan C Liu, Jayson Lynch, and Virginia Vassilevska Williams. Fine-grained I/O complexity via reductions: New lower bounds, faster algorithms, and a time hierarchy. arXiv preprint arXiv:1711.07960, 2017. URL: https://arxiv.org/abs/1711.07960.
  23. Xiangyun Ding, Yan Gu, and Yihan Sun. Parallel and (nearly) work-efficient dynamic programming. In Proceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures, pages 219-232, 2024. URL: https://doi.org/10.1145/3626183.3659958.
  24. Aaron Dunlop, Nathan Bodenstab, and Brian Roark. Reducing the grammar constant: an analysis of CYK parsing efficiency. Technical report, Technical report CSLU-2010-02, OHSU, 2010. Google Scholar
  25. Reza Farivar, Harshit Kharbanda, Shivaram Venkataraman, and Roy H Campbell. An algorithm for fast edit distance computation on GPUs. In 2012 Innovative Parallel Computing (InPar), pages 1-9. IEEE, 2012. Google Scholar
  26. Matteo Frigo, Charles E Leiserson, Harald Prokop, and Sridhar Ramachandran. Cache-oblivious algorithms. In 40th Annual Symposium on Foundations of Computer Science (Cat. No. 99CB37039), pages 285-297. IEEE, 1999. URL: https://doi.org/10.1109/SFFCS.1999.814600.
  27. Zvi Galil and Kunsoo Park. Dynamic programming with convexity, concavity and sparsity. Theoretical Computer Science, 92(1):49-76, 1992. URL: https://doi.org/10.1016/0304-3975(92)90135-3.
  28. Te Chiang Hu and Man Tak Shing. Computation of matrix chain products. Part I. SIAM Journal on Computing, 11(2):362-373, 1982. URL: https://doi.org/10.1137/0211028.
  29. Shachar Itzhaky, Rohit Singh, Armando Solar-Lezama, Kuat Yessenov, Yongquan Lu, Charles Leiserson, and Rezaul Chowdhury. Deriving divide-and-conquer dynamic programming algorithms using solver-aided transformations. ACM SIGPLAN Notices, 51(10):145-164, 2016. Google Scholar
  30. Mohammad Mahdi Javanmard, Pramod Ganapathi, Rathish Das, Zafar Ahmad, Stephen Tschudi, and Rezaul Chowdhury. Toward efficient architecture-independent algorithms for dynamic programs. In High Performance Computing: 34th International Conference, ISC High Performance 2019, Frankfurt/Main, Germany, June 16-20, 2019, Proceedings 34, pages 143-164. Springer, 2019. URL: https://doi.org/10.1007/978-3-030-20656-7_8.
  31. Hong Jia-Wei. I/O complexity: The red-blue pebble game. In Proceedings of the thirteenth annual ACM symposium on Theory of computing, pages 326-333, 1981. Google Scholar
  32. Tadao Kasami. An efficient recognition and syntax-analysis algorithm for context-free languages. Coordinated Science Laboratory Report no. R-257, 1966. Google Scholar
  33. Donald E. Knuth. Optimum binary search trees. Acta informatica, 1:14-25, 1971. URL: https://doi.org/10.1007/BF00264289.
  34. Richard Koch, Tom Leighton, Bruce Maggs, and Satish Rao. Work-preserving emulations of fixed-connection networks. In Proceedings of the twenty-first annual ACM symposium on Theory of computing, pages 227-240, 1989. Google Scholar
  35. Martin Lange and Hans Leiß. To CNF or not to CNF? an efficient yet presentable version of the cyk algorithm. Informatica Didactica, 8(2009):1-21, 2009. URL: http://www.informatica-didactica.de/cmsmadesimple/index.php?page=LangeLeiss2009.
  36. J-S Park, Michael Penner, and Viktor K Prasanna. Optimizing graph algorithms for improved cache performance. IEEE Transactions on parallel and distributed systems, 15(9):769-782, 2004. URL: https://doi.org/10.1109/TPDS.2004.44.
  37. Cynthia A Patterson, Marc Snir, and Susan L Graham. Getting up to speed: The future of supercomputing. National Academies Press, 2005. Google Scholar
  38. Vijaya Ramachandran. Cache-oblivious computation: Algorithms and experimental evaluation. In 2007 International Conference on Computing: Theory and Applications (ICCTA'07), pages 20-26. IEEE, 2007. URL: https://doi.org/10.1109/ICCTA.2007.34.
  39. John E Savage. Extending the Hong-Kung model to memory hierarchies. In International Computing and Combinatorics Conference, pages 270-281. Springer, 1995. URL: https://doi.org/10.1007/BFB0030842.
  40. Jacob Scott, Olga Holtz, and Oded Schwartz. Matrix multiplication I/O-complexity by path routing. In Proceedings of the 27th ACM symposium on Parallelism in Algorithms and Architectures, pages 35-45, 2015. URL: https://doi.org/10.1145/2755573.2755594.
  41. Michael Sipser. Introduction to the Theory of Computation. ACM Sigact News, 27(1):27-29, 1996. URL: https://doi.org/10.1145/230514.571645.
  42. Xinying Song, Shilin Ding, and Chin-Yew Lin. Better binarization for the CYK parsing. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 167-176, 2008. URL: https://aclanthology.org/D08-1018/.
  43. Shanjiang Tang, Ce Yu, Jizhou Sun, Bu-Sung Lee, Tao Zhang, Zhen Xu, and Huabei Wu. Easypdp: An efficient parallel dynamic programming runtime system for computational biology. IEEE Transactions on Parallel and Distributed Systems, 23(5):862-872, 2011. URL: https://doi.org/10.1109/TPDS.2011.218.
  44. Yuan Tang, Ronghui You, Haibin Kan, Jesmin Jahan Tithi, Pramod Ganapathi, and Rezaul A Chowdhury. Cache-oblivious wavefront: Improving parallelism of recursive dynamic programming algorithms without losing cache-efficiency. In Proceedings of the 20th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pages 205-214, 2015. Google Scholar
  45. Leslie Valiant. General context-free recognition in less than cubic time. J. Comput. Syst. Sci., 1974. Google Scholar
  46. F Frances Yao. Efficient dynamic programming using quadrangle inequalities. In Proceedings of the twelfth annual ACM symposium on Theory of computing, pages 429-435, 1980. URL: https://doi.org/10.1145/800141.804691.
  47. Daniel H Younger. Recognition and parsing of context-free languages in time n³. Information and control, 10(2):189-208, 1967. URL: https://doi.org/10.1016/S0019-9958(67)80007-X.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact