Document Open Access Logo

Learning Gradual Typing Performance

Authors Mohammad Wahiduzzaman Khan , Sheng Chen , Yi He

PDF
Thumbnail PDF

File

LIPIcs.ECOOP.2024.21.pdf
  • Filesize: 4.11 MB
  • 27 pages

Document Identifiers

Author Details

Mohammad Wahiduzzaman Khan
  • CACS, University of Louisiana, Lafayette, LA, USA
Sheng Chen
  • CACS, University of Louisiana, Lafayette, LA, USA
Yi He
  • Data Science, College William & Mary, Williamsburg, VA, USA

Funding

This work has been supported in part by the National Science Foundation (NSF) under Grant Nos. IIS-2245946, IIS-2236578, and CCF-1750886 and in part by the Commonwealth Cyber Initiative (CCI) and DARPA.

Cite AsGet BibTex

Mohammad Wahiduzzaman Khan, Sheng Chen, and Yi He. Learning Gradual Typing Performance. In 38th European Conference on Object-Oriented Programming (ECOOP 2024). Leibniz International Proceedings in Informatics (LIPIcs), Volume 313, pp. 21:1-21:27, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2024)
https://doi.org/10.4230/LIPIcs.ECOOP.2024.21

Abstract

Gradual typing has emerged as a promising typing discipline for reconciling static and dynamic typing, which have respective strengths and shortcomings. Thanks to its promises, gradual typing has gained tremendous momentum in both industry and academia. A main challenge in gradual typing is that, however, the performance of its programs can often be unpredictable, and adding or removing the type of a a single parameter may lead to wild performance swings. Many approaches have been proposed to optimize gradual typing performance, but little work has been done to aid the understanding of the performance landscape of gradual typing and navigating the migration process (which adds type annotations to make programs more static) to avert performance slowdowns. Motivated by this situation, this work develops a machine-learning-based approach to predict the performance of each possible way of adding type annotations to a program. On top of that, many supports for program migrations could be developed, such as finding the most performant neighbor of any given configuration. Our approach gauges runtime overheads of dynamic type checks inserted by gradual typing and uses that information to train a machine learning model, which is used to predict the running time of gradual programs. We have evaluated our approach on 12 Python benchmarks for both guarded and transient semantics. For guarded semantics, our evaluation results indicate that with only 40 training instances generated from each benchmark, the predicted times for all other instances differ on average by 4% from the measured times. For transient semantics, the time difference ratio is higher but the time difference is often within 0.1 seconds.

Subject Classification

ACM Subject Classification
  • Theory of computation → Type structures
  • Computing methodologies → Machine learning
  • Computing methodologies → Learning linear models
Keywords
  • Gradual typing performance
  • type migration
  • performance prediction
  • machine learning

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Supplementary Materials

References

  1. Hervé Abdi and Lynne J Williams. Principal component analysis. Wiley interdisciplinary reviews: computational statistics, 2(4):433-459, 2010. Google Scholar
  2. Miltiadis Allamanis, Earl T. Barr, Premkumar Devanbu, and Charles Sutton. A survey of machine learning for big code and naturalness. ACM Comput. Surv., 51(4), July 2018. URL: https://doi.org/10.1145/3212695.
  3. Miltiadis Allamanis, Earl T. Barr, Soline Ducousso, and Zheng Gao. Typilus: neural type hints. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, June 2020. URL: https://doi.org/10.1145/3385412.3385997.
  4. Esteban Allende, Johan Fabry, Ronald Garcia, and Éric Tanter. Confined gradual typing. In Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA '14, pages 251-270, New York, NY, USA, 2014. ACM. URL: https://doi.org/10.1145/2660193.2660222.
  5. Spenser Bauman, Carl Friedrich Bolz-Tereick, Jeremy Siek, and Sam Tobin-Hochstadt. Sound gradual typing: Only mostly dead. Proc. ACM Program. Lang., 1(OOPSLA):54:1-54:24, October 2017. URL: https://doi.org/10.1145/3133878.
  6. John Campora, Sheng Chen, Martin Erwig, and Eric Walkingshaw. Migrating gradual types. In Proceedings of the 45th ACM SIGPLAN Symposium on Principles of Programming Languages, POPL '18, New York, NY, USA, 2018. ACM. Google Scholar
  7. John Peter Campora and Sheng Chen. Taming type annotations in gradual typing. Proc. ACM Program. Lang., 4(OOPSLA), November 2020. URL: https://doi.org/10.1145/3428259.
  8. John Peter Campora, Sheng Chen, and Eric Walkingshaw. Casts and costs: Harmonizing safety and performance in gradual typing. Proc. ACM Program. Lang., 2(ICFP):98:1-98:30, July 2018. URL: https://doi.org/10.1145/3236793.
  9. Giuseppe Castagna, Victor Lanvin, Tommaso Petrucciani, and Jeremy G. Siek. Gradual typing: A new perspective. Proc. ACM Program. Lang., 3(POPL), January 2019. URL: https://doi.org/10.1145/3290329.
  10. Satish Chandra, Colin S. Gordon, Jean-Baptiste Jeannin, Cole Schlesinger, Manu Sridharan, Frank Tip, and Youngil Choi. Type inference for static compilation of javascript. In Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2016, pages 410-429, New York, NY, USA, 2016. ACM. URL: https://doi.org/10.1145/2983990.2984017.
  11. Fernando Cristiani and Peter Thiemann. Generation of typescript declaration files from javascript code. In Proceedings of the 18th ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes, MPLR 2021, pages 97-112, New York, NY, USA, 2021. Association for Computing Machinery. URL: https://doi.org/10.1145/3475738.3480941.
  12. Daniel Feltey, Ben Greenman, Christophe Scholliers, Robert Bruce Findler, and Vincent St-Amour. Collapsible contracts: Fixing a pathology of gradual typing. Proc. ACM Program. Lang., 2(OOPSLA), October 2018. URL: https://doi.org/10.1145/3276503.
  13. Ronald Garcia and Matteo Cimini. Principal type schemes for gradual programs. In Proceedings of the 42Nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL '15, pages 303-315, New York, NY, USA, 2015. ACM. URL: https://doi.org/10.1145/2676726.2676992.
  14. Ben Greenman and Matthias Felleisen. A spectrum of type soundness and performance. Proc. ACM Program. Lang., 2(ICFP):71:1-71:32, July 2018. URL: https://doi.org/10.1145/3236766.
  15. Ben Greenman, Matthias Felleisen, and Christos Dimoulas. How profilers can help navigate type migration. Proc. ACM Program. Lang., 7(OOPSLA2), October 2023. URL: https://doi.org/10.1145/3622817.
  16. Ben Greenman, Asumu Takikawa, Max S. New, Daniel Feltey, Robert Bruce Findler, Jan Vitek, and Matthias Felleisen. How to evaluate the performance of gradual type systems. Journal of Functional Programming, 29:e4, 2019. URL: https://doi.org/10.1017/S0956796818000217.
  17. Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. Unixcoder: Unified cross-modal pre-training for code representation. arXiv preprint, 2022. URL: https://arxiv.org/abs/2203.03850.
  18. Sergiy Kolesnikov, Norbert Siegmund, Christian Kästner, Alexander Grebhahn, and Sven Apel. Tradeoffs in modeling performance of highly configurable software systems. Software & Systems Modeling, 18:2265-2283, June 2019. URL: https://doi.org/10.1007/s10270-018-0662-9.
  19. Erik Krogh Kristensen and Anders Møller. Type test scripts for typescript testing. Proc. ACM Program. Lang., 1(OOPSLA):90:1-90:25, October 2017. URL: https://doi.org/10.1145/3133914.
  20. Andre Kuhlenschmidt, Deyaaeldeen Almahallawi, and Jeremy G. Siek. Toward efficient gradual typing for structural types via coercions. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2019, pages 517-532, New York, NY, USA, 2019. Association for Computing Machinery. URL: https://doi.org/10.1145/3314221.3314627.
  21. Andréa Matsunaga and José A.B. Fortes. On the use of machine learning to predict the time and resources consumed by applications. In 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing, pages 495-504, 2010. URL: https://doi.org/10.1109/CCGRID.2010.98.
  22. Zeina Migeed and Jens Palsberg. What is decidable about gradual types? Proc. ACM Program. Lang., 4(POPL), December 2019. URL: https://doi.org/10.1145/3371097.
  23. Amir M Mir, Evaldas Latoškinas, Sebastian Proksch, and Georgios Gousios. Type4py: practical deep similarity learning-based type inference for python. In Proceedings of the 44th International Conference on Software Engineering, pages 2241-2252, 2022. Google Scholar
  24. Yusuke Miyazaki, Taro Sekiyama, and Atsushi Igarashi. Dynamic type inference for gradual hindley-milner typing. Proc. ACM Program. Lang., 3(POPL):18:1-18:29, January 2019. URL: https://doi.org/10.1145/3290331.
  25. Cameron Moy, Phúc C. Nguy~ên, Sam Tobin-Hochstadt, and David Van Horn. Corpse reviver: Sound and efficient gradual typing via contract verification. Proc. ACM Program. Lang., 5(POPL), January 2021. URL: https://doi.org/10.1145/3434334.
  26. Fabian Muehlboeck and Ross Tate. Sound gradual typing is nominally alive and well. In OOPSLA, New York, NY, USA, 2017. ACM. URL: https://doi.org/10.1145/3133880.
  27. Phúc C. Nguyen, Thomas Gilray, Sam Tobin-Hochstadt, and David Van Horn. Soft contract verification for higher-order stateful programs. Proc. ACM Program. Lang., 2(POPL):51:1-51:30, December 2017. URL: https://doi.org/10.1145/3158139.
  28. Francisco Ortin, Miguel Garcia, and Seán McSweeney. Rule-based program specialization to optimize gradually typed code. Knowledge-Based Systems, 179:145-173, 2019. URL: https://doi.org/10.1016/j.knosys.2019.05.013.
  29. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830, 2011. Google Scholar
  30. Yun Peng, Cuiyun Gao, Zongjie Li, Bowei Gao, David Lo, Qirun Zhang, and Michael Lyu. Static inference meets deep learning: A hybrid type inference approach for python. In Proceedings of the 44th International Conference on Software Engineering, ICSE '22, pages 2019-2030, New York, NY, USA, 2022. Association for Computing Machinery. URL: https://doi.org/10.1145/3510003.3510038.
  31. Luna Phipps-Costin, Carolyn Jane Anderson, Michael Greenberg, and Arjun Guha. Solver-based gradual type migration. Proc. ACM Program. Lang., 5(OOPSLA), October 2021. URL: https://doi.org/10.1145/3485488.
  32. Marius-Constantin Popescu, Valentina E Balas, Liliana Perescu-Popescu, and Nikos Mastorakis. Multilayer perceptron and neural networks. WSEAS Transactions on Circuits and Systems, 8(7):579-588, 2009. Google Scholar
  33. Michael Pradel, Georgios Gousios, Jason Liu, and Satish Chandra. Typewriter: Neural type prediction with search-based validation, 2020. URL: https://arxiv.org/abs/1912.03768.
  34. Aseem Rastogi, Avik Chaudhuri, and Basil Hosmer. The ins and outs of gradual type inference. In Proceedings of the 39th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL '12, pages 481-494, New York, NY, USA, 2012. ACM. URL: https://doi.org/10.1145/2103656.2103714.
  35. Aseem Rastogi, Nikhil Swamy, Cédric Fournet, Gavin M. Bierman, and Panagiotis Vekris. Safe & efficient gradual typing for typescript. In POPL, 2015. Google Scholar
  36. Gregor Richards, Ellen Arteca, and Alexi Turcotte. The vm already knew that: Leveraging compile-time knowledge to optimize gradual typing. Proc. ACM Program. Lang., 1(OOPSLA):55:1-55:27, October 2017. URL: https://doi.org/10.1145/3133879.
  37. Jeremy Siek, Ronald Garcia, and Walid Taha. Exploring the design space of higher-order casts. In Giuseppe Castagna, editor, Programming Languages and Systems, pages 17-31, Berlin, Heidelberg, 2009. Springer Berlin Heidelberg. Google Scholar
  38. Jeremy G. Siek and Walid Taha. Gradual typing for functional languages. In In Scheme and Functional Programming Workshop, pages 81-92, 2006. Google Scholar
  39. Jeremy G. Siek and Manish Vachharajani. Gradual typing with unification-based inference. In Proceedings of the 2008 Symposium on Dynamic Languages, DLS '08, pages 7:1-7:12, New York, NY, USA, 2008. ACM. URL: https://doi.org/10.1145/1408681.1408688.
  40. Dagmar Stumpfe and Jürgen Bajorath. Exploring activity cliffs in medicinal chemistry. Journal of Medicinal Chemistry, 55(7):2932-2942, 2012. PMID: 22236250. URL: https://doi.org/10.1021/jm201706b.
  41. Asumu Takikawa, Daniel Feltey, Ben Greenman, Max S. New, Jan Vitek, and Matthias Felleisen. Is sound gradual typing dead? In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL '16, pages 456-468, New York, NY, USA, 2016. ACM. URL: https://doi.org/10.1145/2837614.2837630.
  42. Sam Tobin-Hochstadt and Matthias Felleisen. Interlanguage migration: From scripts to programs. In Companion to the 21st ACM SIGPLAN Symposium on Object-oriented Programming Systems, Languages, and Applications, OOPSLA '06, pages 964-974, New York, NY, USA, 2006. ACM. URL: https://doi.org/10.1145/1176617.1176755.
  43. Sam Tobin-Hochstadt, Matthias Felleisen, Robert Findler, Matthew Flatt, Ben Greenman, Andrew M. Kent, Vincent St-Amour, T. Stephen Strickland, and Asumu Takikawa. Migratory Typing: Ten Years Later. In Benjamin S. Lerner, Rastislav Bodík, and Shriram Krishnamurthi, editors, 2nd Summit on Advances in Programming Languages (SNAPL 2017), volume 71 of Leibniz International Proceedings in Informatics (LIPIcs), pages 17:1-17:17, Dagstuhl, Germany, 2017. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.SNAPL.2017.17.
  44. Preston Tunnell Wilson, Ben Greenman, Justin Pombrio, and Shriram Krishnamurthi. The behavior of gradual types: A user study. In DLS, number ICFP in DLS 2018, page 1–12, 2018. URL: https://doi.org/10.1145/3393673.3276947.
  45. Michael M. Vitousek, Andrew M. Kent, Jeremy G. Siek, and Jim Baker. Design and evaluation of gradual typing for python. In Proceedings of the 10th ACM Symposium on Dynamic Languages, DLS '14, pages 45-56, New York, NY, USA, 2014. ACM. URL: https://doi.org/10.1145/2661088.2661101.
  46. Michael M. Vitousek, Jeremy G. Siek, and Avik Chaudhuri. Optimizing and evaluating transient gradual typing. In Proceedings of the 15th ACM SIGPLAN International Symposium on Dynamic Languages, DLS 2019, pages 28-41, New York, NY, USA, 2019. ACM. URL: https://doi.org/10.1145/3359619.3359742.
  47. Michael M. Vitousek, Cameron Swords, and Jeremy G. Siek. Big types in little runtime: Open-world soundness and collaborative blame for gradual type systems. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages, POPL 2017, pages 762-774, New York, NY, USA, 2017. ACM. URL: https://doi.org/10.1145/3009837.3009849.
  48. Hieu Dinh Vo and Son Nguyen. Can an old fashioned feature extraction and a light-weight model improve vulnerability type identification performance? arXiv preprint, 2023. URL: https://arxiv.org/abs/2306.14726.
  49. Yao Wan, Yang He, Zhangqian Bi, Jianguo Zhang, Hongyu Zhang, Yulei Sui, Guandong Xu, Hai Jin, and Philip S Yu. Deep learning for code intelligence: Survey, benchmark and toolkit. arXiv preprint, 2023. URL: https://arxiv.org/abs/2401.00288.
  50. Jun Xia, Lecheng Zhang, Xiao Zhu, and Stan Z. Li. Why deep models often cannot beat non-deep counterparts on molecular property prediction? In ICML 3rd Workshop on Interpretable Machine Learning in Healthcare (IMLH), 2023. URL: https://openreview.net/forum?id=hJG8xgj2Y5.
  51. Ming-Ho Yee and Arjun Guha. Do machine learning models produce typescript types that type check? arXiv preprint, 2023. URL: https://arxiv.org/abs/2302.12163.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact