Document Open Access Logo

PoTo: A Hybrid Andersen’s Points-To Analysis for Python

Authors Ingkarat Rak-amnouykit , Ana Milanova , Guillaume Baudart , Martin Hirzel , Julian Dolby

PDF

File

Thumbnail PDF
PDF
LIPIcs.ECOOP.2025.27.pdf
  • Filesize: 2.09 MB
  • 29 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Software and its engineering → General programming languages
Keywords
  • Python
  • Points-to analysis
  • Machine learning libraries

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

As Python is increasingly being adopted for large and complex programs, the importance of static analysis for Python (such as type inference) grows. Unfortunately, static analysis for Python remains a challenging task due to its dynamic language features and its abundant external libraries. To help fill this gap, this paper presents PoTo, an Andersen-style context-insensitive and flow-insensitive points-to analysis for Python. PoTo addresses Python-specific challenges and works for large programs via a novel hybrid evaluation, integrating traditional static points-to analysis with concrete evaluation in the Python interpreter for external library calls. We evaluate the analysis with two clients: type inference and call graph construction. This paper presents PoTo+, a static type inference for Python built on PoTo. We evaluate PoTo+ and compare it to two state-of-the-art Python type inference techniques: (1) the static rule-based Pytype and (2) the deep-learning based DLInfer. Our results show that PoTo+ outperforms both Pytype and DLInfer on existing Python packages. This paper also presents PoToCG, a call-graph construction analysis for Python built on PoTo. We compare PoToCG to PyCG, the state of the art for this problem, and show that PoTo produces more complete and more precise call graphs.

Cite As Get BibTex

Ingkarat Rak-amnouykit, Ana Milanova, Guillaume Baudart, Martin Hirzel, and Julian Dolby. PoTo: A Hybrid Andersen’s Points-To Analysis for Python. In 39th European Conference on Object-Oriented Programming (ECOOP 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 333, pp. 27:1-27:29, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.ECOOP.2025.27

Author Details

Ingkarat Rak-amnouykit
  • Rensselaer Polytechnic Institute, Troy, NY, USA
Ana Milanova
  • Rensselaer Polytechnic Institute, Troy, NY, USA
Guillaume Baudart
  • Université Paris Cité, Inria, CNRS, IRIF, France
Martin Hirzel
  • IBM Research, Yorktown Heights, NY, USA
Julian Dolby
  • IBM Research, Yorktown Heights, NY, USA

Funding

US National Science Foundation for supporting our work through awards #1814898 and #2232061.

Acknowledgements

We thank the ECOOP reviewers for their valuable comments.

Supplementary Materials

References

  1. Miltiadis Allamanis, Earl T. Barr, Soline Ducousso, and Zheng Gao. Typilus: Neural type hints. In Conference on Programming Language Design and Implementation (PLDI), pages 91-105, 2020. URL: https://doi.org/10.1145/3385412.3385997.
  2. Lars Ole Andersen. Program Analysis and Specialization for the C Programming Language. PhD thesis, University of Copenhagen, 1994. DIKU report 94/19. URL: ftp://ftp.diku.dk/pub/diku/semantics/papers/D-203.dvi.Z.
  3. David F. Bacon and Peter F. Sweeney. Fast static analysis of C++ virtual function calls. In Proceedings of the 1996 ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages & Applications, OOPSLA 1996, San Jose, California, USA, October 6-10, 1996, pages 324-341. ACM, 1996. URL: https://doi.org/10.1145/236337.236371.
  4. Kim Barrett, Bob Cassels, Paul Haahr, David A. Moon, Keith Playford, and P. Tucker Withington. A monotonic superclass linearization for dylan. In Proceedings of the 11th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA '96, pages 69-82, 1996. URL: https://doi.org/10.1145/236337.236343.
  5. Madhurima Chakraborty, Renzo Olivares, Manu Sridharan, and Behnaz Hassanshahi. Automatic Root Cause Quantification for Missing Edges in JavaScript Call Graphs. In Karim Ali and Jan Vitek, editors, 36th European Conference on Object-Oriented Programming (ECOOP 2022), volume 222 of Leibniz International Proceedings in Informatics (LIPIcs), pages 3:1-3:28, Dagstuhl, Germany, 2022. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.ECOOP.2022.3.
  6. Ramkrishna Chatterjee, Barbara G. Ryder, and William A. Landi. Relevant context inference. In Proceedings of the 26th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL '99, pages 133-146, New York, NY, USA, 1999. Association for Computing Machinery. URL: https://doi.org/10.1145/292540.292554.
  7. Yu-Fang Chen, Wei-Lun Tsai, Wei-Cheng Wu, Di-De Yen, and Fang Yu. Pyct: A python concolic tester. In Asian Symposium on Programming Languages and Systems (APLAS), pages 38-46, 2021. URL: https://doi.org/10.1007/978-3-030-89051-3_3.
  8. Timothy Clem and Patrick Thomson. Static analysis at GitHub. Communications of the ACM (CACM), 65(2):44-51, January 2022. URL: https://doi.org/10.1145/3486594.
  9. Siwei Cui, Gang Zhao, Zeyu Dai, Luochao Wang, Ruihong Huang, and Jeff Huang. PYInfer: Deep learning semantic type inference for python variables, June 2021. URL: https://doi.org/10.48550/arXiv.2106.14316.
  10. Manuel Fähndrich, Jakob Rehof, and Manuvir Das. Scalable context-sensitive flow analysis using instantiation constraints. In Proceedings of the ACM SIGPLAN 2000 Conference on Programming Language Design and Implementation, PLDI '00, pages 253-263, New York, NY, USA, 2000. Association for Computing Machinery. URL: https://doi.org/10.1145/349299.349332.
  11. Levin Fritz and Jurriaan Hage. Cost versus precision for approximate typing for python. In Workshop on Partial Evaluation and Program Manipulation (PEPM), pages 89-98, January 2017. URL: https://doi.org/10.1145/3018882.3018888.
  12. Google. pytype, January 2016. Retrieved June, 2024. URL: https://github.com/google/pytype.
  13. Neville Grech, George Fourtounis, Adrian Francalanza, and Yannis Smaragdakis. Heaps don't lie: countering unsoundness with heap snapshots. Proc. ACM Program. Lang., 1(OOPSLA), October 2017. URL: https://doi.org/10.1145/3133892.
  14. Mostafa Hassan, Caterina Urban, Marco Eilers, and Peter Müller. MaxSMT-based type inference for Python 3. In Conference on Computer Aided Verification (CAV), pages 12-19, 2018. URL: https://doi.org/10.1007/978-3-319-96142-2_2.
  15. Vincent J. Hellendoorn, Christian Bird, Earl T. Barr, and Miltiadis Allamanis. Deep learning type inference. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2018, pages 152-162, 2018. URL: https://doi.org/10.1145/3236024.3236051.
  16. Martin Hirzel, Amer Diwan, and Michael Hind. Pointer analysis in the presence of dynamic class loading. In European Conference for Object-Oriented Programming (ECOOP), pages 96-122, 2004. URL: https://doi.org/10.1007/978-3-540-24851-4_5.
  17. IBM. T. j. watson libraries for analysis (wala), 2006. Retrieved June, 2024. URL: https://github.com/wala/WALA.
  18. Dongseok Jang and Kwang-Moo Choe. Points-to analysis for javascript. In Sung Y. Shin and Sascha Ossowski, editors, Proceedings of the 2009 ACM Symposium on Applied Computing (SAC), Honolulu, Hawaii, USA, March 9-12, 2009, pages 1930-1937. ACM, 2009. URL: https://doi.org/10.1145/1529282.1529711.
  19. Minseok Jeon and Hakjoo Oh. Return of cfa: call-site sensitivity can be superior to object sensitivity even for object-oriented programs. Proceedings of the ACM on Programming Languages, 6(POPL):1-29, 2022. URL: https://doi.org/10.1145/3498720.
  20. C. Lattner and V. Adve. LLVM: a compilation framework for lifelong program analysis & transformation. In Code Generation and Optimization (CGO), pages 75-86, March 2004. URL: https://doi.org/10.1109/CGO.2004.1281665.
  21. Mathias Rud Laursen, Wenyuan Xu, and Anders Møller. Reducing static analysis unsoundness with approximate interpretation. Proceedings of the ACM on Programming Languages, 8(PLDI):1165-1188, 2024. URL: https://doi.org/10.1145/3656424.
  22. Li Li, Jiawei Wang, and Haowei Quan. Scalpel: The Python static analysis framework, February 2022. URL: https://doi.org/10.48550/arXiv.2202.11840.
  23. Yue Li, Tian Tan, Anders Møller, and Yannis Smaragdakis. Precision-guided context sensitivity for pointer analysis. Proc. ACM Program. Lang., 2(OOPSLA):141:1-141:29, 2018. URL: https://doi.org/10.1145/3276511.
  24. Benjamin Livshits, Manu Sridharan, Yannis Smaragdakis, Ondřej Lhoták, J. Nelson Amaral, Bor-Yuh Evan Chang, Samuel Z. Guyer, Uday P. Khedker, Anders Møller, and Dimitrios Vardoulakis. In defense of soundiness: A manifesto. Communications of the ACM (CACM), 58(2):44-46, January 2015. URL: https://doi.org/10.1145/2644805.
  25. Jingbo Lu and Jingling Xue. Precision-preserving yet fast object-sensitive pointer analysis with partial context sensitivity. Proc. ACM Program. Lang., 3(OOPSLA), October 2019. URL: https://doi.org/10.1145/3360574.
  26. Eva Maia, Nelma Moreira, and Rogério Reis. A static type inference for Python. In Workshop on Dynamic Languages and Applications (DYLA), 2011. URL: http://scg.unibe.ch/download/dyla/2011/dyla11_submission_3.pdf.
  27. Amir M. Mir, Evaldas Latoškinas, Sebastian Proksch, and Georgios Gousios. Type4Py: Practical deep similarity learning-based type inference for Python. In International Conference on Software Engineering (ICSE), pages 2241-2252, May 2022. URL: https://doi.org/10.1145/3510003.3510124.
  28. Joonyoung Park, Jihyeok Park, Dongjun Youn, and Sukyoung Ryu. Accelerating javascript static analysis via dynamic shortcuts. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 1129-1140, 2021. URL: https://doi.org/10.1145/3468264.3468556.
  29. Yun Peng, Cuiyun Gao, Zongjie Li, Bowei Gao, David Lo, Qirun Zhang, and Michael Lyu. Static inference meets deep learning: a hybrid type inference approach for Python. In International Conference on Software Engineering (ICSE), pages 2019-2030, May 2022. URL: https://doi.org/10.1145/3510003.3510038.
  30. Michael Pradel, Georgios Gousios, Jason Liu, and Satish Chandra. TypeWriter: Neural type prediction with search-based validation. In Symposium on the Foundations of Software Engineering (FSE), pages 209-220, November 2020. URL: https://doi.org/10.1145/3368089.3409715.
  31. Ingkarat Rak-amnouykit, Daniel McCrevan, Ana Milanova, Martin Hirzel, and Julian Dolby. Python 3 types in the wild: A tale of two type systems. In Dynamic Languages Symposium (DLS), pages 57-70, 2020. URL: https://doi.org/10.1145/3426422.3426981.
  32. Ingkarat Rak-amnouykit, Ana Milanova, Guillaume Baudart, Martin Hirzel, and Julian Dolby. Principled and practical static analysis for Python: Weakest precondition inference of hyperparameter constraints. Software - Practice and Experience (SP&E), 54(3):363-393, March 2024. URL: https://doi.org/10.1002/spe.3279.
  33. Atanas Rountev, Ana L. Milanova, and Barbara G. Ryder. Points-to analysis for java using annotated constraints. In Linda M. Northrop and John M. Vlissides, editors, Proceedings of the 2001 ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages and Applications, OOPSLA 2001, Tampa, Florida, USA, October 14-18, 2001, pages 43-55. ACM, 2001. URL: https://doi.org/10.1145/504282.504286.
  34. Vitalis Salis, Thodoris Sotiropoulos, Panos Louridas, Diomidis Spinellis, and Dimitris Mitropoulos. Pycg: Practical call graph generation in python. In International Conference on Software Engineering (ICSE), pages 1646-1657, May 2021. URL: https://doi.org/10.1109/ICSE43902.2021.00146.
  35. Qingkai Shi, Xiao Xiao, Rongxin Wu, Jinguo Zhou, Gang Fan, and Charles Zhang. Pinpoint: fast and precise sparse value flow analysis for million lines of code. SIGPLAN Not., 53(4):693-706, June 2018. URL: https://doi.org/10.1145/3192366.3192418.
  36. Yannis Smaragdakis and Martin Bravenboer. Using Datalog for fast and easy program analysis. In Datalog Reloaded, 2011. URL: https://doi.org/10.1007/978-3-642-24206-9_14.
  37. Manu Sridharan, Julian Dolby, Satish Chandra, Max Schäfer, and Frank Tip. Correlation tracking for points-to analysis of javascript. In James Noble, editor, ECOOP 2012 - Object-Oriented Programming - 26th European Conference, Beijing, China, June 11-16, 2012. Proceedings, volume 7313 of Lecture Notes in Computer Science, pages 435-458. Springer, 2012. URL: https://doi.org/10.1007/978-3-642-31057-7_20.
  38. Yulei Sui and Jingling Xue. SVF: interprocedural static value-flow analysis in LLVM. In Ayal Zaks and Manuel V. Hermenegildo, editors, Proceedings of the 25th International Conference on Compiler Construction, CC 2016, Barcelona, Spain, March 12-18, 2016, pages 265-266. ACM, 2016. URL: https://doi.org/10.1145/2892208.2892235.
  39. John Toman and Dan Grossman. Concerto: a framework for combined concrete and abstract interpretation. Proc. ACM Program. Lang., 3(POPL), January 2019. URL: https://doi.org/10.1145/3290356.
  40. Omer Tripp, Pietro Ferrara, and Marco Pistoia. Hybrid security analysis of web javascript code via dynamic partial evaluation. In Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014, pages 49-59, 2014. URL: https://doi.org/10.1145/2610384.2610385.
  41. Raja Vallée-Rai. Soot: A Java bytecode optimization framework. Master’s thesis, McGill University, Montreal, October 2000. URL: https://dl.acm.org/citation.cfm?id=782008.
  42. Yin Wang. Pysonar2 - a semantic indexer for python with interprocedural type inference, December 2017. URL: https://github.com/yinwang0/pysonar2.
  43. Shiyi Wei and Barbara G. Ryder. Practical blended taint analysis for javascript. In Proceedings of the 2013 International Symposium on Software Testing and Analysis, ISSTA 2013, pages 336-346, 2013. URL: https://doi.org/10.1145/2483760.2483788.
  44. Zhaogui Xu, Xiangyu Zhang, Lin Chen, Kexin Pei, and Baowen Xu. Python probabilistic type inference with natural language support. In Symposium on the Foundations of Software Engineering (FSE), pages 607-618, 2016. URL: https://doi.org/10.1145/2950290.2950343.
  45. Yanyan Yan. DLInfer dataset, January 2023. URL: https://doi.org/10.5281/zenodo.7575545.
  46. Yanyan Yan, Yang Feng, Hongcheng Fan, and Baowen Xu. DLInfer: Deep learning with static slicing for python type inference. In International Conference on Software Engineering (ICSE), pages 2009-2021, May 2023. URL: https://doi.org/10.1109/ICSE48619.2023.00170.
  47. Yi Yang, Ana L. Milanova, and Martin Hirzel. Complex Python features in the wild. In 19th IEEE/ACM International Conference on Mining Software Repositories, MSR 2022, Pittsburgh, PA, USA, May 23-24, 2022, pages 282-293. ACM, May 2022. URL: https://doi.org/10.1145/3524842.3528467.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact