Document Open Access Logo

On Leveraging Tests to Infer Nullable Annotations

Authors Jens Dietrich , David J. Pearce , Mahin Chandramohan

PDF
Thumbnail PDF

File

LIPIcs.ECOOP.2023.10.pdf
  • Filesize: 0.83 MB
  • 25 pages

Document Identifiers

Author Details

Jens Dietrich
  • Victoria University of Wellington, New Zealand
David J. Pearce
  • ConsenSys, Wellington, New Zealand
Mahin Chandramohan
  • Oracle Labs, Brisbane, Australia

Funding

  • Dietrich, Jens: The first author was supported by Oracle Labs, Australia.

Acknowledgements

The authors would like to thank Chris Povirk for his feedback on using our tool on guava, and Görel Hedin for assisting us to set up the experiment reported in Section 7.9.

Cite As Get BibTex

Jens Dietrich, David J. Pearce, and Mahin Chandramohan. On Leveraging Tests to Infer Nullable Annotations. In 37th European Conference on Object-Oriented Programming (ECOOP 2023). Leibniz International Proceedings in Informatics (LIPIcs), Volume 263, pp. 10:1-10:25, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2023) https://doi.org/10.4230/LIPIcs.ECOOP.2023.10

Abstract

Issues related to the dereferencing of null pointers are a pervasive and widely studied problem, and numerous static analyses have been proposed for this purpose. These are typically based on dataflow analysis, and take advantage of annotations indicating whether a type is nullable or not. The presence of such annotations can significantly improve the accuracy of null checkers. However, most code found in the wild is not annotated, and tools must fall back on default assumptions, leading to both false positives and false negatives. Manually annotating code is a laborious task and requires deep knowledge of how a program interacts with clients and components.
We propose to infer nullable annotations from an analysis of existing test cases. For this purpose, we execute instrumented tests and capture nullable API interactions. Those recorded interactions are then refined (santitised and propagated) in order to improve their precision and recall. We evaluate our approach on seven projects from the spring ecosystems and two google projects which have been extensively manually annotated with thousands of @Nullable annotations. We find that our approach has a high precision, and can find around half of the existing @Nullable annotations. This suggests that the method proposed is useful to mechanise a significant part of the very labour-intensive annotation task.

Subject Classification

ACM Subject Classification
  • Software and its engineering → Software defect analysis
  • Software and its engineering → Software reliability
  • Software and its engineering → Dynamic analysis
Keywords
  • null analysis
  • null safety
  • testing
  • program analysis

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Supplementary Materials

References

  1. A tool to detect bugs in Java and C/C++/Objective-C code before it ships. URL: https://fbinfer.com/.
  2. JetBrain Developer Ecosystem Survey 2021, 2021. URL: https://www.jetbrains.com/lp/devecosystem-2021/java/.
  3. C. Andreae, J. Noble, S. Markstrum, and T. Millstein. A framework for implementing pluggable type systems. In Proc. OOPSLA, pages 57-74, 2006. Google Scholar
  4. Nathaniel Ayewah and William Pugh. Null dereference analysis in practice. In Proc. PASTE, pages 65-72. ACM Press, 2010. Google Scholar
  5. Subarno Banerjee, Lazaro Clapp, and Manu Sridharan. Nullaway: Practical type-based null safety for java. In Proc. ESEC/FSE'19, pages 740-750. ACM, 2019. Google Scholar
  6. Al Bessey, Ken Block, Ben Chelf, Andy Chou, Bryan Fulton, Seth Hallem, Charles Henri-Gros, Asya Kamsky, Scott McPeak, and Dawson Engler. A few billion lines of code later: using static analysis to find bugs in the real world. Communications of the ACM, 53(2):66-75, 2010. Google Scholar
  7. Eric Bodden, Andreas Sewe, Jan Sinschek, Hela Oueslati, and Mira Mezini. Taming reflection: Aiding static analysis in the presence of reflection and custom class loaders. In Proc. ICSE'11, pages 241-250. IEEE, 2011. Google Scholar
  8. Mehdi Bouaziz, Francesco Logozzo, and Manuel Fähndrich. Inference of necessary field conditions with abstract interpretation. In Proc. APLAS, pages 173-189. Springer-Verlag, 2012. Google Scholar
  9. Gilad Bracha. Pluggable type systems. In Proc. Workshop on Revival of Dynamic Languages, 2004. Google Scholar
  10. Dan Brotherston, Werner Dietl, and Ondrej Lhoták. Granullar: gradual nullable types for java. In Proc. CC, pages 87-97. ACM Press, 2017. Google Scholar
  11. Eric Bruneton, Romain Lenglet, and Thierry Coupaye. Asm: a code manipulation tool to implement adaptable systems. Adaptable and extensible component systems, 30(19), 2002. Google Scholar
  12. Cristiano Calcagno, Dino Distefano, Jérémy Dubreil, Dominik Gabi, Pieter Hooimeijer, Martino Luca, Peter W. O'Hearn, Irene Papakonstantinou, Jim Purbrick, and Dulma Rodriguez. Moving fast with software verification. In Proc. NFM, pages 3-11. Springer-Verlag, 2015. Google Scholar
  13. Patrice Chalin and Perry R James. Non-null references by default in java: Alleviating the nullity annotation burden. In European Conference on Object-Oriented Programming, pages 227-247. Springer, 2007. Google Scholar
  14. Patrice Chalin, Perry R James, and Frédéric Rioux. Reducing the use of nullable types through non-null by default and monotonic non-null. IET Software, 2(6):515-531, 2008. Google Scholar
  15. Brian Chin, Shane Markstrum, and Todd Millstein. Semantic type qualifiers. In Proc. PLDI, pages 85-95. ACM Press, 2005. Google Scholar
  16. Brian Chin, Shane Markstrum, Todd Millstein, and Jens Palsberg. Inference of user-defined type qualifiers and qualifier rules. In Proc. ESOP, 2006. Google Scholar
  17. Jens Dietrich, David J Pearce, Kamil Jezek, and Premek Brada. Contracts in the wild: A study of java programs. In Proc. ECOOP'17. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2017. Google Scholar
  18. Jens Dietrich, Henrik Schole, Li Sui, and Ewan Tempero. XCorpus - An executable corpus of Java programs. Journal of Object Technology, 16(4):1:1-24, August 2017. URL: https://doi.org/10.5381/jot.2017.16.4.a1.
  19. Dino Distefano, Manuel Fähndrich, Francesco Logozzo, and Peter W. O'Hearn. Scaling static analyses at facebook. CACM, 62(8):62-70, 2019. Google Scholar
  20. Kinga Dobolyi and Westley Weimer. Changing Java’s semantics for handling null pointer exceptions. In Proc. ISSRE, pages 47-56. IEEE Computer Society, 2008. Google Scholar
  21. T. Ekman and G. Hedin. Pluggable checking and inferencing of non-null types for Java. JOT, 6(9):455-475, 2007. Google Scholar
  22. Sam Estep, Jenna Wise, Jonathan Aldrich, Éric Tanter, Johannes Bader, and Joshua Sunshine. Gradual program analysis for null pointers. In European Conference on Object-Oriented Programming (ECOOP), pages 3:1-3:25, 2021. Google Scholar
  23. Manuel Fähndrich and K Rustan M Leino. Declaring and checking non-null types in an object-oriented language. In Proceedings of the 18th annual ACM SIGPLAN conference on Object-oriented programing, systems, languages, and applications, pages 302-312, 2003. Google Scholar
  24. C. Flanagan, K. Leino, M. Lillibridge, G. Nelson, J. B. Saxe, and R. Stata. Extended static checking for Java. In Proc. PLDI, pages 234-245, 2002. Google Scholar
  25. Cormac Flanagan and K Rustan M Leino. Houdini, an annotation assistant for esc/java. In International Symposium of Formal Methods Europe, pages 500-517. Springer, 2001. Google Scholar
  26. Jeffrey S. Foster, Manuel Fähndrich, and Alexander Aiken. A theory of type qualifiers. In Proc. PLDI, pages 192-203, 1999. Google Scholar
  27. Jeffrey S. Foster, Tachio Terauchi, and Alex Aiken. Flow-sensitive type qualifiers. In Proc. PLDI, pages 1-12, 2002. Google Scholar
  28. Gordon Fraser and Andrea Arcuri. Evosuite: automatic test suite generation for object-oriented software. In Proc. FSE'11, pages 416-419, 2011. Google Scholar
  29. Maria Anna G. Gaitani, Vassilis Zafeiris, N. A. Diamantidis, and Emmanouel A. Giakoumakis. Automated refactoring to the null object design pattern. Inf. Softw. Technol, 59, 2015. Google Scholar
  30. Adele Goldberg and David Robson. Smalltalk-80: the language and its implementation. Addison-Wesley Longman Publishing Co., Inc., 1983. Google Scholar
  31. Neville Grech, George Fourtounis, Adrian Francalanza, and Yannis Smaragdakis. Heaps don't lie: countering unsoundness with heap snapshots. In Proc. OOPSLA'17, pages 1-27. ACM, 2017. Google Scholar
  32. Warren Harrison. Eating your own dog food. IEEE Software, 23(3):5-7, 2006. Google Scholar
  33. David Hovemeyer, Jaime Spacco, and William Pugh. Evaluating and tuning a static analysis to find null pointer bugs. In Proc. PASTE, pages 13-19. ACM Press, 2005. Google Scholar
  34. Laurent Hubert. A non-null annotation inferencer for java bytecode. In Proceedings of the 8th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering, pages 36-42, 2008. Google Scholar
  35. Laurent Hubert, Thomas Jensen, and David Pichardie. Semantic foundations and inference of non-null annotations. In Proceedings of the International conference on Formal Methods for Open Object-Based Distributed Systems (FMOODS), pages 132-149. Springer-Verlag, 2008. Google Scholar
  36. Laurent Hubert and David Pichardie. Soundly handling static fields: Issues, semantics and analysis. ENTCS, 253(5):15-30, 2009. Google Scholar
  37. Shuhei Kimura, Keisuke Hotta, Yoshiki Higo, Hiroshi Igaki, and Shinji Kusumoto. Does return null matter? In Conference on Software Maintenance, Reengineering, and Reverse Engineering (CSMR-WCRE), pages 244-253. IEEE, 2014. Google Scholar
  38. Alexander Kogtenkov. Practical void safety. In Proc. VSTTE, pages 132-151. Springer-Verlag, 2017. Google Scholar
  39. Manuel Leuenberger, Haidar Osman, Mohammad Ghafari, and Oscar Nierstrasz. Harvesting the wisdom of the crowd to infer method nullness in java. In 2017 IEEE 17th International Working Conference on Source Code Analysis and Manipulation (SCAM), pages 71-80. IEEE, 2017. Google Scholar
  40. Zhenmin Li, Lin Tan, Xuanhui Wang, Shan Lu, Yuanyuan Zhou, and Chengxiang Zhai. Have things changed now?: an empirical study of bug characteristics in modern open source software. In Proceedings of the Workshop on Architectural and System Support for Improving Software Dependability (ASID), pages 25-33. ACM Press, 2006. Google Scholar
  41. Barbara H Liskov and Jeannette M Wing. A behavioral notion of subtyping. ACM Transactions on Programming Languages and Systems (TOPLAS), 16(6):1811-1841, 1994. Google Scholar
  42. Fengyun Liu, Ondrej Lhoták, Aggelos Biboudis, Paolo G. Giarrusso, and Martin Odersky. A type-and-effect system for object initialization. In Proc. OOPSLA, pages 175:1-175:28, 2020. Google Scholar
  43. Fengyun Liu, Ondrej Lhoták, Enze Xing, and Nguyen Cao Pham. Safe object initialization, abstractly. In Proceedings of the Symposium on Scala, pages 33-43. ACM Press, 2021. Google Scholar
  44. Magnus Madsen and Jaco van de Pol. Relational nullable types with boolean unification. In Proc. OOPSLA, pages 1-28, 2021. Google Scholar
  45. C. Male, D.J. Pearce, A. Potanin, and C. Dymnikov. Java bytecode verification for @NonNull types. In Proc. CC, pages 229-244, 2008. Google Scholar
  46. Gianluca Mezzetti, Anders Møller, and Martin Toldam Torp. Type regression testing to detect breaking changes in node. js libraries. In proc. ECOOP'18. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2018. Google Scholar
  47. Ana Milanova and Wei Huang. Inference and checking of context-sensitive pluggable types. In Proc. ESEC/FSE, page 26. ACM Press, 2012. Google Scholar
  48. Fabian Muehlboeck and Ross Tate. Empowering union and intersection types with integrated subtyping. PACMPL, 2(OOPSLA):112:1-112:29, 2018. Google Scholar
  49. Mangala Gowri Nanda and Saurabh Sinha. Accurate interprocedural null-dereference analysis for java. In 2009 IEEE 31st International Conference on Software Engineering, pages 133-143. IEEE, 2009. Google Scholar
  50. Abel Nieto, Marianna Rapoport, Gregor Richards, and Ondrej Lhoták. Blame for null. In Proc. ECOOP, pages 3:1-3:28, 2020. Google Scholar
  51. Abel Nieto, Yaoyu Zhao, Ondřej Lhoták, Angela Chang, and Justin Pu. Scala with Explicit Nulls. In Proc. ECOOP, volume 166, pages 25:1-25:26, 2020. Google Scholar
  52. Haidar Osman, Manuel Leuenberger, Mircea Lungu, and Oscar Nierstrasz. Tracking null checks in open-source java systems. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), volume 1, pages 304-313. IEEE, 2016. Google Scholar
  53. Carlos Pacheco and Michael D Ernst. Randoop: feedback-directed random testing for java. In Proc. OOPSLA'07, pages 815-816. ACM, 2007. Google Scholar
  54. Matthew M Papi, Mahmood Ali, Telmo Luis Correa Jr, Jeff H Perkins, and Michael D Ernst. Practical pluggable types for java. In Proc. ISSTA'08, pages 201-212. ACM, 2008. Google Scholar
  55. D. J. Pearce. Sound and complete flow typing with unions, intersections and negations. In Proc. VMCAI, pages 335-354, 2013. Google Scholar
  56. P. Pominville, F. Qian, R. Vallée-Rai, L. Hendren, and C. Verbrugge. A framework for optimizing Java using attributes. In Proc. CC, pages 334-554, 2001. Google Scholar
  57. Xin Qi and Andrew C. Myers. Masked types for sound object initialization. In Proc. POPL, pages 53-65. ACM Press, 2009. Google Scholar
  58. Baber Rehman, Xuejing Huang, Ningning Xie, and Bruno C. d. S. Oliveira. Union Types with Disjoint Switches. In Proc. ECOOP, volume 222, pages 25:1-25:31, 2022. Google Scholar
  59. Caitlin Sadowski, Edward Aftandilian, Alex Eagle, Liam Miller-Cushon, and Ciera Jaspan. Lessons from building static analysis tools at google. Communications of the ACM, 61(4):58-66, 2018. Google Scholar
  60. Marco Servetto, Julian Mackay, Alex Potanin, and James Noble. The billion-dollar fix - safe modular circular initialisation with placeholders and placeholder types. In Proc. ECOOP, volume 7920, pages 205-229. Springer-Verlag, 2013. Google Scholar
  61. Jeremy Siek and Walid Taha. Gradual typing for objects. In Proc. ECOOP, pages 151-175. Springer-Verlag, 2007. Google Scholar
  62. Nicholas Smith, Danny Van Bruggen, and Federico Tomassetti. Javaparser: visited. Leanpub, oct. de, 2017. Google Scholar
  63. Fausto Spoto. Nullness analysis in boolean form. In Proc. SEFM, pages 21-30. IEEE, 2008. Google Scholar
  64. Fausto Spoto and Michael D. Ernst. Inference of field initialization. In Proc. ICSE, pages 231-240. ACM Press, 2011. Google Scholar
  65. Li Sui, Jens Dietrich, Amjed Tahir, and George Fourtounis. On the recall of static call graph construction in practice. In Proc. ICSE'20, pages 1049-1060. IEEE, 2020. Google Scholar
  66. Alexander J. Summers and Peter Mueller. Freedom before commitment: A lightweight type system for object initialisation. In Proc. OOPSLA, pages 1013-1032, 2011. Google Scholar
  67. Timothy A. V. Teatro, J. Mikael Eklund, and Ruth Milman. Maybe and either monads in plain C++ 17. In Proc. CCECE, pages 1-4. IEEE, 2018. Google Scholar
  68. Brian Vermeer. Spring dominates the java ecosystem with 60% using it for their main applications, 2020. URL: https://snyk.io/blog/spring-dominates-the-java-ecosystem-with-60-using-it-for-their-main-applications/.
  69. Ying Wang, Ming Wen, Zhenwei Liu, Rongxin Wu, Rui Wang, Bo Yang, Hai Yu, Zhiliang Zhu, and Shing-Chi Cheung. Do the dependency conflicts in my project matter? In Proc. ESEC/FSE'18, pages 319-330, 2018. Google Scholar
  70. Rafael Winterhalter. Byte Buddy - A code generation and manipulation library for creating and modifying Java classes during the runtime, 2014. URL: https://bytebuddy.net/.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact