Document Open Access Logo

PerfBlower: Quickly Detecting Memory-Related Performance Problems via Amplification

Authors Lu Fang, Liang Dou, Guoqing Xu

PDF
Thumbnail PDF

File

LIPIcs.ECOOP.2015.296.pdf
  • Filesize: 0.98 MB
  • 25 pages

Document Identifiers

Author Details

Lu Fang
Liang Dou
Guoqing Xu

Cite As Get BibTex

Lu Fang, Liang Dou, and Guoqing Xu. PerfBlower: Quickly Detecting Memory-Related Performance Problems via Amplification. In 29th European Conference on Object-Oriented Programming (ECOOP 2015). Leibniz International Proceedings in Informatics (LIPIcs), Volume 37, pp. 296-320, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2015) https://doi.org/10.4230/LIPIcs.ECOOP.2015.296

Abstract

Performance problems in managed languages are extremely difficult to find. Despite many efforts to find those problems, most existing work focuses on how to debug a user-provided test execution in which performance problems already manifest. It remains largely unknown how to effectively find performance bugs before software release.  As a result, performance bugs often escape to production runs, hurting software reliability and user experience. This paper describes PerfBlower, a general performance testing framework that allows developers to quickly test Java programs to find memory-related performance problems. PerfBlower provides (1) a novel specification language ISL to describe a general class of performance problems that have observable symptoms; (2) an automated test oracle via \emph{virtual amplification}; and (3) precise reference-path-based diagnostic information via object   mirroring.  Using this framework, we have amplified three different types of problems. Our experimental results demonstrate that (1) ISL is expressive enough to describe various memory-related performance problems; (2) PerfBlower successfully distinguishes executions with and without problems; 8 unknown problems are quickly discovered under small workloads; and (3) PerfBlower outperforms existing detectors and does not miss any bugs studied before in the literature.

Subject Classification

Keywords
  • Performance bugs
  • memory problems
  • managed languages
  • garbage collection}

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

References

  1. Apache JIRA issue tracker. URL: https://issues.apache.org/jira/browse/XALANJ-796.
  2. S. M. Blackburn and et al. The DaCapo benchmarks: Java benchmarking development and analysis. In OOPSLA, pages 169-190, 2006. Google Scholar
  3. Michael D. Bond and Kathryn S. McKinley. Bell: Bit-encoding online memory leak detection. In ASPLOS, pages 61-72, 2006. Google Scholar
  4. Michael D. Bond and Kathryn S. McKinley. Probabilistic calling context. In OOPSLA, pages 97-112, 2007. Google Scholar
  5. Michael D. Bond and Kathryn S. McKinley. Leak pruning. In ASPLOS, pages 277-288, 2009. Google Scholar
  6. Randal E. Bryant and David R. O'Hallaron. Computer Systems: A Programmer’s Perspective. Addison-Wesley, 2010. Google Scholar
  7. Cristian Cadar, Vijay Ganesh, Peter M. Pawlowski, David L. Dill, and Dawson R. Engler. EXE: automatically generating inputs of death. In CCS, pages 322-335, 2006. Google Scholar
  8. Dawson R. Engler. Incorporating application semantics and control into compilation. In DSL, pages 9-9, 1997. Google Scholar
  9. Patrice Godefroid, Nils Klarlund, and Koushik Sen. DART: directed automated random testing. In PLDI, pages 213-223, 2005. Google Scholar
  10. Brian Grant, Markus Mock, Matthai Philipose, Craig Chambers, and Susan J. Eggers. DyC: an expressive annotation-directed dynamic compiler for C. Theor. Comput. Sci., 248(1-2):147-199, October 2000. Google Scholar
  11. Samuel Z. Guyer and Calvin Lin. An annotation language for optimizing software libraries. In DSL, pages 39-52, 1999. Google Scholar
  12. Shi Han, Yingnong Dang, Song Ge, Dongmei Zhang, and Tao Xie. Performance debugging in the large via mining millions of stack traces. In ICSE, pages 145-155, 2012. Google Scholar
  13. Guoliang Jin, Linhai Song, Xiaoming Shi, Joel Scherpelz, and Shan Lu. Understanding and detecting real-world performance bugs. In PLDI, pages 77-88, 2012. Google Scholar
  14. Maria Jump and Kathryn S. McKinley. Cork: Dynamic memory leak detection for garbage-collected languages. In POPL, pages 31-38, 2007. Google Scholar
  15. JUnitPerf. http://www.clarkware.com/software/JUnitPerf.html, 2003.
  16. Alan Leung, Manish Gupta, Yuvraj Agarwal, Rajesh Gupta, Ranjit Jhala, and Sorin Lerner. Verifying GPU kernels by test amplification. In PLDI, pages 383-394, 2012. Google Scholar
  17. Load Test Tools. http://www.softwareqatest.com/qatweb1.html, 2015.
  18. Microbenchmarking framework for Java. https://code.google.com/p/caliper/, 2013.
  19. Nick Mitchell and Gary Sevitsky. The causes of bloat, the limits of health. In OOPSLA, pages 245-260, 2007. Google Scholar
  20. Nick Mitchell, Gary Sevitsky, and Harini Srinivasan. Modeling runtime behavior in framework-based applications. In ECOOP, pages 429-451, 2006. Google Scholar
  21. Adrian Nistor, Po-Chun Chang, Cosmin Radoi, and Shan Lu. Caramel: Detecting and fixing performance problems that have non-intrusive fixes. In ICSE, 2015. Google Scholar
  22. Adrian Nistor, Lintao Song, Darko Marinov, and Shan Lu. Toddler: Detecting performance problems via similar memory-access patterns. In ICSE, pages 562-571, 2013. Google Scholar
  23. Oswaldo Olivo, Isil Dillig, and Calvin Lin. Static detection of asymptotic performance bugs in collection traversals. In PLDI, 2015. Google Scholar
  24. Christoph Reichenbach, Neil Immerman, Yannis Smaragdakis, Edward Aftandilian, and Samuel Z. Guyer. What can the GC compute efficiently? A language for heap assertions at GC time. In OOPSLA, pages 256-269, 2010. Google Scholar
  25. Koushik Sen, Darko Marinov, and Gul Agha. CUTE: a concolic unit testing engine for C. In FSE, pages 263-272, 2005. Google Scholar
  26. Ohad Shacham, Martin Vechev, and Eran Yahav. Chameleon: Adaptive selection of collections. In PLDI, pages 408-418, 2009. Google Scholar
  27. Yannis Smaragdakis and Don Batory. DiSTiL: a transformation library for data structures. In DSL, pages 20-20, 1997. Google Scholar
  28. Linhai Song and Shan Lu. Statistical debugging for real-world performance problems. In OOPSLA, pages 561-578, 2014. Google Scholar
  29. C. Stewart, Kai Shen, A. Iyengar, and Jian Yin. Entomomodel: Understanding and avoiding performance anomaly manifestations. In MASCOTS, pages 3-13, 2010. Google Scholar
  30. The Grinder Java load testing framework. http://grinder.sourceforge.net/, 2013.
  31. The SmartBear distributed testing framework. http://support.smartbear.com/articles/testcomplete/distributed-testing-tutorial, 2013.
  32. M. Vandevoorde. Exploiting Specifications to Improve Program Performance. PhD thesis, Massachusetts Institute of Technology, 1994. Google Scholar
  33. Jeannette M. Wing. Writing larch interface language specifications. ACM Transactions on Programming Languages and Systems, 9(1):1-24, January 1987. Google Scholar
  34. Xusheng Xiao, Shi Han, Dongmei Zhang, and Tao Xie. Context-sensitive delta inference for identifying workload-dependent performance bottlenecks. In ISSTA, pages 90-100, 2013. Google Scholar
  35. Guoqing Xu, Matthew Arnold, Nick Mitchell, Atanas Rountev, and Gary Sevitsky. Go with the flow: Profiling copies to find runtime bloat. In PLDI, pages 419-430, 2009. Google Scholar
  36. Guoqing Xu, Michael D. Bond, Feng Qin, and Atanas Rountev. LeakChaser: Helping programmers narrow down causes of memory leaks. In PLDI, pages 270-282, 2011. Google Scholar
  37. Guoqing Xu, Nick Mitchel, Matthew Arnold, Atanas Rountev, Edith Schonberg, and Gary Sevitsky. Finding low-utility data structures. In PLDI, pages 174-186, 2010. Google Scholar
  38. Guoqing Xu and Atanas Rountev. Precise memory leak detection for Java software using container profiling. In ICSE, pages 151-160, 2008. Google Scholar
  39. Guoqing Xu and Atanas Rountev. Detecting inefficiently-used containers to avoid bloat. In PLDI, pages 160-173, 2010. Google Scholar
  40. Guoqing Xu, Dacong Yan, and Atanas Rountev. Static detection of loop-invariant data structures. In ECOOP, pages 738-763, 2012. Google Scholar
  41. YourKit Java Profiling. http://www.yourkit.com, 2015.
  42. Xiao Yu, Shi Han, Dongmei Zhang, and Tao Xie. Comprehending performance from real-world execution traces: A device-driver case. In ASPLOS, pages 193-206, 2014. Google Scholar
  43. Pingyu Zhang and Sebastian Elbaum. Amplifying tests to validate exception handling code. In ICSE, pages 595-605, 2012. Google Scholar
  44. Bowen Zhou, Jonathan Too, Milind Kulkarni, and Saurabh Bagchi. WuKong: automatically detecting and localizing bugs that manifest at large system scales. In HPDC, pages 131-142, 2013. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact