Document Open Access Logo

Automated Large-Scale Multi-Language Dynamic Program Analysis in the Wild (Tool Insights Paper)

Authors Alex Villazón , Haiyang Sun, Andrea Rosà, Eduardo Rosales , Daniele Bonetta, Isabella Defilippis, Sergio Oporto, Walter Binder

PDF
Thumbnail PDF

File

LIPIcs.ECOOP.2019.20.pdf
  • Filesize: 0.75 MB
  • 27 pages

Document Identifiers

Author Details

Alex Villazón
  • Universidad Privada Boliviana, Bolivia
Haiyang Sun
  • Università della Svizzera italiana, Switzerland
Andrea Rosà
  • Università della Svizzera italiana, Switzerland
Eduardo Rosales
  • Università della Svizzera italiana, Switzerland
Daniele Bonetta
  • Oracle Labs, United States
Isabella Defilippis
  • Universidad Privada Boliviana, Bolivia
Sergio Oporto
  • Universidad Privada Boliviana, Bolivia
Walter Binder
  • Università della Svizzera italiana, Switzerland

Acknowledgements

This work has been supported by Oracle (ERO project 1332), Swiss National Science Foundation (scientific exchange project IZSEZ0_177215), Hasler Foundation (project 18012), and by a Bridging Grant with Japan (BG 04-122017).

Cite AsGet BibTex

Alex Villazón, Haiyang Sun, Andrea Rosà, Eduardo Rosales, Daniele Bonetta, Isabella Defilippis, Sergio Oporto, and Walter Binder. Automated Large-Scale Multi-Language Dynamic Program Analysis in the Wild (Tool Insights Paper). In 33rd European Conference on Object-Oriented Programming (ECOOP 2019). Leibniz International Proceedings in Informatics (LIPIcs), Volume 134, pp. 20:1-20:27, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2019)
https://doi.org/10.4230/LIPIcs.ECOOP.2019.20

Abstract

Today’s availability of open-source software is overwhelming, and the number of free, ready-to-use software components in package repositories such as NPM, Maven, or SBT is growing exponentially. In this paper we address two straightforward yet important research questions: would it be possible to develop a tool to automate dynamic program analysis on public open-source software at a large scale? Moreover, and perhaps more importantly, would such a tool be useful? We answer the first question by introducing NAB, a tool to execute large-scale dynamic program analysis of open-source software in the wild. NAB is fully-automatic, language-agnostic, and can scale dynamic program analyses on open-source software up to thousands of projects hosted in code repositories. Using NAB, we analyzed more than 56K Node.js, Java, and Scala projects. Using the data collected by NAB we were able to (1) study the adoption of new language constructs such as JavaScript Promises, (2) collect statistics about bad coding practices in JavaScript, and (3) identify Java and Scala task-parallel workloads suitable for inclusion in a domain-specific benchmark suite. We consider such findings and the collected data an affirmative answer to the second question.

Subject Classification

ACM Subject Classification
  • Software and its engineering → Dynamic analysis
Keywords
  • Dynamic program analysis
  • code repositories
  • GitHub
  • Node.js
  • Java
  • Scala
  • promises
  • JIT-unfriendly code
  • task granularity

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Supplementary Materials

References

  1. S. Alimadadi, D. Zhong, M. Madsen, and F. Tip. Finding Broken Promises in Asynchronous JavaScript Programs. OOPSLA, pages 162:1-162:26, 2018. Google Scholar
  2. Atlassian. Bitbucket API. https://developer.atlassian.com/bitbucket/api/2/reference/, 2018.
  3. R. Bell Jr and L. K. John. The Case for Automatic Synthesis of Miniature Benchmarks. In MoBS, pages 4-8, 2005. Google Scholar
  4. D. Bernstein. Containers and Cloud: From LXC to Docker to Kubernetes. IEEE Cloud Computing, 1(3):81-84, 2014. Google Scholar
  5. S. M. Blackburn, R. Garner, C. Hoffmann, A. M. Khang, K. S. McKinley, R. Bentzur, et al. The DaCapo Benchmarks: Java Benchmarking Development and Analysis. In OOPSLA, pages 169-190, 2006. Google Scholar
  6. C. F. Bolz, L. Diekmann, and L. Tratt. Storage Strategies for Collections in Dynamically Typed Languages. In OOPSLA, pages 167-182, 2013. Google Scholar
  7. R. P.L. Buse and W. R. Weimer. Automatically Documenting Program Changes. In ASE, pages 33-42, 2010. Google Scholar
  8. B. Cornelissen, A. Zaidman, A. van Deursen, L. Moonen, and R. Koschke. A Systematic Survey of Program Comprehension through Dynamic Analysis. IEEE Transactions on Software Engineering, 35(5):684-702, 2009. Google Scholar
  9. Oracle Corporation. Java SE HotSpot at a Glance. https://www.oracle.com/technetwork/java/javase/tech/index-jsp-136373.html, 2018.
  10. H. J. Curnow and Brian A. Wichmann. A Synthetic Benchmark. Comput. J., 19:43-49, 1976. Google Scholar
  11. J. Dietrich, D.J. Pearce, K. Jezek, and P. Brada. Contracts in the Wild: A Study of Java Programs. In ECOOP, pages 9:1-9:29, 2017. Google Scholar
  12. Y. Ding, M. Zhou, Z. Zhao, S. Eisenstat, and X. Shen. Finding the Limit: Examining the Potential and Complexity of Compilation Scheduling for JIT-based Runtime Systems. In ASPLOS, pages 607-622, 2014. Google Scholar
  13. L. Eeckhout, K. de Bosschere, and H. Neefs. Performance Analysis Through Synthetic Trace Generation. In ISPASS, pages 1-6, 2000. Google Scholar
  14. International Organization for Standardization. ISO/IEC 20922:2016. https://www.iso.org/standard/69466.html, 2018.
  15. GHArchive. Public GitHub Timeline and Archive. https://www.gharchive.org/, 2018.
  16. GitLab. GitLab API. https://docs.gitlab.com/ee/api/, 2018.
  17. L. Gong, M. Pradel, and K. Sen. JITProf: Pinpointing JIT-unfriendly JavaScript Code. In ESEC/FSE, pages 357-368, 2015. Google Scholar
  18. G. Gousios. The GHTorrent Dataset and Tool Suite. In MSR, pages 233-236, 2013. Google Scholar
  19. HAProxy. HAProxy Community Edition. http://www.haproxy.org/, 2018.
  20. B. Hindman, A. Konwinski, M. Zaharia, A. Ghodsi, A. D. Joseph, R. Katz, S. Shenker, and I. Stoica. Mesos: A Platform for Fine-grained Resource Sharing in the Data Center. In NSDI, pages 295-308, 2011. Google Scholar
  21. Docker Inc. Docker Compose. https://docs.docker.com/compose/, 2018.
  22. Docker Inc. Docker Technology. https://docs.docker.com/, 2018.
  23. Docker Inc. Swarm Mode Overview. https://docs.docker.com/engine/swarm/, 2018.
  24. GitHub Inc. REST API v3. https://developer.github.com/v3/search/, 2018.
  25. MongoDB Inc. mongos. https://docs.mongodb.com/manual/reference/program/mongos/, 2018.
  26. MongoDB Inc. Scalable and Flexible document database. https://www.mongodb.com/, 2018.
  27. MongoDB Inc. Sharding. https://docs.mongodb.com/manual/sharding/, 2018.
  28. Innovative Computing Laboratory (ICL) - University of Tennessee. PAPI. http://icl.utk.edu/papi/, 2017.
  29. ECMA International. ECMAScript 2015 Language Specification (ECMA-262 6th Edition). https://www.ecma-international.org/ecma-262/6.0/, 2015.
  30. ECMA International. ECMAScript 2017 Language Specification (ECMA-262 8th Edition). https://www.ecma-international.org/ecma-262/8.0/, 2017.
  31. D. Jin, P. O. N. Meredith, C. Lee, and G. Roşu. JavaMOP: Efficient Parametric Runtime Monitoring Framework. In ICSE, pages 1427-1430, 2012. Google Scholar
  32. A. Joshi, L. Eeckhout, and L. K. John. The Return of Synthetic Benchmarks. In SPEC Benchmark Workshop, pages 1-11, 2008. Google Scholar
  33. E. Kalliamvakou, G. Gousios, K. Blincoe, L. Singer, D. M. German, and D. Damian. The Promises and Perils of Mining GitHub. In MSR, pages 92-101, 2014. Google Scholar
  34. G. Kiczales, E. Hilsdale, J. Hugunin, M. Kersten, J. Palm, and W. G. Griswold. An Overview of AspectJ. In ECOOP, pages 327-353, 2001. Google Scholar
  35. O. Legunsen, W. U. Hassan, X. Xu, G. Roşu, and D. Marinov. How Good Are the Specs? A Study of the Bug-Finding Effectiveness of Existing Java API Specifications. In ASE, pages 602-613, 2016. Google Scholar
  36. Lightbend, Inc. Cluster Specification. https://doc.akka.io/docs/akka/2.5/common/cluster.html, 2018.
  37. C. V. Lopes, P. Maj, P. Martins, V. Saini, D. Yang, J. Zitny, H. Sajnani, and J. Vitek. DéjàVu: A Map of Code Duplicates on GitHub. OOPSLA, pages 84:1-84:28, 2017. Google Scholar
  38. C. V. Lopes and J. Ossher. How Scale Affects Structure in Java Programs. In OOPSLA, pages 675-694, 2015. Google Scholar
  39. Y. Lu, X. Mao, Z. Li, Y. Zhang, T. Wang, and G. Yin. Does the Role Matter? An Investigation of the Code Quality of Casual Contributors in GitHub. In APSEC, pages 49-56, 2016. Google Scholar
  40. M. Madsen, O. Lhoták, and F. Tip. A Model for Reasoning About JavaScript Promises. OOPSLA, pages 86:1-86:24, 2017. Google Scholar
  41. L. Marek, A. Villazón, Y. Zheng, D. Ansaloni, W. Binder, and Z. Qi. DiSL: A Domain-specific Language for Bytecode Instrumentation. In AOSD, pages 239-250, 2012. Google Scholar
  42. P. D. Marinescu, P. Hosek, and C. Cadar. COVRIG: A Framework for the Analysis of Code, Test, and Coverage Evolution in Real Software. In ISSTA, pages 93-104, 2014. Google Scholar
  43. Mercurial. Mercurial Source Control Management. https://www.mercurial-scm.org/, 2018.
  44. Microsoft. Microsoft Azure. https://azure.microsoft.com/en-us/, 2018.
  45. A. Møller and M. I. Schwartzbach. Static Program Analysis, October 2018. Department of Computer Science, Aarhus University, http://cs.au.dk/~amoeller/spa/. Google Scholar
  46. R. Mudduluru and M. K. Ramanathan. Efficient Flow Profiling for Detecting Performance Bugs. In ISSTA, pages 413-424, 2016. Google Scholar
  47. S. Nussbaum and J. E. Smith. Modeling Superscalar Processors via Statistical Simulation. In PACT, pages 15-24, 2001. Google Scholar
  48. OpenJDK. JMH - Java Microbenchmark Harness. http://openjdk.java.net/projects/code-tools/jmh/, 2018.
  49. M. Oskin, F. T. Chong, and M. Farrens. HLS: Combining Statistical and Symbolic Simulation to Guide Microprocessor Designs. In ISCA, pages 71-82, 2000. Google Scholar
  50. C. Pacheco, S. K. Lahiri, M. D. Ernst, and T. Ball. Feedback-Directed Random Test Generation. In ICSE, pages 75-84, May 2007. Google Scholar
  51. S. Rastkar, G. C. Murphy, and G. Murray. Summarizing Software Artifacts: A Case Study of Bug Reports. In ICSE, pages 505-514, 2010. Google Scholar
  52. B. Ray, D. Posnett, P. Devanbu, and V. Filkov. A Large-scale Study of Programming Languages and Code Quality in GitHub. CACM, pages 91-100, 2017. Google Scholar
  53. Veselin Raychev, Martin Vechev, and Andreas Krause. Predicting Program Properties from "Big Code". In POPL, pages 111-124, 2015. Google Scholar
  54. A. Rosà, E. Rosales, and W. Binder. Analyzing and Optimizing Task Granularity on the JVM. In CGO, pages 27-37, 2018. Google Scholar
  55. O. Rudenko. Best Practices for Using Promises in JS. https://60devs.com/best-practices-for-using-promises-in-js.html, 2015.
  56. Scala Benchmarking Project. ScalaBench. http://www.scalabench.org/, 2018.
  57. K. Sen, S. Kalasapur, T. Brutch, and S. Gibbs. Jalangi: A Selective Record-replay and Dynamic Analysis Framework for JavaScript. In ESEC/FSE, pages 488-498, 2013. Google Scholar
  58. A. Sewe, M. Mezini, A. Sarimbekov, and W. Binder. DaCapo con Scala: Design and Analysis of a Scala Benchmark Suite for the Java Virtual Machine. In OOPSLA, pages 657-676, 2011. Google Scholar
  59. S. Shamshiri, R. Just, J. M. Rojas, G. Fraser, P. McMinn, and A. Arcuri. Do Automatically Generated Unit Tests Find Real Faults? An Empirical Study of Effectiveness and Challenges (T). In ASE, 2015, pages 201-211, 2015. Google Scholar
  60. K. Shiv, K. Chow, Y. Wang, and D. Petrochenko. SPECjvm2008 Performance Characterization. In SPEC Benchmark Workshop on Computer Performance Evaluation and Benchmarking, pages 17-35, 2009. Google Scholar
  61. L. A. Smith, J. M. Bull, and J. Obdrzálek. A Parallel Java Grande Benchmark Suite. In SC, pages 6-16, 2001. Google Scholar
  62. H. Sun, D. Bonetta, C. Humer, and W. Binder. Efficient Dynamic Analysis for Node.Js. In CC, pages 196-206, 2018. Google Scholar
  63. The JUnit Team. JUnit. https://junit.org, 2018.
  64. The DaCapo Benchmark Suite. DaCapo. http://www.dacapobench.org/, 2018.
  65. The Netty project. Netty project. https://netty.io/, 2018.
  66. L. Van Ertvelde and L. Eeckhout. Benchmark Synthesis for Architecture and Compiler Exploration. In IISWC, pages 1-11, 2010. Google Scholar
  67. R. P. Weicker. Dhrystone: A Synthetic Systems Programming Benchmark. Commun. ACM, 27(10):1013-1030, 1984. Google Scholar
  68. T. Würthinger, C. Wimmer, C. Humer, A. Wöß, L. Stadler, C. Seaton, G. Duboscq, D. Simon, and M. Grimmer. Practical Partial Evaluation for High-performance Dynamic Language Runtimes. In PLDI, pages 662-676, 2017. Google Scholar
  69. Y. Zheng, A. Rosà, L. Salucci, Y. Li, H. Sun, O. Javed, L. Bulej, L. Y. Chen, Z. Qi, and W. Binder. AutoBench: Finding Workloads That You Need Using Pluggable Hybrid Analyses. In SANER, pages 639-643, 2016. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact