Document Open Access Logo

Fuzzing as Editor Feedback

Authors Marcel Garus , Jens Lincke , Robert Hirschfeld

PDF HTML 5

Files

Thumbnail PDF
PDF
OASIcs.Programming.2025.8.pdf
  • Filesize: 2.3 MB
  • 15 pages
HTML5 Logo HTML (experimental)
OASIcs.Programming.2025.8.html

Document Identifiers

Subject Classification

ACM Subject Classification
  • Software and its engineering → Development frameworks and environments
  • Software and its engineering → Empirical software validation
  • Software and its engineering → Functionality
Keywords
  • Fuzzing
  • Example-based Programming
  • Babylonian Programming
  • Dynamic Analysis
  • Code Coverage
  • Randomized Testing
  • Function-Level Fuzzing

Metrics

Abstract

Live programming requires concrete examples, but coming up with examples takes effort. However, there are ways to execute code without specifying examples, such as fuzzing. Fuzzing is a technique that synthesizes program inputs to find bugs in security-critical software.
While fuzzing focuses on finding crashes, it also produces valid inputs as a byproduct. Our approach is to make use of this to show examples, including edge cases, directly in the editor.
To provide examples for individual pieces of code, we implement fuzzing at the granularity of functions. We integrate it into the compiler pipeline and language tooling of Martinaise, a custom programming language with a limited feature set. Initially, our examples are random and then mutate based on coverage feedback to reach interesting code locations and become smaller.
We evaluate our tool in small case studies, showing generated examples for numbers, strings, and composite objects. Our fuzzed examples still feel synthetic, but since they are grounded in the dynamic behavior of code, they can cover the entire execution and show edge cases.

Cite As Get BibTex

Marcel Garus, Jens Lincke, and Robert Hirschfeld. Fuzzing as Editor Feedback. In Companion Proceedings of the 9th International Conference on the Art, Science, and Engineering of Programming (Programming 2025). Open Access Series in Informatics (OASIcs), Volume 134, pp. 8:1-8:15, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/OASIcs.Programming.2025.8

Author Details

Marcel Garus
  • Hasso Plattner Institute, University of Potsdam, Germany
Jens Lincke
  • Hasso Plattner Institute, University of Potsdam, Germany
Robert Hirschfeld
  • Hasso Plattner Institute, University of Potsdam, Germany

Supplementary Materials

References

  1. Marcel Böhme, Van-Thuan Pham, Manh-Dung Nguyen, and Abhik Roychoudhury. Directed Greybox Fuzzing. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, Dallas Texas USA, October 2017. ACM. URL: https://doi.org/10.1145/3133956.3134020.
  2. Marcel Böhme, Van-Thuan Pham, and Abhik Roychoudhury. Coverage-based Greybox Fuzzing as Markov Chain. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna Austria, October 2016. ACM. URL: https://doi.org/10.1145/2976749.2978428.
  3. Peng Chen and Hao Chen. Angora: Efficient Fuzzing by Principled Search. In 2018 IEEE Symposium on Security and Privacy (SP), May 2018. URL: https://doi.org/10.1109/SP.2018.00046.
  4. Shuitao Gan, Chao Zhang, Peng Chen, Bodong Zhao, Xiaojun Qin, Dong Wu, and Zuoning Chen. GREYONE: Data Flow Sensitive Fuzzing. In Proceedings of the 29th USENIX Conference on Security Symposium, SEC'20, 2020. URL: https://doi.org/10.5555/3489212.3489357.
  5. Marcel Garus. Martinaise. Software, (visited on 2025-09-08). URL: https://github.com/MarcelGarus/martinaise
    Software Heritage Logo archived version
    full metadata available at: https://doi.org/10.4230/artifacts.24697
  6. Christian Holler, Kim Herzig, and Andreas Zeller. Fuzzing with Code Fragments. In Proceedings of the 21st USENIX Conference on Security Symposium, Security'12, USA, 2012. USENIX Association. URL: https://doi.org/10.5555/2362793.2362831.
  7. Eva Krebs, Patrick Rein, and Robert Hirschfeld. Example Mining: Assisting Example Creation to Enhance Code Comprehension. In Companion Proceedings of the 6th International Conference on the Art, Science, and Engineering of Programming, Porto Portugal, March 2022. ACM. URL: https://doi.org/10.1145/3532512.3535226.
  8. Sorin Lerner. Projection boxes: On-the-fly reconfigurable visualization for live programming. In Regina Bernhaupt, Florian 'Floyd' Mueller, David Verweij, Josh Andres, Joanna McGrenere, Andy Cockburn, Ignacio Avellino, Alix Goguey, Pernille Bjøn, Shengdong Zhao, Briane Paul Samson, and Rafal Kocielnik, editors, CHI '20: CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, April 25-30, 2020, CHI '20, pages 1-7, New York, NY, USA, 2020. ACM. URL: https://doi.org/10.1145/3313831.3376494.
  9. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. CoRR, abs/2305.01210, December 2023. URL: https://doi.org/10.48550/arXiv.2305.01210.
  10. Dominik Christian Maier, Lukas Seidel, and Shinjo Park. Basesafe: baseband sanitized fuzzing through emulation. In René Mayrhofer and Michael Roland, editors, WiSec '20: 13th ACM Conference on Security and Privacy in Wireless and Mobile Networks, Linz, Austria, July 8-10, 2020, pages 122-132. ACM, July 2020. URL: https://doi.org/10.1145/3395351.3399360.
  11. Toni Mattis, Lukas Böhme, Stefan Ramson, Tom Beckmann, Martin C. Rinard, and Robert Hirschfeld. Dimensions of examples: Toward a framework for qualifying examples in programming. In Proceedings of the Programming Experience 2025 (PX/25) Workshop, Programming '25. Schloss Dagstuhl - Leibniz-Zentrum für Informatik (OASIcs), 2025. URL: https://doi.org/10.4230/OASIcs.Programming.2025.22.
  12. Toni Mattis, Eva Krebs, Martin C. Rinard, and Robert Hirschfeld. Examples out of Thin Air: AI-Generated Dynamic Context to Assist Program Comprehension by Example. In Companion Proceedings of the 8th International Conference on the Art, Science, and Engineering of Programming, Programming '24, New York, NY, USA, July 2024. Association for Computing Machinery. URL: https://doi.org/10.1145/3660829.3660845.
  13. Ruijie Meng, Martin Mirchev, Marcel Böhme, and Abhik Roychoudhury. Large Language Model guided Protocol Fuzzing. In Proceedings 2024 Network and Distributed System Security Symposium, San Diego, CA, USA, 2024. Internet Society. URL: https://doi.org/10.14722/ndss.2024.24556.
  14. Barton P. Miller, Lars Fredriksen, and Bryan So. An empirical study of the reliability of UNIX utilities. Communications of the ACM, 33(12), December 1990. URL: https://doi.org/10.1145/96267.96279.
  15. Arghavan Moradi Dakhel, Vahid Majdinasab, Amin Nikanjam, Foutse Khomh, Michel C. Desmarais, and Zhen Ming (Jack) Jiang. GitHub Copilot AI pair programmer: Asset or Liability? Journal of Systems and Software, 203, September 2023. URL: https://doi.org/10.1016/j.jss.2023.111734.
  16. Joe Nelson. The design and use of quickcheck, January 2017. URL: https://begriffs.com/posts/2017-01-14-design-use-quickcheck.html.
  17. Fabio Niephaus, Patrick Rein, Jakob Edding, Jonas Hering, Bastian König, Kolya Opahle, Nico Scordialo, and Robert Hirschfeld. Example-based live programming for everyone: building language-agnostic tools for live programming with LSP and GraalVM. In Proceedings of the 2020 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Virtual USA, November 2020. ACM. URL: https://doi.org/10.1145/3426428.3426919.
  18. David Rauch, Patrick Rein, Stefan Ramson, Jens Lincke, and Robert Hirschfeld. Babylonian-style Programming: Design and Implementation of an Integration of Live Examples into General-purpose Source Code. The Art, Science, and Engineering of Programming, 3(3), February 2019. URL: https://doi.org/10.22152/programming-journal.org/2019/3/9.
  19. Sergej Schumilo, Cornelius Aschermann, Ali Abbasi, Simon Wörner, and Thorsten Holz. Nyx: Greybox hypervisor fuzzing using fast snapshots and affine types. In Michael D. Bailey and Rachel Greenstadt, editors, 30th USENIX Security Symposium, USENIX Security 2021, August 11-13, 2021, pages 2597-2614. USENIX Association, 2021. URL: https://www.usenix.org/conference/usenixsecurity21/presentation/schumilo.
  20. Tijs Van der Storm and Felienne Hermans. Live Literals, 2016. Presented at the Workshop on Live Programming (LIVE). URL: https://homepages.cwi.nl/~storm/livelit/livelit.html.
  21. Bret Victor. Inventing on Principle, February 2012. URL: https://www.youtube.com/watch?v=PUv66718DII.
  22. Yanhao Wang, Xiangkun Jia, Yuwei Liu, Kyle Zeng, Tiffany Bao, Dinghao Wu, and Purui Su. Not all coverage measurements are equal: Fuzzing by coverage accounting for input prioritization. In 27th Annual Network and Distributed System Security Symposium, NDSS 2020, San Diego, California, USA, February 23-26, 2020, San Diego, CA, 2020. The Internet Society. URL: https://doi.org/10.14722/ndss.2020.24422.
  23. Chunqiu Steven Xia, Matteo Paltenghi, Jia Le Tian, Michael Pradel, and Lingming Zhang. Fuzz4All: Universal Fuzzing with Large Language Models. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ICSE '24, New York, NY, USA, April 2024. Association for Computing Machinery. URL: https://doi.org/10.1145/3597503.3639121.
  24. Meng Xu, Sanidhya Kashyap, Hanqing Zhao, and Taesoo Kim. Krace: Data race fuzzing for kernel file systems. In 2020 IEEE Symposium on Security and Privacy, SP 2020, San Francisco, CA, USA, May 18-21, 2020, pages 1643-1660. IEEE, 2020. URL: https://doi.org/10.1109/SP40000.2020.00078.
  25. Michal Zalewski. Binary fuzzing strategies: what works, what doesn't, August 2014. URL: https://lcamtuf.blogspot.com/2014/08/binary-fuzzing-strategies-what-works.html.
  26. Michal Zalewski. Technical "whitepaper" for afl-fuzz, January 2015. URL: https://lcamtuf.coredump.cx/afl/technical_details.txt.
  27. Yaowen Zheng, Ali Davanian, Heng Yin, Chengyu Song, Hongsong Zhu, and Limin Sun. FIRM-AFL: High-Throughput greybox fuzzing of IoT firmware via augmented process emulation. In Proceedings of the 28th USENIX Conference on Security Symposium, SEC'19, USA, 2019. USENIX Association. URL: https://doi.org/10.5555/3361338.3361415.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact