Document Open Access Logo

Optimizing Record/Replay Through Relaxed Total Ordering and Multi-Version eXecution

Authors David Schwartz , Luís Pina

PDF

File

Thumbnail PDF
PDF
LIPIcs.ECOOP.2026.24.pdf
  • Filesize: 1.25 MB
  • 28 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Software and its engineering → Software post-development issues
  • Software and its engineering → Software reliability
  • Software and its engineering → Software maintenance tools
  • Software and its engineering → Software testing and debugging
Keywords
  • Record Replay
  • Multi-version Execution
  • Java Virtual Machine

Metrics

Abstract

Record/Replay (RR) allows developers to record an execution and then replay it exactly as it was recorded. RR enables deterministic replay of non-deterministic behaviors in a different environment than the one used for the recording, which can capture complex bugs in production and find the root cause during development. Unfortunately, support for RR still introduces a non-negligible amount of performance overhead, which limits its applicability. Two main sources of such overhead in state-of-the-art RR systems are: multi-threading, and I/O bound workloads. To ensure high-fidelity when replaying multi-threading execution, recordings either capture the total order of events, which the replayer then enforces, or capture a partial order that requires further processing before replay. Recording also effectively doubles the I/O performed as the recorder needs to perform the original I/O and then record it to a log. Such increased I/O severely limits the performance of I/O dominated workloads.
In this paper, we present two complimentary techniques to reduce the overhead of RR. First, we introduce Relaxed Total Order (RTO), an online-computable weakening of total order that preserves the cross-thread constraints needed for replay while avoiding unnecessary serialization. We design RTO to be compatible with Multi-Version eXecution (MVX), enabling online deterministic replay without pre-processing the recording log or heavyweight coordination. We formalize RTO’s strictness and correctness, showing that it is a novel point between partial- and total-order. Our prototype implementation on top of an existing state-of-the-art RR system reduces recording overhead from 21.0% to 15.3% and halves replay overhead from 67.5% to 31.7%.
Second, we combine RR with Multi-Version eXecution (MVX) to eliminate RR’s poor performance on I/O-bound workloads. Our hybrid design uses a follower variant to absorb the extra I/O needed for logging, and to backfill as much I/O as possible from the same underlying system, keeping the user-facing leader off the critical path. Our prototype reduces the overhead required to record I/O bound programs from 192.5% to just 25.5%, without penalizing other more common workloads. Together, RTO and hybrid MVX/RR substantially narrow the gap between today’s RR systems and practical, low-overhead, always-on deployment.

Cite As Get BibTex

David Schwartz and Luís Pina. Optimizing Record/Replay Through Relaxed Total Ordering and Multi-Version eXecution. In 40th European Conference on Object-Oriented Programming (ECOOP 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 372, pp. 24:1-24:28, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.ECOOP.2026.24

Author Details

David Schwartz
  • University of Illinois Chicago, IL, USA
Luís Pina
  • University of Illinois Chicago, IL, USA

Funding

This work was funded in part by NSF CCF-2227183.

Acknowledgements

We would like to thank: William Mansky for his review of early versions of our formal model, John Fike for IT support during development and testing, and the anonymous reviewers for their insightful comments.

Supplementary Materials

References

  1. Luca Aceto, Duncan Paul Attard, Adrian Francalanza, and Anna Ingólfsdóttir. On benchmarking for concurrent runtime verification. In International Conference on Fundamental Approaches to Software Engineering, pages 3-23. Springer, 2021. URL: https://doi.org/10.1007/978-3-030-71500-7_1.
  2. Luca Aceto, Duncan Paul Attard, Adrian Francalanza, and Anna Ingólfsdóttir. Runtime Instrumentation for Reactive Components. LIPIcs, Volume 313, ECOOP 2024, 313:2:1-2:33, 2024. URL: https://doi.org/10.4230/LIPIcs.ECOOP.2024.2.
  3. Paul-Antoine Arras, Anastasios Andronidis, Luís Pina, Karolis Mituzas, Qianyi Shu, Daniel Grumberg, and Cristian Cadar. SaBRe: Load-time selective binary rewriting. International Journal on Software Tools for Technology Transfer, 24(2):205-223, April 2022. URL: https://doi.org/10.1007/S10009-021-00644-W.
  4. Jonathan Bell and Luís Pina. CROCHET: Checkpoint and Rollback via Lightweight Heap Traversal on Stock JVMs. In Todd Millstein, editor, 32nd European Conference on Object-Oriented Programming (ECOOP 2018), volume 109 of Leibniz International Proceedings in Informatics (LIPIcs), pages 17:1-17:31, Dagstuhl, Germany, 2018. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.ECOOP.2018.17.
  5. Jonathan Bell, Nikhil Sarda, and Gail Kaiser. Chronicler: Lightweight recording to reproduce field failures. In 2013 35th International Conference on Software Engineering (ICSE), pages 362-371, 2013. URL: https://doi.org/10.1109/ICSE.2013.6606582.
  6. Stephen M. Blackburn, Robin Garner, Chris Hoffmann, Asjad M. Khang, Kathryn S. McKinley, Rotem Bentzur, Amer Diwan, Daniel Feinberg, Daniel Frampton, Samuel Z. Guyer, Martin Hirzel, Antony Hosking, Maria Jump, Han Lee, J. Eliot B. Moss, Aashish Phansalkar, Darko Stefanović, Thomas VanDrunen, Daniel von Dincklage, and Ben Wiedermann. The DaCapo benchmarks: Java benchmarking development and analysis. In Proceedings of the 21st Annual ACM SIGPLAN Conference on Object-oriented Programming Systems, Languages, and Applications, OOPSLA '06, pages 169-190, New York, NY, USA, 2006. Association for Computing Machinery. URL: https://doi.org/10.1145/1167473.1167488.
  7. Michael D. Bond, Milind Kulkarni, Man Cao, Meisam Fathi Salmi, and Jipeng Huang. Efficient Deterministic Replay of Multithreaded Executions in a Managed Language Virtual Machine. In Proceedings of the Principles and Practices of Programming on The Java Platform, PPPJ '15, pages 90-101, New York, NY, USA, 2015. Association for Computing Machinery. URL: https://doi.org/10.1145/2807426.2807434.
  8. Michael D. Bond, Milind Kulkarni, Man Cao, Minjia Zhang, Meisam Fathi Salmi, Swarnendu Biswas, Aritra Sengupta, and Jipeng Huang. OCTET: Capturing and controlling cross-thread dependences efficiently. SIGPLAN Not., 48(10):693-712, 2013. URL: https://doi.org/10.1145/2544173.2509519.
  9. David Dice, Mark S. Moir, and William N. Scherer III. Quickly reacquirable locks, October 2010. US Patent. Google Scholar
  10. Andy Georges, Mark Christiaens, Michiel Ronsse, and Koenraad De Bosschere. JaRec: A portable record/replay environment for multi-threaded Java applications. Software: practice and experience, 34(6):523-547, 2004. URL: https://doi.org/10.1002/SPE.579.
  11. Maurice Herlihy and Nir Shavit. The Art of Multiprocessor Programming. Morgan Kaufmann, Burlington, MA, 2008. Google Scholar
  12. Petr Hosek and Cristian Cadar. Safe software updates via multi-version execution. In 2013 35th International Conference on Software Engineering (ICSE), pages 612-621, 2013. URL: https://doi.org/10.1109/ICSE.2013.6606607.
  13. Petr Hosek and Cristian Cadar. VARAN the Unbelievable: An Efficient N-version Execution Framework. ACM SIGARCH Computer Architecture News, 43(1):339-353, 2015. URL: https://doi.org/10.1145/2786763.2694390.
  14. Jeff Huang, Peng Liu, and Charles Zhang. LEAP: Lightweight deterministic multi-processor replay of concurrent java programs. In Proceedings of the Eighteenth ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 207-216, New York, NY, USA, 2010. ACM. URL: https://doi.org/10.1145/1882291.1882323.
  15. Jong-Deok Choi, B. Alpern, T. Ngo, M. Sridharan, and J. Vlissides. A perturbation-free replay platform for cross-optimized multithreaded applications. In Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001, pages 10 pp.-. IEEE, 2001. URL: https://doi.org/10.1109/IPDPS.2001.924957.
  16. Oren Laadan, Nicolas Viennot, and Jason Nieh. Transparent, lightweight application execution replay on commodity multiprocessor operating systems. ACM SIGMETRICS Performance Evaluation Review, 38(1):155-166, 2010. URL: https://doi.org/10.1145/1811099.1811057.
  17. Leslie Lamport. Time, clocks, and the ordering of events in a distributed system. In Concurrency: The Works of Leslie Lamport, pages 179-196. Association for Computing Machinery, New York, NY, USA, October 2019. URL: https://doi.org/10.1145/3335772.3335934.
  18. Dongyoon Lee, Benjamin Wester, Kaushik Veeraraghavan, Satish Narayanasamy, Peter M. Chen, and Jason Flinn. Respec: Efficient online multiprocessor replayvia speculation and external determinism. SIGPLAN notices, 45(3):77-90, 2010. Google Scholar
  19. Jeremy Manson, William Pugh, and Sarita V. Adve. The java memory model. In Proceedings of the 32nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL '05, pages 378-391, New York, NY, USA, 2005. Association for Computing Machinery. URL: https://doi.org/10.1145/1040305.1040336.
  20. Ali José Mashtizadeh, Tal Garfinkel, David Terei, David Mazieres, and Mendel Rosenblum. Towards Practical Default-On Multi-Core Record/Replay. Computer architecture news, 45(1):693-708, 2017. URL: https://doi.org/10.1145/3093337.3037751.
  21. Robert O'Callahan, Chris Jones, Nathan Froyd, Kyle Huey, Albert Noll, and Nimrod Partush. Engineering record and replay for deployability. In 2017 USENIX Annual Technical Conference (USENIX ATC 17), pages 377-389, 2017. URL: https://www.usenix.org/conference/atc17/technical-sessions/presentation/ocallahan.
  22. Luís Pina, Anastasios Andronidis, and Cristian Cadar. FreeDA: Deploying incompatible stock dynamic analyses in production via multi-version execution. In Proceedings of the 15th ACM International Conference on Computing Frontiers, pages 1-10, New York, NY, USA, 2018. ACM. URL: https://doi.org/10.1145/3203217.3203237.
  23. Luís Pina, Anastasios Andronidis, Michael Hicks, and Cristian Cadar. MVEDSUA: Higher Availability Dynamic Software Updates via Multi-Version Execution. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS '19, pages 573-585, New York, NY, USA, 2019. Association for Computing Machinery. URL: https://doi.org/10.1145/3297858.3304063.
  24. Luís Pina, Luís Veiga, and Michael Hicks. Rubah: DSU for java on a stock JVM. In Proceedings of the ACM 2014 International Conference on Object-Oriented Programming Languages, Systems, and Applications, OOPSLA '14. ACM, October 2014. URL: https://doi.org/10.1145/2660193.2660220.
  25. Paul Rubin, David MacKenzie, and Stuart Kemp. Dd. https://man7.org/linux/man-pages/man1/dd.1.html. Google Scholar
  26. Ugnius Rumsevicius, Siddhanth Venkateshwaran, Ellen Kidane, and Luís Pina. Sinatra: Stateful Instantaneous Updates for Commercial Browsers Through Multi-Version eXecution. In 37th European Conference on Object-Oriented Programming (ECOOP 2023). Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2023. URL: https://doi.org/10.4230/LIPIcs.ECOOP.2023.26.
  27. David Schwartz, Ankith Kowshik, and Luís Pina. Jmvx: Fast Multi-threaded Multi-Version eXecution and Record-Replay for Managed Languages. In Proceedings of the 21st Annual ACM SIGPLAN Conference on Object-oriented Programming Systems, Languages, and Applications, OOPSLA 2024, Pasadena, California, United States, October 2024. Association for Computing Machinery. Google Scholar
  28. David Schwartz and Luís Pina. Jmvx github repository. https://github.com/bitslab/jmvx. Accessed: 2025-07-15.
  29. David Schwartz and Luís Pina. JMVX Github Repository - RTO implementation branch. https://github.com/bitslab/jmvx/tree/rto. Google Scholar
  30. David Schwartz and Luís Pina. Artifact for Optimizing Record/Replay through Relaxed Total Ordering and Multi-Version eXecution. https://zenodo.org/records/19643216, April 2026. URL: https://doi.org/10.5281/zenodo.19643216.
  31. David Schwartz and Luís Pina. Artifact for Jmvx: Fast Multi-threaded Multi-Version eXecution and Record-Replay for Managed Languages. https://doi.org/10.5281/zenodo.12637140, July 2024.
  32. Konstantin Serebryany and Timur Iskhodzhanov. Threadsanitizer: data race detection in practice. In Proceedings of the Workshop on Binary Instrumentation and Applications, WBIA '09, pages 62-71, New York, NY, USA, 2009. Association for Computing Machinery. URL: https://doi.org/10.1145/1791194.1791203.
  33. Nalini Vasudevan, Kedar S. Namjoshi, and Stephen A. Edwards. Simple and fast biased locks. In Proceedings of the 19th International Conference on Parallel Architectures and Compilation Techniques, PACT '10, pages 65-74, New York, NY, USA, 2010. Association for Computing Machinery. URL: https://doi.org/10.1145/1854273.1854287.
  34. Kaushik Veeraraghavan, Dongyoon Lee, Benjamin Wester, Jessica Ouyang, Peter M. Chen, Jason Flinn, and Satish Narayanasamy. DoublePlay: Parallelizing sequential logging and replay. ACM transactions on computer systems, 30(1):1-24, 2012. URL: https://doi.org/10.1145/2110356.2110359.
  35. Stijn Volckaert, Bart Coppens, Bjorn De Sutter, Koen De Bosschere, Per Larsen, and Michael Franz. Taming Parallelism in a Multi-Variant Execution Environment. In Proceedings of the Twelfth European Conference on Computer Systems, EuroSys '17, pages 270-285, New York, NY, USA, 2017. Association for Computing Machinery. URL: https://doi.org/10.1145/3064176.3064178.
  36. Stijn Volckaert, Bart Coppens, Alexios Voulimeneas, Andrei Homescu, Per Larsen, Bjorn De Sutter, and Michael Franz. Secure and Efficient Application Monitoring and Replication. In 2016 USENIX Annual Technical Conference (USENIX ATC 16), pages 167-179, Denver, CO, June 2016. USENIX Association. URL: https://www.usenix.org/conference/atc16/technical-sessions/presentation/volckaert.
  37. Stijn Volckaert, Bjorn De Sutter, Tim De Baets, and Koen De Bosschere. Ghumvee: Efficient, effective, and flexible replication. In Joaquin Garcia-Alfaro, Frédéric Cuppens, Nora Cuppens-Boulahia, Ali Miri, and Nadia Tawbi, editors, Foundations and Practice of Security, pages 261-277, Berlin, Heidelberg, 2013. Springer Berlin Heidelberg. Google Scholar
  38. Meng Xu, Kangjie Lu, Taesoo Kim, and Wenke Lee. Bunshin: Compositing Security Mechanisms through Diversification. In 2017 USENIX Annual Technical Conference (USENIX ATC 17), pages 271-283, 2017. URL: https://www.usenix.org/conference/atc17/technical-sessions/presentation/xu-meng.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact