A Pragmatic Approach to Replay Compilation

Reproducibility of compilation results from a dynamic compiler is often desired but difficult to achieve. For example, when a JIT compilation crashes, a compiler developer may need to diagnose the issue by reproducing the crashing compilation with additional diagnostic options. However, the process may require re-executing the application under the same load on the same platform, which is often impossible. Moreover, the crash could occur only in specific conditions that may not be satisfied in every execution of the virtual machine (VM). Reproducibility may also be required in other scenarios that require a fixed input to the compiler, for example, to measure compilation metrics such as compilation speed accurately.

It is often difficult to reproduce compilation results because they depend on multiple inputs from the VM that vary in time and between VM executions [12]. These inputs include, but are not limited to, the collected profiles and other runtime feedback, the set and content of the loaded classes, the VM configuration and memory layout, the target platform, the contents of compilation-final fields, and the resolution and initialization status of classes, methods, and fields. The input data differs due to the non-deterministic behavior of the VM, the application and the libraries, dynamic class loading, runtime bytecode modification, dynamically generated classes, and implementation details of the VM’s data structures. To replay a compilation and obtain an equivalent result, one needs to ensure that the inputs to the compiler are equivalent and the compiler itself is deterministic. Note that the goal of an accurate replay differs from that of code reuse [12] – the machine code originating from an accurately replayed compilation of an earlier VM instance is typically incompatible with other VM instances and impossible to execute.

Replay compilation is a well-known term in the community. Nonetheless, contemporary VMs lack support for dependable and accurate replay. Jikes RVM [7] could record information such as the optimization level of particular methods and profiling information. The VM can reuse these records in subsequent executions, which can be leveraged as part of a performance evaluation methodology [3, 1, 5, 15]. Similarly, the HotSpot JVM [13] enables replay compilation by recording selected runtime information [4]. However, the replay in HotSpot can fail and does not capture all inputs to the compiler, leading to varying compilation results. The IBM J9 VM (now known as Eclipse OpenJ9) [6] could systematically record all inputs to the compiler while achieving low overhead [9] by reusing information from core dumps. In OpenJ9, the approach was abandoned, citing issues with portability and fragility [8].

Our approach takes inspiration from the one in IBM J9 [9], but we prioritize maintainability and portability rather than recording overhead. We systematically record and replay all communication between the compiler and the VM, operating within the scope of a high-level object-oriented interface. The GraalVM compiler queries the VM using the JVM Compiler Interface (JVMCI) [14], which is an object-oriented interface implemented in Java that conveys information about VM objects, such as the resolved classes, methods, fields, constants, and other information. Every one of these VM objects is exposed to the compiler in the form of a Java object instance that implements a specific Java interface.

To record a compilation, we create proxies for JVMCI objects whose methods return information from the VM. The created proxies handle method invocation by invoking the intended receiver method and serializing both the arguments and the return value. To replay a compilation, we use proxy classes that handle method invocation by looking up the appropriate return value using the arguments as a key.

We serialize the recorded operations for every compilation unit in a single JSON file. This allows us to invoke the dynamic compiler programmatically to replay the recorded compilation without running the original application or requiring access to its bytecode.

For example, Listing 1 is a snippet of the recorded file from a compilation of the Object.<init> method. The file indicates that the root compiled method is represented by an instance of JVMCI’s method class, and the instance ID is 0. The file then dictates the return values of method invocations that use the instance with ID 0 as a receiver. The recorded operations indicate that the getCode() method should return a single-element byte array (containing the return bytecode). Additionally, the declaring class of the method is an instance of JVMCI’s type class with ID 1, whose getName() method returns "Ljava/lang/Object;". The file captures all information required to complete the compilation. In addition, it includes the serialized compilation result (low-level intermediate representation), allowing us to verify the equivalence of the replayed compilation to the recorded compilation.

Our prototype implementation aims for minimal overhead when recording is disabled and minimal maintenance overhead to ensure that the feature can be integrated into the upstream GraalVM compiler. The prototype requires relatively minor modifications in 51 source files in the compiler and the HotSpot JVM, totaling less than 700 inserted lines and 300 deleted lines. On top of this, we added about 8600 new lines in 45 files. The semi-automatically generated proxy classes comprise about 38% of the added lines. We executed standard pre-merge tests and benchmarks (with recording disabled), and we did not detect any significant regressions in the compilation speed, memory allocated by the compiler, or the compiled code size. Our changes increased the binary size of the compiler¹¹1The GraalVM compiler is compiled ahead-of-time to a native shared library using Native Image [10]. from about $50\mathrm{MB}$ to $52\mathrm{MB}$, which is a consequence of introducing the proxy classes. In summary, however, we did not encounter substantial barriers preventing integration in the GraalVM compiler.

Conversely, a preliminary evaluation indicates a roughly 10x slowdown in compilation speed for the recorded compilation units. We consider the overhead acceptable for the intended use cases, and we have not attempted to optimize it yet because such optimizations could conflict with keeping the code maintainable or increase the overhead when recording is disabled. We can mitigate the effect on the running application by recording only compilations of a few selected methods. For example, when an unrecorded JIT compilation crashes, the compiler can immediately repeat the same compilation with recording enabled. This is a pragmatic strategy used by the GraalVM compiler, although it does not guarantee that repeated compilation will experience the issue that caused the crash. We can also enable recording only for the hottest methods, which may be appropriate for other use cases.

While our primary use case is debugging and diagnostics, we conclude by sketching other potential applications unlocked by replay compilation. Extending previous work [11], we can replay a compilation using two distinct compiler revisions. This could enable comparing how optimization decisions, code quality, and compilation speed change between two compiler revisions without any noise caused by the VM’s non-determinism. A challenge with this and similar use cases is that the replay file may not contain all the required information when the replayed compilation diverges. We also plan to apply replay compilation to reduce the costs of tracking the compiler’s performance [2] by running benchmarks only if the compilation results change.