Wastrumentation: Portable WebAssembly Dynamic Analysis with Support for Intercession

Dynamic analyses written on Wastrumentation can not only observe all the runtime behaviour of a target application but also control it. More concretely, analyses can change part of the target program behavior during the execution of the trap function. To illustrate this, consider the trap function const_ that intercepts pushing a constant on the Wasm value stack. This trap function receives two arguments: the constant value v that would be pushed and the source code location l of the instruction. Wastrumentation allows the analysis to decide what value is pushed on the stack when the control flow returns to the target program by using the return value of the trap function. For example, the “DeNaN” analysis (cf. Section 2.1) uses this to alter the constant value v to a deterministic NaN value.

Generally, the API of Wastrumentation uses the return value of trap functions to alter the behavior of the target instruction. Whereas this is not always applicable (due to reasons further explained in Section 2.4), Table 1 shows when a return value dictates (part of) the target program behavior in the column “Return Type”. Below, we further outline how the analysis is in control once a trap function is invoked.

• $\mathrm{■}$

  Intercession of control flow: For the trap functions if_pre, if-then-else_pre select, and br-if, the return value dictates control flow of the branch expression as a boolean. For the trap function br-table the return value dictates the target label that control-flow will jump to. The return value of the trap function call-indirect_pre determines the index in the function table of the instruction that will be invoked.

• $\mathrm{■}$

  Intercession of value stack manipulation: The following trap functions intercept instructions that push on the value stack: unary, binary, const, local, global, load, store, memory-size and memory-grow. The return value of the corresponding trap function is used as the outcome value that will be pushed on the stack.

• $\mathrm{■}$

  Introspection only: Aside for general intercession, the following trap functions cannot influence their respective instruction: br, call_pre, call_post, call-indirect_post, block_pre, loop_pre, if_post, if-then-else_post, block_post, loop_post, drop and return. This is due to how the respective program events do not depend on runtime information (eg. br) or report the termination of a program event (eg. if-then-else_post).

• $\mathrm{■}$

  Intercession of function application: The trap function apply can control a Wasm function application. When a function application takes place, this trap receives two pointers. The former points to an array containing the function arguments, and the latter points to the array containing the function results. This trap can alter the passed arguments before the function application takes place, and it can alter the return values before the control flow returns to the caller. Furthermore, this trap function is in control of whether the function application takes place at all.

Even if an analysis does not need to modify the target program, it remains responsible for the execution of intercepted program events. For example, if the trap function const_ must remain transparent then the unaltered argument v must be the return value of the trap function. For the traps involved, the analysis is thus responsible for ensuring that the target operation takes place within the execution of the trap function. As mentioned before, Table 1 shows within the column “Transparent Base” the functions to be called within the trap function to forward the intercepted operation back to the target program.

Actually, the functions in the column “Transparent Base” represent the continuation of the intercepted operation that yields a return value. This design enables the analysis developer to add analysis code around the continuation of the intercepted operation and decide either to call the continuation or skip it altogether. This is most notable for the program events “apply” and “memory-grow”. For “apply” the analysis controls function applications. This may further imply calling recursive functions. For “memory-grow”, the return value either points to the next allocated page in linear memory or $-1$ if no allocation could be performed. This enables the analysis to artificially limit the available memory by not calling the memory to growth continuation and early returning with $-1$. For all other operations, the transparent base continuation is side-effect free and has no significant impact on analysis code that may go before or after the continuation.

The API of the “apply” trap receives a pointer to the continuation function and a pointer to a buffer containing the runtime arguments and results. The continuation function represents the next step in the execution flow. For the apply hook, this is the original function being intercepted. As Section 2.2.2 explains, the “apply” trap is responsible for calling the continuation function to allow a transparent execution of the target program. Inspired by the CLOS auxiliary “around:” method definition [3], the analysis code can (a) manipulate the arguments before the continuation, (b) determine when or if at all the function application takes place and (c) manipulate the results after the function application.

Listing 2 shows the minimal transparent analysis as a Wasm program that instruments function applications. The only implementation is the trap definition $$apply$ (line 2) that receives five arguments. The first argument ($f_{base}$) is a pointer to the continuation function. The second and third arguments ($bu{f}_{sig}$, $bu{f}_{sigtype}$) are pointers to buffers that contain the arguments and results for the current intercepted function application. The last two arguments ($c_{arg}$, $c_{res}$) dictate the number of arguments and results for that call respectively (to bound the value access of $bu{f}_{sig}$, $bu{f}_{sigtype}$). Line 4 calls the continuation, which may be preceded or succeeded with additional analysis code that is left out of the listing for brevity. Since the implementation of $$appl{y}_{base}$ resides in the transformed target program, the analysis must declare it as a function import (line 6).

Listing 3 shows the input program for the transformation, containing a single function definition $$f$. Whereas the transformation generalizes to any input set of function definitions, we restrict the example transformation to that of $$f$ for brevity. Some intentional design considerations that influenced the transformation approach are the following:

• $\mathrm{■}$

  The analysis must be notified of all function applications of a target function, including (a) internal function applications through call or call_indirect statements local to the target module, (b) external function applications that may stem from the host and (c) for target functions that have no implementation details specification but are defined through an import statement.

• $\mathrm{■}$

  The API of the $$apply$ trap is polymorphic for the target function application signature. This design choice enables low coupling between the analysis and the target input program, as the analysis program needs not to monomorphize the $$apply$ trap implementation per signature as shown in [19]. Consequently, the analysis implementation must dispatch on function-specific behavior at runtime which may incur a performance penalty.

Listing 4 shows the input program result after transformation. The transformation of target function $$f$ from Listing 3 results in two definitions $$f_{original}$ (line 2) and $$f$ (line 3). It is transparent to client code of $$f$ (both internal and external to the Wasm module) since after transformation $$f$ preserves its identity³³3 Identifiers are defined as alphanumeric tokens prefixed with a $-sign. In practice, Wasm functions are defined by an index as an i32-value, so a function name is actually a number that dictates its index. , its signature and its export declaration. However, $$f$ will now call the trap $$apply$ defined in Listing 2 with the required information.

In preparation for the call to $$apply$, $$f$ first stores arguments and runtime types to buffers (lines 5–9) that $$apply$ may manipulate at will. Next, $$f$ yields control flow to $$apply$ (line 11), after which the results are loaded from the buffers (lines 12–14) and eventually $$f$ returns. The arguments to $$apply$ (line 11) are $0$, the function pointer to the continuation function index in the table $$tabl{e}_{base-functions}$, the pointers $$bu{f}_{sig}$ and $$bu{f}_{sigvalues}$ pointing to the value- and types-buffers respectively and the the number of arguments $m$ and results $n$.

If the trap $$apply$ in Listing 2 calls the continuation $$appl{y}_{base}$, in Listing 4 the applicable continuation is resolved at runtime with the function pointer argument $$f_{base}$ that is looked up in the function table $$tabl{e}_{base-functions}$ (line 30). The call to the resolved applicable continuation $$f_{base}$ loads the arguments from the buffer $$bu{f}_{sig}$ (line 17) and calls $$f_{original}$ (line 19), after which the results are stored to $$bu{f}_{sig}$ (line 21) and control is yielded to the post-analysis code in the trap $$apply$.

The buffers used to communicate the runtime arguments and results between the function application, the analysis trap and the continuation function are handled by an external “instrumentation” module. This “instrumentation” module can orchestrate its linear memory while leaving the linear memory of the input program and the analysis code untouched. Functions to allocate and free memory, to load and store values in the buffers are imported in the instrumented module (lines 23–28) and can be imported by the input analysis, though the pre- and post-analysis code is left out.

All other instructions follow more concise rewrite strategies by Wastrumentation. These rewrite strategies rewrite each target operation into instructions that prepare context information on the value stack, followed by a call to an associated analysis trap function.

Table 2 summarizes the rewrite strategy, grouping instructions with similar rewrite patterns. The “Original instruction” column lists the unaltered instructions, while the “Rewritten instruction” column shows the instrumented result. The trap function is denoted as “$\$\mathrm{𝗍𝗋𝖺𝗉}$” for brevity but is specific to each instruction, such as “$\$\mathrm{𝗋𝖾𝗍𝗎𝗋𝗇}$” for “return” or “$\$\mathrm{𝖻𝗋}$” for “br”.

As shown in Table 1, each trap function also receives the instruction’s location. To provide this information, two constants (function index and instruction offset) are pushed on the stack before each trap call. These details are omitted from Table 2 for simplicity.

Maintaining the transformed program’s well-typed property is essential. Thus the instrumented hooks must preserve the type correctness, including their interaction with the VM stack. Therefore, the analysis may modify the values handled by the target program, but it cannot alter their types. For instance, the rewrite of $t.\mathrm{𝖼𝗈𝗇𝗌𝗍}c$ involves pushing the constant onto the value stack, followed by a call to the trap function “$\$\mathrm{𝖼𝗈𝗇𝗌𝗍}$”. This trap function receives the constant, returning a value of the same type while controlling its final value. Similarly, instructions like $t.\mathrm{𝑢𝑛𝑜𝑝}$ and $t.\mathrm{𝑏𝑖𝑛𝑜𝑝}$ are replaced with trap function calls that take the instruction operands and compute the resulting value on behalf of the program. The operands are available on the stack as they must have been for the original instruction.

For control-flow-related instructions such as “br-if”, “br-table”, and “select”, the rewrite introduces a preceding trap function that controls the dynamic control flow. For “br-if” and “select,” this is an i32 value representing the condition. For “br-table,” it is an i32 value selecting a branch label from the statically predefined table associated with the instruction.

Instructions like “if-then-else”, “if-then”, “call-indirect”, “call”, “block”, and “loop”, include both pre- and post-trap functions. This design allows the analysis to trace the flow of execution per instruction, as additional instrumented code can be executed in between. For “if-then-else” and “if-then,” the pre-trap function may modify the boolean condition controlling the flow. In contrast, trap functions for “return” and “drop” are observational, i.e. unable to alter control flow. For instructions like “local-get,” “global-get,” “local-set,” “local-tee,” “global-set,” and “global-tee,” the trap functions can alter the values being read or written to the respective variables.

We now discuss the details of the memoization analysis. Listing 6 shows its implementation in Wastrumentation. The analysis maintains a global hashmap as cache where it associates a function index and its arguments with the results for that function call (lines 4–6). The analysis targets the apply trap since only function applications will be memoized (line 23). The trap implementation computes the cache key (lines 25–26) and returns the associated result values if the key is stored in the cache. If the cache had no associated entry for the computed key, the function application continues and the computed result is stored in the cache for successive function applications (lines 29–31).

To assess the effectiveness of the memoization analysis, we conduct an experiment to check whether it would benefit the programs in the WasmR3 benchmark suite executed on Wasmtime. We first perform a static analysis phase to identify the programs with pure functions which could be good candidates for memoization. This phase identified 14 candidate programs (out of the 27) featuring pure functions. Figure 4 plots the measured runtimes of those programs and their instrumented variants. Our results show that the analysis can incur a performance penalty for some programs, and it can speed up others. The performance penalty ranges from 1.02x slowdown (rguilayout) to 3.10x slowdown (funky-kart), while the speed-up ranges from 1.02x speedup (rguistyler) to 1,1526.50x speedup (fib). This experiment shows that such analyses can be developed at low cost for exploration, and yield the potential to then be ported within concrete VM implementations if they offer promising results.

As mentioned before, state-of-the-art dynamic analysis platforms for Wasm do not support intercession, but bytecode rewriting frameworks such as Binaryen [10] and Walrus [32] do. We observe that Binaryen v122 offers six instrumentation passes: “--safe-heap”, “--denan”, “--instrument-locals”, “--instrument-memory”, “--log-execution”, “--trace-calls”. Out of the six, all offer support for intercession except for “--log-execution” . Unfortunately, the “--denan” analysis is the only instrumentation pass that rewrites the input program into a variant that can execute standalone. All other instrumentation passes rewrite the target program such the host must implement a part of the analysis, eg. what to perform when observing reads or writes into locals for the “--instrument-locals” pass. This couples the analysis with host-provided functionality. Titzer et al. developed in [38] two standalone ad-hoc analyses, “branches” and “opcodes”, in the Walrus bytecode rewriting framework. To further evaluate RQ1, we compare the implementation of these three ad-hoc analysis implementations (“DeNaN”, “branches”, and “opcodes”) that can execute standalone with an equivalent analysis on Wastrumentation.

When comparing the implementation of a given analysis on Wastrumentation to an ad-hoc solution, we observe three main limitations of bytecode rewriting frameworks.

Higher implementation effort.

We observe repeated efforts of constructing the low-level Wasm constructs and injecting these within the target program when using bytecode rewriting frameworks. This most notably affects the code size of the analyses: “DeNaN” (195 LoC on Binaryen, 35 on Wastrumentation), “branches” (291 LoC on Walrus, 24 on Wastrumentation), and “opcodes” (242 LoC on Walrus, 81 on Wastrumentation). The lines of code for analyses in Wastrumentation are smaller since the transformation efforts are reused among analyses as they happen within Wastrumentation and Wasm-Merge.

Semantic abstraction loss.

We observe a notable loss of high-level semantic expressiveness in bytecode rewriting systems because analysis implementations are restricted to low-level, elementary operations due to the lack of access to rich abstractions, such as complex data structures and a strong type system. For example, “DeNaN” analysis is limited to changing a value to 0 while the “branches” and “opcodes” increment a fixed set of global counters by 1. The other analyses implemented by Binaryen ( “--safe-heap”, “--instrument-locals”, “--instrument-memory”, “--log-execution”, “--trace-calls” ) avoid additional complexity by forcing the analysis developer to implement more complex data structures within the host, such as logging infrastructure, hashmap dictionaries, or debugging infrastructure. In contrast, implementing the analyses on a general-purpose analysis platform is easier since developers do not need to manipulate Wasm instructions nor ensure the type safety of the analysis code. In Wastrumentation, developers can implement the analysis using a high-level API and in a high-level language like Rust. This reduces the complexity of the analysis by raising the abstraction level, allowing for a compiler to type-check, validate, and optimize the analysis code.

Lack of Separation of Concerns.

We observe the analysis logic is tightly interwoven with the transformation pass, resulting in reduced modularity and maintainability. This means that instructions such as ’increment a counter’ (analysis code) are scattered among instructions such as ’extend target body with increment’ (transformation code). Wastrumentation decouples these concerns by treating the analysis implementation as a unit separate from the instrumentation phases. As a result, it becomes easier to maintain, comprehend, and evolve the semantics of the analysis independently from instrumentation.

Finally, we would like to note that the Binaryen instrumentation pass “DeNaN” is incomplete. The implementation of the “--denan” pass in Binaryen documents does not support the instrumentation of local.get, as doing so “would cause problems if we ran this pass more than once (the added functions use gets, and we don’t want to instrument them)” [11]. Since Wastrumentation decouples instrumentation from the analysis implementation, the instrumentation of local.get operations do not run the risk that analysis code and its instructions of local.get are instrumented too.

Figure 5 shows the ratio $s$ for each instrumentation in the cross-product. In cases where Wasabi failed to instrument a specific input program, we leave the cell blank as no ratio can be computed. Additionally, Figure 6 shows the size of the input programs prior to any instrumentation. The code size increases for each platform individually are presented in Appendix B in [25].

Out of a total of 180 successful ratios, 140 indicate the output Wasm module code generated by Wastrumentation is smaller, while the remaining 40 favor Wasabi. The analyses “Basic Block Profiling”, “Branch Coverage” and “Call Graph” tend to favor Wasabi, which aligns with the low number of target operations in these analyses (6, 5 and 1, respectively). For the other analyses, which involve a larger number of traps, the size ratio generally favor Wastrumentation, resulting in smaller binary sizes compared to Wasabi.

The noticeable “red stripe” for the input program “game-of-life” indicates a significant advantage for Wasabi’s instrumentation. This is largely due to the significant difference in binary sizes among the input programs. Whereas “game-of-life” has an uninstrumented size of just 1.199 bytes, the other input programs range from 38.965–22.264.896 bytes. This suggests that Wastrumentation introduces a relatively larger size overhead for smaller programs compared to Wasabi, which may be expected as this metric counts for Wastrumentation the additional analysis code, whereas it does not count the JavaScript analysis code for Wasabi.

In this section, we evaluate the cost of loading, validating and executing programs for each stage for the input programs of the WasmR3 input suite when instrumenting with the “Forward” analysis. Figure 7 plots the overhead for each stage for 20 instrumented input programs with Wastrumentation. The overhead is computed using the Wizard engine. The median is computed for both the uninstrumented and instrumented variant over 30 runs. For the remaining input programs, 5 timed out, and 2 could not execute.

We observe that the relative increase for loading and validating an instrumented variant correlates with the absolute code size increase from Section 4.4. This is because both phases require reading the input program into the execution engine and performing a static validation on it. For loading, this is in the range 1.13–5.37, for validating in the range 1.84–5.65.

Second, we observe that the increase in execution time spent for the input programs differs significantly depending on the input program. We further ported 10 analyses other than “Forward” to Wastrumentation for further comparison, however other analyses execution failed with the message “!trap[MEM_OUT_OF_BOUNDS]”. We believe this error does not originate for the “Forward” analysis since it does not reserve additional linear memory in that analysis code. To the best of our knowledge, this may point out a regression for the Wizard baseline engine, since no such exceptions occurred when executing the same experiments on other execution engines such as NodeJS, Wasmtime, and Wasmer.

To determine the runtime overhead of Wastrumentation, we compute the ratio $o$ as the performance of instrumented execution relative to uninstrumented execution. An $o$ value of 2 means instrumentation slows the execution time down by a factor of 2.

To compare the performance overhead for Wastrumentation and Wasabi, we computed a ratio $p$ as the execution time of a Wasabi-instrumented module relative to the execution time of a Wastrumentation-instrumented module. A value $p>1$ indicates Wastrumentation-instrumented execution requires less time to complete, and a value of indicates Wasabi-instrumented execution requires less time to complete. For example, a $p$ value of 0.5 means that “Wasabi is twice as fast at runtime to analyse the program”, while a $p$ value of 2 means that “Wastrumentation is twice as fast at runtime to analyse the program”.

Similar to Section 4.4, we compute the ratio’s $o$ and $p$ for the programs in WasmR3 using the analyses supported by both platforms. We profile the execution of uninstrumented and instrumented programs using the performance observation API from NodeJS “perf_hooks”. The recorded times do not include the program loading from main memory or the verification or compilation of the module. As mentioned in Section 4.1, we calculate the median of 30 executions of the program within a single NodeJS instance. During our experiments, the execution of an instrumented program was limited to a timeout of 300 seconds.

Figure 8 shows the results for the overhead ratio $o$. The white cells indicate missing data, either due to the instrumented execution exceeding the timeout (T) or due to raising an uncaught exception (MD). In total, 51 combinations timed out, while 1 raised an uncaught exception (“memory-tracing” for the input program “pathfinding”).

The results show that the instrumentation overhead for Wastrumentation ranges from 1x–514.0x. The analyses “coverage-instruction” and “taint” generally incur a higher overhead of 5.5x–514.0x and 12.85x–376.9x, respectively. These analyses are, however, heavy-weight as they implement all traps and maintain heap-allocated data structures that grow during the target program execution (a coverage hashmap and growable vectors, respectively). Nevertheless, Wastrumentation’s overhead aligns with expectations for similar source code instrumentation platforms such as Wasabi [19], Jalangi [33] and Aran [6].

Figure 9 shows the results for $p$ ratio used to compare the runtime overhead for Wastrumentationand Wasabi. Note that figure only plots 20 input programs, as 7 input programs could not be instrumented by Wasabi so no ratio could be computed. The white cells indicate missing data, labeling T for “timeout”, followed by the platform “M” (Wastrumentation), “S” (Wasabi) or “M&S” (both). For a total of 180 combinations where both Wastrumentation and Wasabi yield a valid instrumented module, 140 combinations yield a measurement. For 31 combinations both Wastrumentation and Wasabi timeout. For Wastrumentation an additional 3 combinations timeout and for Wasabi an additional 6. The performance overhead individually per platform for additional reference are further detailed in Appendix C in [25].

The results of this experiment show that the distribution does not favor any particular instrumentation platform. Overall, the most suitable platform is determined by the specific input program, rather than the type of analysis applied. For example, for some input programs (“figma-startpage”–“hydro”) Wastrumentation incurs a notable lower overhead across all analyses, while for other input programs (“jqkungfu”, “rtexpacker”, “rtexviewer”) the opposite holds. Some notable outliers are [“jqkungfu”,“taint”] where Wasabi-instrumented execution is 17.5x faster and the “memory-tracing” analysis for “bullet”, “funky”-kart, “guiicons”, “ffxgen”, and “rguistyler” where Wastrumentation is >9.0x faster than Wasabi.

To coroborate that our findings are consistent across virtual machines, we also deployed the “Forward” analysis for Wastrumentation on Wasmtime. Appendix D in [25] plots our findings, and confirms that across NodeJS and Wasmtime the overhead for the “Forward” analysis remains within the same order of magnitude.

Lastly, we compare the performance overhead of Wastrumentation with that of analyses implemented in bytecode rewriting systems. To this end, we compute the overhead of the bytecode rewriting analyses that we ported to Wastrumentation as described in Section 4.2.2.

Figure 10 plots the performance overhead for various input programs of the WasmR3 benchmark suite. For all analyses and input programs, the runtime overhead of bytecode rewriting is one or two orders of magnitude lower. These results are confirmed by earlier experiments, as pointed out by Titzer et al. [38].

The performance advantage of bytecode rewriting systems stems from two aspects. First, bytecode rewriting systems allow developers to have more fine-grained control over which specific locations to inject instrumentation code statically. Second, they can directly inline the analysis within the target operations. In contrast, Wastrumentation, requires runtime dispatching on generic operations. For example, our “binary” trap encapsulates 76 different instructions requiring additional dispatch at runtime.

Figure 11 shows the results for each input program. For Wasabi, memory overhead could not be computed for the seven programs that failed to be instrumented, and these programs show no value in the figure. We observe that Wastrumentation’s memory overhead ranges from 1.01x to 37.65x, while the memory overhead for Wasabi ranges from 1.03x to 12.34x. We suspect that the higher memory overhead in Wasabi is due to Node.js JIT employing more memory, as the analysis is written in JavaScript, but further experimentation is needed to quantify this hypothesis.

In the context of Wasm programs, runtime memory is often a limited resource, particularly in environments such as cloud-based platforms or microcontrollers. As a result, minimizing memory overhead is critical for ensuring efficient execution. These results indicate that Wastrumentation may be better suited for deployment in memory-constrained environments.