Bridging Language Barriers: A Comparative Review and Empirical Evaluation of Source-To-Source Transpilers

The parsing phase can be split into lexical analysis (tokenisation) of source text into identifiers, literals and symbols, and syntactic analysis with grammar validation and parse-tree construction [5]. Top-down parsers (like the descent) are straightforward to extend but cannot handle left recursion; bottom-up parsers (LR) support more complex grammars at the expense of implementation effort [8]. Generalised LR (GLR) parsing extends this flexibility, resolving ambiguities and accommodating context-sensitive constructs [15].

A parse tree is distilled into an AST, which removes redundant syntactic details and emphasizes the programme’s semantic structure. Key characteristics include a hierarchical node representation, elimination of syntactic sugar, and explicit preservation of control and data flow dependencies which are essential for language-agnostic transformations [13, 10, 9, 23].

Code transformation represents the core computational process of transpilation, where the AST is systematically modified to generate equivalent code in the target language or version [27]. This phase involves multiple strategies to ensure semantic preservation while adapting to the target language’s specific syntactic and structural requirements [2].

The transformation process typically encompasses several key strategies:

• $\mathrm{■}$

  Structural transformation of language constructs;

• $\mathrm{■}$

  Semantic mapping between different language features;

• $\mathrm{■}$

  Handling of language-specific idioms and patterns;

• $\mathrm{■}$

  Generating optimized and compatible target code.

Complex transformations may involve multiple traversals over the AST, each one addressing different aspects of code translation [27]. These can include:

• $\mathrm{■}$

  Feature adaptation (e.g., translating modern JavaScript features to ES5);

• $\mathrm{■}$

  Paradigm translation (converting functional programming constructs to imperative styles);

• $\mathrm{■}$

  Semantic equivalence preservation;

• $\mathrm{■}$

  Performance optimization.

Reading the available literature, we found that the landscape of transpilation tools can be classified according to the type of translation they perform. They usually are categorised in three classes, as will be described below.

The empirical evaluation uses a uniform test suite in which each transpiler is asked to translate a short programme that (a) computes the sum of the even numbers in a given list and (b) finds the maximum value among the list elements. This logic was expressed in the various input languages supported by the tools under test: JavaScript/TypeScript, Nim, Clojure, Haxe, C and Java. The six code snippets written for each test can be found in Appendix A. By holding the computational intent constant, we ensure that performance and correctness metrics reflect the capabilities of each transpiler rather than variations in algorithmic complexity.

The compilers were selected primarily on the basis of the popularity in GitHub, measured by star count, to capture tools with significant community adoption and support. Additionally, the Java2Python converter was included despite its lower visibility, as it fulfilled the specific requirement of translating Java code to Python. During setup, several candidates could not be installed or executed on our Linux environment due to outdated dependencies or compatibility issues; the final benchmark comprises only the subset of tools successfully integrated into the testbed.

All experiments were conducted on a Linux workstation equipped with a multi-core (8 cores) CPU and 32 GB of RAM. Transpilers were installed via their standard package managers or repositories (e.g. npm for JavaScript-based tools, cargo for Rust-based tools). Where necessary, minor adjustments (such as version pinning or patching build scripts) were applied to resolve compatibility issues. Despite these efforts, some tools remained unusable and were excluded from the study.

Each transpiler was invoked with default settings except where command-line options were required to specify input and output languages. Execution time, CPU utilization and peak memory usage were recorded using system profiling utilities, like the time tool integrated in Linux. Output code was then compiled or interpreted to verify correctness, and a precision score from 0 to 100 was assigned based on successful execution and fidelity to the original specification.

Execution times varied markedly across tools. Nim – transpiling to C, C++ and JavaScript – achieved average runtimes of 0.61 s, 0.59 s and 0.12 s respectively, with moderate memory use (77.8 MB, 79.4 MB, 24.5 MB). In contrast, ClojureScript exhibited the slowest performance at 27.44 s despite low memory consumption (36.0 MB), rendering it unsuitable for time-sensitive workflows. The jscodeshift transformer delivered the fastest translation (0.06 s) and minimal memory footprint (1.76 MB), although this speed came at the expense of lower precision (see Section 4.2).

Because the experiment was carried out in multiple tools involving multiple programming languages, it was not possible to use a single application that would evaluate all codes using the same parameters.

So in order to quantify how faithfully each transpiler preserved the semantics of the original programs, we applied a composite scoring methodology on a 0–100 scale. We developed custom evaluation scripts in Python, JavaScript, Nim, and other host languages, one per transpiled output, to execute identical input sets and compare results against reference implementations. Although the code under test spanned multiple languages, all source files implement the same algorithmic logic, ensuring a uniform basis for comparison.

The overall precision score $S\in [0,100]$ for each tool is computed as a weighted sum of five orthogonal categories:

Each sub-score is normalized to the range $[0,100]$ before weighting:

• $\mathrm{■}$

  Functional Correctness (40%): Does the transpiled program compile/run and produce the correct outputs for all test inputs? This is the highest-weight category, since a single failure yields a 0 in this dimension.

• $\mathrm{■}$

  Semantic Equivalence (25%): Do intermediate states, control flow structure, and side effects match the behaviour of the original program with equivalent inputs?

• $\mathrm{■}$

  Code Quality (15%): Does the output follow best practices and idiomatic style for the target language (e.g., Pythonic constructs, proper naming, concise expressions)?

• $\mathrm{■}$

  Structural Similarity (10%): Is the high-level structure (functions, classes, loops) aligned with the source?

• $\mathrm{■}$

  Error Handling (10%): Are edge cases and exceptions handled appropriately (e.g., null checks, boundary conditions)?

A perfect score (100/100) indicates that the transpiler produced code that compiles/runs without modification, maintains equivalent logic and control flow, adheres to target-language idioms, preserves structural patterns, and robustly handles errors across all test inputs.

Memory usage showed significant divergence. TypeScriptToLua consumed the most RAM (302.3 MB), followed by the TypeScript Compiler (180.2 MB) and c2rust (135.3 MB). Conversely, jscodeshift (1.76 MB) and Fennel (7.8 MB) demonstrated exceptional frugality, making them attractive for resource-constrained environments. CPU utilisation generally remained below 100% of a single core; jscodeshift peaked at 235%, reflecting its multi-threaded execution across several cores.

Developer experience was assessed qualitatively. Babel and jscodeshift stood out for ease of use, due to intuitive command-line interfaces and extensive plugin ecosystems. The TypeScript Compiler and Babel also showed high stability with minimal runtime errors. Nim’s tooling proved reliable but is under active development, which may introduce future breaking changes. Although Java2Python delivered perfect accuracy and stability, its support for only Python 2 renders it impractical for modern codebases.

Based on these findings, the research reach the conclusion that:

• $\mathrm{■}$

  For JavaScript/TypeScript projects: use Babel or the TypeScript Compiler for a balance of precision, stability and moderate resource demands.

• $\mathrm{■}$

  Where raw performance is critical: employ Nim for its rapid and accurate transpilation to C, C++ or JavaScript.

• $\mathrm{■}$

  For quick, lightweight transformations: opt for jscodeshift, acknowledging potential trade-offs in precision.

• $\mathrm{■}$

  In specialised migrations (e.g. C→Rust, Fennel→Lua): consider c2rust or Fennel with the understanding that additional verification may be required.

Building on both theoretical and practical insights, several avenues for further research were identified:

If this article interests you, it is important to mention that there is a complementary website where you can find the article you read, the transpilers (and their version) used along with the links so you can easily access the same tools used in the experiment. After that, the webpage displays all the scripts used to evaluate the transpiled code, so that you can replicate the same experiment and further extend it. Right at the end there is a link to another document that dives into the state-of-the-art study about transpilers.

Here is the link:
https://justandre02.github.io/Comparative-Review-and-Empirical-Evaluation-about-transpilers/