Search Results

Documents authored by Tian, Cong

Document
Technical Track Paper
DOI: 10.4230/LIPIcs.ESEM.2026.25
An Empirical Study of Problem-Aligned Multilingual Code Generation and Directed Code Translation by Large Language Models

Authors: Quanhe Wang, Cheng Wen, Dugang Liu, Xingjian Han, Bin Yu, Ping Chen, Shengchao Qin, and Cong Tian

Published in: LIPIcs, Volume 394, 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)

Abstract
Background. Large language models (LLMs) can generate executable code, but their reliability across programming languages remains difficult to characterize. Aims. We investigate multilingual code capability under problem alignment, focusing on functional reliability, cross-language consistency, translation behavior, problem and sampling effects, and execution quality. Method. We present MciBench, covering 1,489 programming problems and eight languages, of which 1,466 have execution-validated coverage in all eight languages. We evaluate representative LLMs through multilingual generation and analyze source-code-conditioned translation on 100 randomly selected all-language-covered problems. Results. Performance varies substantially across models and target languages, while source-code-conditioned translation also exhibits marked variation across source-target settings. Problem difficulty and candidate budget materially affect correctness, strong aggregate performance does not necessarily imply cross-language consistency, and accepted programs differ in runtime and memory efficiency. Conclusions. Aggregate pass@k alone is insufficient for characterizing multilingual code capability, motivating evaluation that considers language consistency, problem sensitivity, sampling behavior, translation settings, and post-correctness execution quality.
Cite as

Quanhe Wang, Cheng Wen, Dugang Liu, Xingjian Han, Bin Yu, Ping Chen, Shengchao Qin, and Cong Tian. An Empirical Study of Problem-Aligned Multilingual Code Generation and Directed Code Translation by Large Language Models. In 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 394, pp. 25:1-25:21, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026)

Copy BibTex To Clipboard

@InProceedings{wang_et_al:LIPIcs.ESEM.2026.25,
  author =	{Wang, Quanhe and Wen, Cheng and Liu, Dugang and Han, Xingjian and Yu, Bin and Chen, Ping and Qin, Shengchao and Tian, Cong},
  title =	{{An Empirical Study of Problem-Aligned Multilingual Code Generation and Directed Code Translation by Large Language Models}},
  booktitle =	{20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)},
  pages =	{25:1--25:21},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-450-5},
  ISSN =	{1868-8969},
  year =	{2026},
  volume =	{394},
  editor =	{Feldt, Robert and Paasivaara, Maria and Mendez, Daniel and Wagner, Stefan and Bar\'{o}n, Marvin Mu\~{n}oz},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.ESEM.2026.25},
  URN =		{urn:nbn:de:0030-drops-279936},
  doi =		{10.4230/LIPIcs.ESEM.2026.25},
  annote =	{Keywords: Large language models, code generation, code translation, multilingual programming, empirical software engineering}
}

Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact