Search Results

Documents authored by Lenarduzzi, Valentina

Document
Emerging Results, Vision & Reflection Track Paper
DOI: 10.4230/LIPIcs.ESEM.2026.59
On Coding Agent Issue Localization Accuracy - An Exploratory Study

Authors: Antonino Coppola, Matteo Esposito, Rick Kazman, and Valentina Lenarduzzi

Published in: LIPIcs, Volume 394, 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)

Abstract
Context. Generative AI coding agents are increasingly used to automate software maintenance and issue resolution. However, current evaluations mainly focus on test-passing behavior and provide limited insights into whether agents modify the same software entities selected by developers. Aim. This study investigates the accuracy of issue localization performed by agents across different software granularities. Method. We analyzed 2,441 issue-fixing commits from 10 large-scale Java projects and evaluated three agents, each combining the OpenCode harness with a different open-weight LLM. We compared agent-modified entities against human-implemented fixes at the package, class, and method levels using Accuracy, Precision, Recall, F1-score, and MCC. Results. Agents partially identified the software entities requiring modification, but localization performance strongly depended on software granularity. Agents achieved the strongest results at the package level, while performance progressively degraded at the class and method levels. GLM-5 consistently achieved the strongest localization performance, although practical differences among agents remained limited. Our findings showed a limitation of agents implementing functionally plausible yet structurally different packages, class or methods leading to consistently negative MCC values worsening with finer granularity. Conclusion. Current agents can often identify the general architectural region affected by an issue, but still struggle to precisely localize fine-grained implementation points. These findings highlight both the potential and the limitations of agents for issue localization and motivate larger-scale investigations on localization behavior, architectural impact, and long-term maintainability implications.
Cite as

Antonino Coppola, Matteo Esposito, Rick Kazman, and Valentina Lenarduzzi. On Coding Agent Issue Localization Accuracy - An Exploratory Study. In 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 394, pp. 59:1-59:13, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026)

Copy BibTex To Clipboard

@InProceedings{coppola_et_al:LIPIcs.ESEM.2026.59,
  author =	{Coppola, Antonino and Esposito, Matteo and Kazman, Rick and Lenarduzzi, Valentina},
  title =	{{On Coding Agent Issue Localization Accuracy - An Exploratory Study}},
  booktitle =	{20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)},
  pages =	{59:1--59:13},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-450-5},
  ISSN =	{1868-8969},
  year =	{2026},
  volume =	{394},
  editor =	{Feldt, Robert and Paasivaara, Maria and Mendez, Daniel and Wagner, Stefan and Bar\'{o}n, Marvin Mu\~{n}oz},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.ESEM.2026.59},
  URN =		{urn:nbn:de:0030-drops-280271},
  doi =		{10.4230/LIPIcs.ESEM.2026.59},
  annote =	{Keywords: AI, Coding Agents, Empirical Study}
}
Document
Emerging Results, Vision & Reflection Track Paper
DOI: 10.4230/LIPIcs.ESEM.2026.62
Can Agents Reconstruct Microservice Architecture?

Authors: Venla Liljas, Matteo Esposito, Valentina Lenarduzzi, and Davide Taibi

Published in: LIPIcs, Volume 394, 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)

Abstract
Microservice architectures are increasingly becoming the standard design for cloud-native applications, yet their complex dependency structures make architectural understanding a challenge. Our research explores how Large Language Models (LLMs) can be leveraged to automatically reconstruct microservice architectures directly from source code, a task that requires both high-level architectural reasoning and fine-grained code understanding. We introduce a Minimum Viable Agent (MVA) as a single-agent that combines tool-assisted repository exploration with role-guided prompting to infer the services, inter-service connections, and exposed endpoints in Java microservice systems. We evaluate two state-of-the-art LLMs, GPT-4o and Llama4-16:17B, on 17 open-source microservice applications with 20 independent runs per system. GPT-4o achieves a mean F1 score of 0.604, outperforming Llama4 (F1 = 0.254). Both models show higher precision than recall, suggesting a tendency to under-represent architectural relationships rather than hallucinate them. Results also reveal substantial variation across applications and architectural element types. Analysis by architectural element indicates that service-to-service connections are the most challenging to recover for both models, while large performance differences emerge from endpoint reconstruction. Overall, our findings suggest that LLMs show promise for automated architecture recovery, but current capabilities remain insufficient for fully reliable reverse engineering. We discuss implications for integrating LLMs into architecture analysis workflows and outline directions for future research.
Cite as

Venla Liljas, Matteo Esposito, Valentina Lenarduzzi, and Davide Taibi. Can Agents Reconstruct Microservice Architecture?. In 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 394, pp. 62:1-62:13, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026)

Copy BibTex To Clipboard

@InProceedings{liljas_et_al:LIPIcs.ESEM.2026.62,
  author =	{Liljas, Venla and Esposito, Matteo and Lenarduzzi, Valentina and Taibi, Davide},
  title =	{{Can Agents Reconstruct Microservice Architecture?}},
  booktitle =	{20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)},
  pages =	{62:1--62:13},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-450-5},
  ISSN =	{1868-8969},
  year =	{2026},
  volume =	{394},
  editor =	{Feldt, Robert and Paasivaara, Maria and Mendez, Daniel and Wagner, Stefan and Bar\'{o}n, Marvin Mu\~{n}oz},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.ESEM.2026.62},
  URN =		{urn:nbn:de:0030-drops-280302},
  doi =		{10.4230/LIPIcs.ESEM.2026.62},
  annote =	{Keywords: Large Language Models, Microservice Architecture, Software Architecture Reconstruction, Agentic AI}
}

Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact