Search Results

Documents authored by Laval, Jannik


Document
Emerging Results, Vision & Reflection Track Paper
Parameterizing LLMs in Practice: An Empirical Study of LLMs Integrated into Software Systems

Authors: Agustín Olmedo, Jannik Laval, Christelle Urtado, and Sylvain Vauttier

Published in: LIPIcs, Volume 394, 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)


Abstract
Large Language Models (LLMs) are increasingly embedded in software projects, yet little is known about how developers configure LLM parameters in the wild. Characterizing which parameters are defined, how many per project, which groups are co-defined, and what values are preferred can inform academics and tool builders about practices. This article mines open-source Python repositories on GitHub that integrate LLMs and uses AST-based static analysis to extract parameter assignments. Project-level definitions are analyzed to address prevalence (RQ1), configuration complexity and its structure (RQ2), and value distributions (RQ3). The final corpus includes 363 projects and gathers 7892 parameter definitions, with both the dataset and the code available for reproducibility. We observe that temperature is most frequently defined (90.36%), followed by max_tokens (56.75%), top_p (49.86%), and top_k (42.70%); penalty parameters are comparatively rare (RQ1). Projects typically define few parameters (mean ≈ 3), and this limited set expands incrementally around a stable core (temperature + max_tokens) (RQ2). Distributions suggest "defaults-in-practice": temperature ≈ 0.0, 0.7 and 1.0, top_p ≈ 0.9, top_k ≈ 0-50, and max_tokens at 2^k values (e.g., 256/512/1024) (RQ3). In conclusion, developers favor minimalist, sampling-centric configurations with convergence around specific ranges. These descriptive findings support clearer configuration reporting, offer practical baselines for tools and education, and motivate future work on causes, longitudinal evolution and task/domain stratification.

Cite as

Agustín Olmedo, Jannik Laval, Christelle Urtado, and Sylvain Vauttier. Parameterizing LLMs in Practice: An Empirical Study of LLMs Integrated into Software Systems. In 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 394, pp. 60:1-60:12, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026)


Copy BibTex To Clipboard

@InProceedings{olmedo_et_al:LIPIcs.ESEM.2026.60,
  author =	{Olmedo, Agust{\'\i}n and Laval, Jannik and Urtado, Christelle and Vauttier, Sylvain},
  title =	{{Parameterizing LLMs in Practice: An Empirical Study of LLMs Integrated into Software Systems}},
  booktitle =	{20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)},
  pages =	{60:1--60:12},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-450-5},
  ISSN =	{1868-8969},
  year =	{2026},
  volume =	{394},
  editor =	{Feldt, Robert and Paasivaara, Maria and Mendez, Daniel and Wagner, Stefan and Bar\'{o}n, Marvin Mu\~{n}oz},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.ESEM.2026.60},
  URN =		{urn:nbn:de:0030-drops-280280},
  doi =		{10.4230/LIPIcs.ESEM.2026.60},
  annote =	{Keywords: Software Engineering for AI, Large Language Model, Hyperparameter, LLM configuration, LLM API, Mining Software Repository}
}

Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail