Search Results

Documents authored by Magalhães, Cleyton

Document
Technical Track Paper
DOI: 10.4230/LIPIcs.ESEM.2026.6
The Influence of Fraudulent AI-Generated Responses on Software Engineering Surveys

Authors: Ronnie de Souza Santos, Italo Santos, Maria Teresa Baldassarre, Cleyton Magalhães, and Mairieli Wessel

Published in: LIPIcs, Volume 394, 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)

Abstract
Background. Large Language Models (LLMs) introduce new concerns regarding fraudulent or AI-assisted participation in software engineering surveys. Aims. This study investigates how suspicious or potentially AI-assisted responses may affect the validity of software engineering survey findings. Method. We conducted a secondary analysis of four software engineering survey datasets using manual identification of suspicious responses, automated AI-generated text detection, descriptive statistical analysis, and thematic analysis. We compared findings obtained from the original and manually cleaned datasets. Results. Quantitative findings generally remained stable after filtering suspicious responses, although some demographic and analytical variables showed moderate variation, affecting the interpretation of specific participant groups and contextual characteristics. In contrast, qualitative findings were more strongly influenced by changes in contextual framing, code prominence, and the nature of the evidence supporting interpretation, shaping how participants' experiences and study contexts were interpreted and characterized. Conclusions. AI-assisted participation may influence software engineering survey findings differently depending on the type of analysis being conducted. The findings reinforce the importance of combining multiple validation procedures, particularly in studies relying on open-ended responses.
Cite as

Ronnie de Souza Santos, Italo Santos, Maria Teresa Baldassarre, Cleyton Magalhães, and Mairieli Wessel. The Influence of Fraudulent AI-Generated Responses on Software Engineering Surveys. In 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 394, pp. 6:1-6:21, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026)

Copy BibTex To Clipboard

@InProceedings{desouzasantos_et_al:LIPIcs.ESEM.2026.6,
  author =	{de Souza Santos, Ronnie and Santos, Italo and Baldassarre, Maria Teresa and Magalh\~{a}es, Cleyton and Wessel, Mairieli},
  title =	{{The Influence of Fraudulent AI-Generated Responses on Software Engineering Surveys}},
  booktitle =	{20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)},
  pages =	{6:1--6:21},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-450-5},
  ISSN =	{1868-8969},
  year =	{2026},
  volume =	{394},
  editor =	{Feldt, Robert and Paasivaara, Maria and Mendez, Daniel and Wagner, Stefan and Bar\'{o}n, Marvin Mu\~{n}oz},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.ESEM.2026.6},
  URN =		{urn:nbn:de:0030-drops-279748},
  doi =		{10.4230/LIPIcs.ESEM.2026.6},
  annote =	{Keywords: LLMs, survey, threats to validity}
}
Document
Emerging Results, Vision & Reflection Track Paper
DOI: 10.4230/LIPIcs.ESEM.2026.61
How Many Interviews Are Enough in a Software Engineering Study? Preliminary Findings on Sample Size and Saturation

Authors: Ronnie de Souza Santos, Italo Santos, Mauricio Rodrigues Lima, and Cleyton Magalhães

Published in: LIPIcs, Volume 394, 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)

Abstract
Background. Interview-based studies are widely used in empirical software engineering to investigate human, organizational, and socio-technical phenomena, yet interview sample adequacy and saturation are reported inconsistently across the literature. Aims. This paper investigates how interview sample adequacy and saturation are reported in empirical software engineering research. Method. We analyzed 427 papers published between 2016 and 2025 across major software engineering venues, focusing on interview sample sizes, saturation discussions, and sample adequacy justifications. Results. Preliminary findings indicate substantial variation in interview sample sizes, ranging from highly specialized small-sample studies to broader investigations involving large interview datasets. Studies involving 12 or fewer interviewees were common and frequently associated with specialized industrial contexts or constrained organizational access. However, the most common range was 13 to 24 interviewees, suggesting that moderate-sized samples represent the most common configuration in empirical software engineering research. Saturation and sample adequacy justifications were heterogeneous, with many studies relying on implicit or contextual reasoning rather than explicit methodological discussion. Conclusions. Our findings provide empirical insights into methodological reporting practices in interview-based software engineering research and contribute to ongoing discussions on qualitative rigor and transparency.
Cite as

Ronnie de Souza Santos, Italo Santos, Mauricio Rodrigues Lima, and Cleyton Magalhães. How Many Interviews Are Enough in a Software Engineering Study? Preliminary Findings on Sample Size and Saturation. In 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 394, pp. 61:1-61:12, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026)

Copy BibTex To Clipboard

@InProceedings{desouzasantos_et_al:LIPIcs.ESEM.2026.61,
  author =	{de Souza Santos, Ronnie and Santos, Italo and Lima, Mauricio Rodrigues and Magalh\~{a}es, Cleyton},
  title =	{{How Many Interviews Are Enough in a Software Engineering Study? Preliminary Findings on Sample Size and Saturation}},
  booktitle =	{20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)},
  pages =	{61:1--61:12},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-450-5},
  ISSN =	{1868-8969},
  year =	{2026},
  volume =	{394},
  editor =	{Feldt, Robert and Paasivaara, Maria and Mendez, Daniel and Wagner, Stefan and Bar\'{o}n, Marvin Mu\~{n}oz},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.ESEM.2026.61},
  URN =		{urn:nbn:de:0030-drops-280299},
  doi =		{10.4230/LIPIcs.ESEM.2026.61},
  annote =	{Keywords: qualitative research, interviews, publications}
}

Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact