Document Open Access Logo

Interactive Evaluation of Complex Programming Assignments Using LLM Assistant

Authors Tomáš Kormaník , Viktória Lukáčová, Jaroslav Porubän

PDF HTML 5

Files

Thumbnail PDF
PDF
OASIcs.ICPEC.2025.13.pdf
  • Filesize: 0.49 MB
  • 10 pages
HTML5 Logo HTML (experimental)
OASIcs.ICPEC.2025.13.html

Document Identifiers

Subject Classification

ACM Subject Classification
  • General and reference → Empirical studies
  • Software and its engineering → Empirical software validation
  • Computing methodologies → Natural language generation
  • Social and professional topics → Student assessment
Keywords
  • Artificial Intelligence
  • Generative Models
  • Programming Assessment
  • Software Engineering Education

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

Generative language models present significant advancements in artificial intelligence with increasing applications in software engineering education. This paper explores the potential of customized generative dialogue models for automated assessment of programming assignments. The research introduces KP Assistant, a tailored implementation based on GPT-4o developed for a Component Programming university course. The research evaluated the effectiveness of this approach in generating relevant questions about source code, assessing student understanding, and providing objective feedback through a series of experiments with various game implementations and student testing. The findings demonstrate the feasibility of integrating such models into educational workflows while also acknowledging their present limitations. The study provides a framework for implementing similar systems in programming education, showing how generative AI can augment traditional assessment methods while maintaining pedagogical integrity.

Cite As Get BibTex

Tomáš Kormaník, Viktória Lukáčová, and Jaroslav Porubän. Interactive Evaluation of Complex Programming Assignments Using LLM Assistant. In 6th International Computer Programming Education Conference (ICPEC 2025). Open Access Series in Informatics (OASIcs), Volume 133, pp. 13:1-13:10, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/OASIcs.ICPEC.2025.13

Author Details

Tomáš Kormaník
  • Department of Computers and Informatics, FEI TU of Košice, Slovakia
Viktória Lukáčová
  • Department of Computers and Informatics, FEI TU of Košice, Slovakia
Jaroslav Porubän
  • Department of Computers and Informatics, FEI TU of Košice, Slovakia

Funding

This work was supported by project KEGA No. 061TUKE-4/2025 Building Bridges between University and High School ICT Education.

References

  1. Július Baráth and Michal Turčaník. Detection of selected attacks based on ann and classifiers. In 2023 Communication and Information Technologies (KIT), pages 1-4, 2023. URL: https://doi.org/10.1109/KIT59097.2023.10297105.
  2. Rishi Bommasani et.al. On the opportunities and risks of foundation models. In Center for Research on Foundation Models (CRFM) at the Stanford Institute for Human-Centered Artificial Intelligence (HAI), July 2022. URL: https://doi.org/10.48550/arXiv.2108.07258.
  3. Alfonso Javier García, Facundo A Froment, and María Rocío Bohórquez. University teacher credibility as a strategy to motivate students. Journal of New Approaches in Educational Research, 12(2):292-306, 2023. URL: https://doi.org/10.7821/naer.2023.7.1469.
  4. Rim Gouia-Zarrad and Cindy Gunn. Enhancing students’ learning experience in mathematics class through ChatGPT. International Electronic Journal of Mathematics Education, 2024. URL: https://doi.org/10.29333/iejme/14614.
  5. Marek Horváth, Tomáš Kormaník, and Jaroslav Porubän. Adaptation of Automated Assessment System for Large Programming Courses. In André L. Santos and Maria Pinto-Albuquerque, editors, 5th International Computer Programming Education Conference (ICPEC 2024), volume 122 of Open Access Series in Informatics (OASIcs), pages 4:1-4:11, Dagstuhl, Germany, 2024. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/OASIcs.ICPEC.2024.4.
  6. Marek Horváth, Lenka Bubeňková, and Emília Pietriková. Exploring gpt-generated variations in c programming assignments. In 2025 IEEE 23rd World Symposium on Applied Machine Intelligence and Informatics (SAMI), pages 000331-000336, 2025. URL: https://doi.org/10.1109/SAMI63904.2025.10883288.
  7. Marek Horváth and Filip Gurbáľ. Code clones: A novel approach to detecting plagiarism in binary decomposition of c programs. Acta Electrotechnica et Informatica, 24(2):13-18, 2024. URL: https://www.proquest.com/scholarly-journals/code-clones-novel-approach-detecting-plagiarism/docview/3173384211/se-2.
  8. Ramazan Yilmaz and Fatma Gizem Karaoglan Yilmaz. The effect of generative artificial intelligence (AI)-based tool use on students' computational thinking skills, programming self-efficacy and motivation. Computers and Education: Artificial Intelligence, 4:100147, 2023. URL: https://doi.org/10.1016/j.caeai.2023.100147.
  9. Albert Ziegler, Eirini Kalliamvakou, Shawn Simister, Ganesh Sittampalam, Alice Li, Andrew Rice, Devon Rifkin, and Edward Aftandilian. Productivity assessment of neural code completion. In Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming, May 2022. URL: https://doi.org/10.48550/arXiv.2205.06537.
  10. Faysal Özdaş. Teachers’ immediacy behaviors and academic achievement: A relational analysis. SAGE Open, 12(2):21582440221091722, 2022. URL: https://doi.org/10.1177/21582440221091722.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact