Document Open Access Logo

Adaptation of Automated Assessment System for Large Programming Courses

Authors Marek Horváth , Tomáš Kormaník , Jaroslav Porubän

PDF
Thumbnail PDF

File

OASIcs.ICPEC.2024.4.pdf
  • Filesize: 0.67 MB
  • 11 pages

Document Identifiers

Author Details

Marek Horváth
  • Department of Computers and Informatics, FEI TU of Košice, Slovakia
Tomáš Kormaník
  • Department of Computers and Informatics, FEI TU of Košice, Slovakia
Jaroslav Porubän
  • Department of Computers and Informatics, FEI TU of Košice, Slovakia

Funding

This work was supported by project VEGA No. 1/0630/22 “Lowering Programmers’ Cognitive Load Using Context-Dependent Dialogs”.

Cite As Get BibTex

Marek Horváth, Tomáš Kormaník, and Jaroslav Porubän. Adaptation of Automated Assessment System for Large Programming Courses. In 5th International Computer Programming Education Conference (ICPEC 2024). Open Access Series in Informatics (OASIcs), Volume 122, pp. 4:1-4:11, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2024) https://doi.org/10.4230/OASIcs.ICPEC.2024.4

Abstract

This paper presents a new automated assessment system tailored for programming courses, addressing the challenge of evaluating a large number of students in extensive courses at the Technical University of Košice. The primary issue with current systems is their inability to handle massive course loads while ensuring objective evaluation and timely feedback. Our proposed system enhances the scalability of the assessment process, allowing for the simultaneous handling of a greater volume of assignments. It is designed to provide regular and systematic feedback to students, supporting their continuous learning and improvement. To ensure the objectivity of evaluations, the system utilizes a variety of unit test suites, selecting them randomly in each assessment to discourage students from hardcoding solutions. This approach not only supports fair and precise assessments but also significantly reduces the administrative burden on educators, enabling them to meet a wide range of educational demands.

Subject Classification

ACM Subject Classification
  • Applied computing → Interactive learning environments
  • Applied computing → Computer-assisted instruction
  • Software and its engineering → Software creation and management
Keywords
  • Automated Assessment
  • Informatics Education
  • Programming Feedback Systems
  • Continuous Integration in Education
  • Code Quality Analysis
  • Educational Technology
  • Computer Science Education

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

References

  1. Burcu Alper, Selma Nazlioglu, and Hurevren Kilic. Ace-pe: An automated code evaluation software tool for programming education. In 2023 11th International Symposium on Digital Forensics and Security (ISDFS), pages 1-5, 2023. URL: https://doi.org/10.1109/ISDFS58141.2023.10131776.
  2. Anton Balaz, Norbert Adam, Emilia Pietrikova, and Branislav Mados. Modsecurity idmef module. In 2018 IEEE 16th World Symposium on Applied Machine Intelligence and Informatics (SAMI 2018): Dedicated to the Memory of Pioneer of Robotics Antal (Tony) K. Bejczy, pages 43-48. IEEE, 2018. IEEE 16th World Symposium on Applied Machine Intelligence and Informatics (SAMI) Dedicated to the Memory of Pioneer of Robotics Antal (Tony) K. Bejczy, Kosice, SLOVAKIA, FEB 07-10, 2018. Google Scholar
  3. S. Benford, E. Burke, E. Foxley, N. Gutteridge, and A. M. Zin. Experiences with the ceilidh system. In Proceedings of the International Conference in Computer Based Learning in Science, 1993. Google Scholar
  4. M. Binas. Identifying web services for automatic assessments of programming assignments. In 12th IEEE International Conference on emerging E-learning Technologies and Applications (ICETA 2014), pages 45-50. IEEE, 2014. 12th IEEE International Conference on Emerging eLearning Technologies and Applications (ICETA), Slovakia, Dec 04-05, 2014. Google Scholar
  5. Miroslav Biňas and Emília Pietriková. Impact of virtual assistant on programming novices' performance, behavior and motivation. Acta Electrotechnica et Informatica, 22(1):30-36, 2022. URL: https://doi.org/10.2478/aei-2022-0005.
  6. Skanda V. C, S. S. Prasad, and G. R. Dheemanth. Assessment of quality of program based on static analysis. In IEEE Transactions on Learning Technologies, 2019. 2019 IEEE. URL: https://doi.org/10.1109/T4E.2019.00072.
  7. P. Duch and T. Jowrki. Dante, automated assessment of programming assignments. In IEEE Transactions on Learning Technologies, 2018. 2018 IEEE. Google Scholar
  8. S. H. Edwards and M. A. Perez-Quinones. Web-cat: automatically grading programming assignments. In Proc. Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE), pages 328-328, 2008. Google Scholar
  9. M. Fabijanic, G. Dambic, B. Skracic, and M. Kolaric. Automatic evaluation of student software solutions in a virtualized environment. In 2023 46th MIPRO ICT and Electronics Convention (MIPRO), pages 642-647, 2023. URL: https://doi.org/10.23919/MIPRO57284.2023.10159927.
  10. Xiang Fu, Kai Qian, Lixin Tao, and J. Liu. Apogee – automated project grading and instant feedback system for web based computing. In SIGCSE’08, March 12–15, 2008, Portland, Oregon, USA, 2008. Copyright 2008 ACM. Google Scholar
  11. Sebastian Geiss, Tim Jentzsch, Nils Wild, and Christian Plewnia. Automatic programming assessment system for a computer science bridge course - an experience report. In 2022 29th Asia-Pacific Software Engineering Conference (APSEC), pages 527-536, 2022. URL: https://doi.org/10.1109/APSEC57359.2022.00074.
  12. J. B. Hext and J. W. Winings. An automatic grading scheme for simple programming exercises. Commun. ACM, 12(5):272-275, May 1969. Google Scholar
  13. Marek Horváth and Emília Pietriková. An experimental comparison of three code similarity tools on over 1,000 student projects. In 2024 IEEE 22nd World Symposium on Applied Machine Intelligence and Informatics (SAMI), pages 000423-000428, 2024. URL: https://doi.org/10.1109/SAMI60510.2024.10432863.
  14. S. Imam and V. Sarkar. Habanero-java library: a java 8 framework for multicore programming. In Proceedings of the 2014 International Conference on Principles and Practices of Programming on the Java Platform Virtual Machines, Languages, and Tools, pages 75-86, 2014. ACM DL. Google Scholar
  15. Mike Joy, Nathan Griffiths, and Russell Boyatt. The boss online submission and assessment system. J. Educ. Resour. Comput., 5(3):Article 2, September 2005. Google Scholar
  16. Jan Juhar and Liberios Vokorokos. Separation of concerns and concern granularity in source code. In V Novitzka, S Korecko, and A Szakal, editors, 2015 IEEE 13th International Scientific Conference on Informatics, pages 139-144. i'15; SSAKI KPI; KPI; Technicka University - Vkosiciach; ISVTS; IEEE, 2015. IEEE 13th International Scientific Conference on Informatics, Poprad, Slovakia, Nov 18-20, 2015. Google Scholar
  17. Oscar Karnalim and Simon. Promoting code quality via automated feedback on student submissions. In 2021 IEEE Frontiers in Education Conference (FIE), pages 1-5, 2021. URL: https://doi.org/10.1109/FIE49875.2021.9637193.
  18. Christian Kaufmann, Joao Pavão, and Harald Wahl. Is there a need for automated code review to be used in teaching? : From the perspective of students. In 2022 17th Iberian Conference on Information Systems and Technologies (CISTI), pages 1-6, 2022. URL: https://doi.org/10.23919/CISTI54924.2022.9820030.
  19. Matej Madeja, Miroslav Biňas, and Lukáš Prokein. Continuous analysis of assignment evaluation results from automated testing platform in iterative-style programming courses. In 2019 17th International Conference on Emerging eLearning Technologies and Applications (ICETA), pages 486-492, 2019. URL: https://doi.org/10.1109/ICETA48886.2019.9040122.
  20. Urs Von Matt. Kassandra: the automatic grading system. SIGCUE Outlook, 22(1):26-40, January 1994. Google Scholar
  21. Emilia Pietrikova and Sergej Chodarev. Profile-driven source code exploration. In M Ganzha, L Maciaszek, and M Paprzycki, editors, Proceedings of the 2015 Federated Conference on Computer Science and Information Systems, volume 5 of ACSIS-Annals of Computer Science and Information Systems, pages 929-934. IEEE Comp Soc; Polish Informat Proc Soc; IEEE Reg 8; IEEE Poland Sect Comp Soc Chapter; IEEE Poland Gdansk Sect Comp Soc Chapter; IEEE CIS Poland Sect Chapter; ACM Special Interest Grp Applied Comp; ACM Lodz Chapter; European Alliance Innovat; Polish Acad Sci, Comm Comp Sci; Polish Operat & Syst Res Soc; Eastern Cluster ICT Poland; Mazovia Cluster ICT, 2015. 3rd International Conference on Innovative Network Systems and Applications (iNetSApp) held in conjunction with Federated Conference on Computer Science and Information Systems (FedCSIS), Technical Univ Lodz, Lodz, Poland, SEP 13-16, 2015. URL: https://doi.org/10.15439/2015F238.
  22. Emilia Pietrikova, Jan Juhar, and Jana Stastna. Towards automated assessment in game-creative programming courses. In 2015 13th International Conference on emerging E-learning Technologies and Applications (ICETA), pages 307-312. The Amer Chamber of Commerce in the Slovak Republic; Elfa; TU; IEEE; Stu Fiit; Sanet; CTF atm; PPP; It Asociacia Slovenska; It News; EurActiv; PC revue; Education.sk; Infoware, 2015. 13th International Conference on Emerging eLearning Technologies and Applications (ICETA), Stary Smokovec, Slovakia, NOV 26-27, 2015. Google Scholar
  23. Adam Pinter and Sandor Szenasi. Automatic analysis and evaluation of student source codes. In 2020 IEEE 20th International Symposium on Computational Intelligence and Informatics (CINTI), pages 000161-000166, 2020. URL: https://doi.org/10.1109/CINTI51262.2020.9305819.
  24. G. Polito and M. Temperini. 2tsw: Automated assessment of computer programming assignments, in a gamified web-based system. In IEEE Transactions on Learning Technologies, 2019. 2019 IEEE. Google Scholar
  25. Kenneth A. Reek. The TRY system - or how to avoid testing student programs. In Proceedings of the twentieth SIGCSE technical symposium on Computer science education, SIGCSE '89, pages 112-116, New York, NY, USA, 1989. ACM. Google Scholar
  26. Shao Tianyi, Kuang Yulin, Huang Yihong, and Quan Yujuan. Paaa: An implementation of programming assignments automatic assessing system. In ICDEL 2019, May 24–27, 2019, Shanghai, China, 2019. 2019 Association for Computing Machinery. Google Scholar
  27. Erika Baksane Varga and Antal Kristof Fekete. Applications for automatic c code assessment. In 2023 24th International Carpathian Control Conference (ICCC), pages 21-26, 2023. URL: https://doi.org/10.1109/ICCC57093.2023.10178987.
  28. Liberios Vokorokos, Anton Balaz, and Branislav Mados. Application security through sandbox virtualization. Acta Polytechnica Hungarica, 12(1):83-101, 2015. Google Scholar
  29. Soundous Zougari, Mariam Tanana, and Abdelouahid Lyhyaoui. Towards an automatic assessment system in introductory programming courses. In 2016 International Conference on Electrical and Information Technologies (ICEIT), pages 496-499, 2016. URL: https://doi.org/10.1109/EITech.2016.7519649.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact