Document Open Access Logo

Using ChatGPT During Implementation of Programs in Education

Authors Norbert Baláž, Jaroslav Porubän , Marek Horváth , Tomáš Kormaník

PDF
Thumbnail PDF

File

OASIcs.ICPEC.2024.18.pdf
  • Filesize: 0.53 MB
  • 9 pages

Document Identifiers

Author Details

Norbert Baláž
  • Department of Computers and Informatics, Technical University of Košice, Slovakia
Jaroslav Porubän
  • Department of Computers and Informatics, Technical University of Košice, Slovakia
Marek Horváth
  • Department of Computers and Informatics, Technical University of Košice, Slovakia
Tomáš Kormaník
  • Department of Computers and Informatics, Technical University of Košice, Slovakia

Funding

This work was supported by project VEGA No. 1/0630/22 Lowering Programmers Cognitive Load Using Context-Dependent Dialogs.

Cite AsGet BibTex

Norbert Baláž, Jaroslav Porubän, Marek Horváth, and Tomáš Kormaník. Using ChatGPT During Implementation of Programs in Education. In 5th International Computer Programming Education Conference (ICPEC 2024). Open Access Series in Informatics (OASIcs), Volume 122, pp. 18:1-18:9, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2024)
https://doi.org/10.4230/OASIcs.ICPEC.2024.18

Abstract

This paper examines the impact of ChatGPT on programming education by conducting an empirical study with computer science students at the Department of Computers and Informatics at the Technical University in Košice. The study involves an experiment where students in a Component Programming course use ChatGPT to solve a programming task involving linked lists, comparing their performance and understanding with a control group that does not use the AI (artificial intelligence) tool. The task necessitated the implementation of a function to add two numbers represented as linked lists in reverse order. Our findings indicate that while ChatGPT significantly enhances the speed of task completion - students using it were nearly three times quicker on average - it may also detract from deep understanding and critical thinking, as evidenced by the uniformity and superficial engagement in solutions among the ChatGPT group. On the other hand, the group working independently displayed a broader variety of solutions and deeper interaction with the problem, despite slower completion times and occasional inaccuracies. The results highlight a dual-edged impact of AI tools in education: while they enhance efficiency, they may undermine the development of critical thinking and problem-solving skills. We discuss the implications of these findings for educational practices, emphasizing the need for a balanced approach that integrates AI tools without compromising the depth of learning and understanding in students.

Subject Classification

ACM Subject Classification
  • Software and its engineering → Software creation and management
Keywords
  • generative artificial intelligence
  • chatbot
  • ChatGPT
  • prompt engineering
  • source code generation

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

References

  1. Christos-Nikolaos Anagnostopoulos. Anagnostopoulos CN. ChatGPT impacts in programming education: A recent literature overview that debates ChatGPT responses [version 1; peer review: 1 approved with reservations]. F1000Research, 2023. URL: https://doi.org/10.12688/f1000research.141958.1.
  2. Rishabh Balse, Prajish Prasad, and Jayakrishnan Madathil Warriem. Exploring the potential of GPT-4 in automated mentoring for programming courses. In ACM Conference on Global Computing Education Vol 2, page 191, 2023. URL: https://doi.org/10.1145/3617650.3624946.
  3. Miroslav Binas. Affecting students behavior with plagiarism detection in evaluation process. In International Conference on Overcoming the Challenges and Barriers in Open Education, pages 441-441, 2018. Google Scholar
  4. Miroslav Binas and Emilia Pietrikova. Useful recommendations for successful implementation of programming courses. In IEEE International Conference of Emerging Elearning Technologies and Applications, pages 397-401, 2014. URL: https://doi.org/10.1109/ICETA.2014.7107618.
  5. Simone Grassini. Shaping the future of education: exploring the potential and consequences of AI and ChatGPT in educational settings. Education Sciences, 13(7):692, 2023. URL: https://doi.org/10.3390/educsci13070692.
  6. Mohanad Halaweh. ChatGPT in education: Strategies for responsible implementation. CONT ED TECHNOLOGY, 2023. URL: https://doi.org/10.30935/cedtech/13036.
  7. Arto Hellas, Juho Leinonen, Sami Sarsa, Charles Koutcheme, Lilja Kujanpää, and Juha Sorva. Exploring the responses of large language models to beginner programmers’ help requests. In Proceedings of the 2023 ACM Conference on International Computing Education Research - Volume 1, ICER '23, pages 93-105, New York, NY, USA, 2023. Association for Computing Machinery. URL: https://doi.org/10.1145/3568813.3600139.
  8. Marek Horvath and Emilia Pietrikova. An experimental comparison of three code similarity tools on over 1,000 student projects. In IEEE World Symposium on Applied Machine Intelligence and Informatics, pages 423-428, 2024. URL: https://doi.org/10.1109/SAMI60510.2024.10432863.
  9. Chung Kwan Lo. What is the impact of ChatGPT on education? a rapid review of the literature. Education Sciences, 13(4):410, 2023. URL: https://doi.org/10.3390/educsci13040410.
  10. Marta Montenegro-Rueda, José Fernández-Cerero, José María Fernández-Batanero, and Eloy López-Meneses. Impact of the implementation of ChatGPT in education: A systematic review. Computers, 12(8):153, 2023. URL: https://doi.org/10.3390/computers12080153.
  11. Eng Lieh Ouh, Benjamin Kok Siew Gan, Kyong Jin Shim, and Swavek Wlodkowski. ChatGPT, can you generate solutions for my coding exercises? an evaluation on its effectiveness in an undergraduate java programming course. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1, pages 54-60, 2023. URL: https://doi.org/10.1145/3587102.3588794.
  12. Emilia Pietrikova and Sergej Chodarev. Profile-driven source code exploration. In IEEE Federated Conference on Computer Science and Information Systems, pages 929-934, 2015. URL: https://doi.org/10.15439/2015F238.
  13. Jaromir Savelka, Arav Agarwal, Christopher Bogart, Yifan Song, and Majd Sakr. Can generative pre-trained transformers (GPT) pass assessments in higher education programming courses? In Conference on Innovation and Technology in Computer Science Education V. 1, pages 117-123. ACM, 2023. URL: https://doi.org/10.1145/3587102.3588792.
  14. Jody Serrano. New york city schools lift ban on ChatGPT, say initial fear "overlooked the potential" of AI. URL: https://gizmodo.com/new-york-city-public-schools-lift-ban-chatgpt-ai-1850453424.
  15. Matus Sulir, Michaela Bacikova, Matej Madeja, Sergej Chodarev, and Jan Juhar. Large-scale dataset of local java software build results. Data, 5(3):86:1-86:11, 2020. URL: https://doi.org/10.3390/data5030086.
  16. Ramazan Yilmaz and Fatma Gizem Karaoglan Yilmaz. Augmented intelligence in programming learning: Examining student views on the use of ChatGPT for programming learning. Computers in Human Behavior: Artificial Humans, 1(2):100005, 2023. URL: https://doi.org/10.1016/j.chbah.2023.100005.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact