Document Open Access Logo

Evaluating Usability and Cognitive Load in Programming Education with ChatGPT Integration

Authors Gustavo Gutiérrez Carreón , Rigoberto López Escalera

PDF HTML 5

Files

Thumbnail PDF
PDF
OASIcs.ICPEC.2025.12.pdf
  • Filesize: 397 kB
  • 7 pages
HTML5 Logo HTML (experimental)
OASIcs.ICPEC.2025.12.html

Document Identifiers

Subject Classification

ACM Subject Classification
  • Computing methodologies → Apprenticeship learning
Keywords
  • Usability
  • Cognitive Load
  • ChatGPT-assisted Programming Education

Metrics

Abstract

This study analyzes the impact of ChatGPT on usability and cognitive load in programming education for undergraduate students in an Information Technology Management program. The goal is to evaluate whether ChatGPT improves learning outcomes in programming topics. A comparative research design was used with two groups: a traditional instruction control group and an experimental group using ChatGPT as a support tool. Data were collected using the System Usability Scale (SUS), a custom usability questionnaire, cognitive load surveys, and programming performance evaluations. The results indicate that the students using ChatGPT reported greater usability and a lower cognitive load. They also showed improved comprehension and problem-solving skills. However, improper use of the tool can lead to superficial learning, highlighting the need for structured guidance. The findings suggest that, when integrated appropriately, ChatGPT can improve the learning experience by reducing mental effort and enhancing participation in programming education. Recommendations are offered to help educators incorporate AI tools effectively and responsibly.

Cite As Get BibTex

Gustavo Gutiérrez Carreón and Rigoberto López Escalera. Evaluating Usability and Cognitive Load in Programming Education with ChatGPT Integration. In 6th International Computer Programming Education Conference (ICPEC 2025). Open Access Series in Informatics (OASIcs), Volume 133, pp. 12:1-12:7, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/OASIcs.ICPEC.2025.12

Author Details

Gustavo Gutiérrez Carreón
  • Universidad Michoacana de San Nicolás de Hidalgo, Morelia, Mexico
Rigoberto López Escalera
  • Universidad Michoacana de San Nicolás de Hidalgo, Morelia, Mexico

Funding

  • Carreón, Gustavo Gutiérrez: CIC-UMSNH.
  • Escalera, Rigoberto López: CIC-UMSNH.

References

  1. Ibrahim Adeshola and Adeola Praise Adepoju. The opportunities and challenges of chatgpt in education. Interactive Learning Environments, 32(10):6159-6172, 2024. Google Scholar
  2. Aaron Bangor, Philip Kortum, and James Miller. Determining what individual sus scores mean: Adding an adjective rating scale. Journal of usability studies, 4(3):114-123, 2009. Google Scholar
  3. Aaron Bangor, Philip T Kortum, and James T Miller. An empirical evaluation of the system usability scale. Intl. Journal of Human-Computer Interaction, 24(6):574-594, 2008. Google Scholar
  4. John Brooke et al. Sus-a quick and dirty usability scale. Usability evaluation in industry, 189(194):4-7, 1996. Google Scholar
  5. Ruth C Clark, Frank Nguyen, and John Sweller. Efficiency in learning: Evidence-based guidelines to manage cognitive load. John Wiley & Sons, 2011. Google Scholar
  6. Laode Muhamad Fathun, Chiara Vincha, Aisha Saharani, Algis Zalita Pitra, et al. Knowledge sharing implications of the emergence of gpt chat on learning methods at smp 4 muhammadiyah depok, west java. Jurnal Pengabdian dan Pemberdayaan Masyarakat Indonesia, 3(3):130-138, 2023. Google Scholar
  7. Alberto Grájeda, Johnny Burgos, Pamela Córdova, and Alberto Sanjinés. Assessing student-perceived impact of using artificial intelligence tools: Construction of a synthetic index of application in higher education. Cogent Education, 11(1):2287917, 2024. Google Scholar
  8. Ahmad Haidar. Chatgpt and generative ai in educational ecosystems: Transforming student engagement and ensuring digital safety. In Preparing Students for the Future Educational Paradigm, pages 70-100. IGI Global Scientific Publishing, 2024. Google Scholar
  9. James R Lewis and Jeff Sauro. Item benchmarks for the system usability scale. Journal of Usability studies, 13(3), 2018. Google Scholar
  10. Elena Novak, Jerry Daday, and Kerrie McDaniel. Assessing intrinsic and extraneous cognitive complexity of e-textbook learning. Interacting with Computers, 30(2):150-161, 2018. Google Scholar
  11. Matthew Schmidt and Rui Huang. Defining learning experience design: Voices from the field of learning design & technology. TechTrends, 66(2):141-158, 2022. Google Scholar
  12. Muhammad Shidiq. The use of artificial intelligence-based chat-gpt and its challenges for the world of education; from the viewpoint of the development of creative writing skills. In Proceeding of international conference on education, society and humanity, volume 1, pages 353-357, 2023. Google Scholar
  13. John Sweller. Cognitive load theory and educational technology. Educational technology research and development, 68(1):1-16, 2020. Google Scholar
  14. Andrew A Tawfik, Linda Payne, Andrew M Olney, and Heather Ketter. Exploring the relationship between usability and cognitive load in data science education. The Journal of Applied Instructional Design, 12(3), 2023. Google Scholar
  15. Alessia Tripaldelli, Brian Butka, and Casey Elder. The development and evaluation of artificial intelligence (ai) tutor for a java programming class. Google Scholar
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact