Programming Exercises Interoperability: The Case of a Non-Picky Consumer

Authors Ricardo Queirós , José Carlos Paiva , José Paulo Leal

Thumbnail PDF


  • Filesize: 0.65 MB
  • 9 pages

Document Identifiers

Author Details

Ricardo Queirós
  • CRACS - INESC-Porto LA, Portugal
  • uniMAD - ESMAD, Polytechnic Institute of Porto, Portugal
José Carlos Paiva
  • CRACS - INESC-Porto LA, Portugal
  • DCC - FCUP, Porto, Portugal
José Paulo Leal
  • CRACS - INESC-Porto LA, Portugal
  • DCC - FCUP, Porto, Portugal

Cite AsGet BibTex

Ricardo Queirós, José Carlos Paiva, and José Paulo Leal. Programming Exercises Interoperability: The Case of a Non-Picky Consumer. In 10th Symposium on Languages, Applications and Technologies (SLATE 2021). Open Access Series in Informatics (OASIcs), Volume 94, pp. 5:1-5:9, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2021)


Problem-solving is considered one of the most important skills to retain in the coming decades for building a modern and proactive society. In this realm, computer programming learning is vital to enrich those skills. Practicing in this area boils down to solve programming exercises. In order to foster this practice, it is necessary to provide students with the best of the breed automated tools and a good set of exercises in a fair quantity covering the curricula of a typical programming course. Despite the increasing appearance of automated tools such as program evaluators, gamification engines and sophisticated web environments, access to exercises remains problematic. In fact, although the existence of several code repositories (most for feed computer programming contests), the majority of them store the exercises in proprietary formats and without any access facilities hindering their use. This leaves no other option to teachers but to manually create programming exercises which is time-consuming and error prone, or simply, reuse the same exercises, from previous years, which is considered as a detrimental and limiting approach to enhance multi-faceted and creative programmers. The article surveys the current interoperability efforts on programming exercises, more precisely, in terms of serialization formats and communication protocols. This study will sustain the selection of an API to feed a code playground called LearnJS with random programming exercises.

Subject Classification

ACM Subject Classification
  • Applied computing → Computer-managed instruction
  • Applied computing → Interactive learning environments
  • Applied computing → E-learning
  • programming exercises format
  • interoperability
  • automated assessment
  • learning programming


  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    PDF Downloads


  1. Kirsti M. Ala-Mutka. A survey of automated assessment approaches for programming assignments. Computer Science Education, 15(2):83-102, 2005. URL:
  2. Yorah Bosse and Marco Aurélio Gerosa. Why is programming so difficult to learn? patterns of difficulties related to programming learning mid-stage. SIGSOFT Softw. Eng. Notes, 41(6):1–6, 2017. URL:
  3. Patrick Buckley and Elaine Doyle. Gamification and student motivation. Interactive Learning Environments, 24(6):1162-1175, 2016. URL:
  4. Stephen H. Edwards, Jürgen Börstler, Lillian N. Cassel, Mark S. Hall, and Joseph Hollingsworth. Developing a common format for sharing programming assignments. SIGCSE Bull., 40(4):167–182, 2008. URL:
  5. José Paulo Leal and Fernando Silva. Mooshak: a web-based multi-site programming contest system. Software: Practice and Experience, 33(6):567-581, 2003. URL:
  6. José Carlos Paiva, Ricardo Queirós, José Paulo Leal, and Jakub Swacha. Fgpe authorkit – a tool for authoring gamified programming educational content. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education, ITiCSE '20, page 564, New York, NY, USA, 2020. Association for Computing Machinery. URL:
  7. José Carlos Paiva, Ricardo Queirós, José Paulo Leal, and Jakub Swacha. Yet Another Programming Exercises Interoperability Language (Short Paper). In Alberto Simões, Pedro Rangel Henriques, and Ricardo Queirós, editors, 9th Symposium on Languages, Applications and Technologies (SLATE 2020), volume 83 of OpenAccess Series in Informatics (OASIcs), pages 14:1-14:8, Dagstuhl, Germany, 2020. Schloss Dagstuhl-Leibniz-Zentrum für Informatik. URL:
  8. Ricardo Queirós. LearnJS - A JavaScript Learning Playground (Short Paper). In Pedro Rangel Henriques, José Paulo Leal, António Menezes Leitão, and Xavier Gómez Guinovart, editors, 7th Symposium on Languages, Applications and Technologies (SLATE 2018), volume 62 of OpenAccess Series in Informatics (OASIcs), pages 2:1-2:9, Dagstuhl, Germany, 2018. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik. URL:
  9. Ricardo Queiros and Jose Paulo Leal. Babelo—an extensible converter of programming exercises formats. IEEE Trans. Learn. Technol., 6(1):38–45, January 2013. URL:
  10. Ricardo Queiros and José Leal. Pexil - programming exercises interoperability language. In title = "XATA 2011: XML: associated technologies and applications", January 2011. Google Scholar
  11. Ricardo Queirós and José Paulo Leal. crimsonhex: a learning objects repository for programming exercises. Software: Practice and Experience, 43(8):911-935, 2013. URL:
  12. Alberto Simões and Ricardo Queirós. On the Nature of Programming Exercises. In Ricardo Queirós, Filipe Portela, Mário Pinto, and Alberto Simões, editors, First International Computer Programming Education Conference (ICPEC 2020), volume 81 of OpenAccess Series in Informatics (OASIcs), pages 24:1-24:9, Dagstuhl, Germany, 2020. Schloss Dagstuhl-Leibniz-Zentrum für Informatik. URL:
  13. Jakub Swacha, José Carlos Paiva, José Paulo Leal, Ricardo Queirós, Raffaele Montella, and Sokol Kosta. Gedil—gamified education interoperability language. Information, 11(6), 2020. URL:
  14. Tom Verhoeff. Programming task packages: Peach exchange format. Olympiads in Informatics, 2:192-207, January 2008. Google Scholar
Questions / Remarks / Feedback

Feedback for Dagstuhl Publishing

Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail