Integration of Computer Science Assessment into Learning Management Systems with JuezLTI

Authors Juan V. Carrillo , Alberto Sierra, José Paulo Leal , Ricardo Queirós , Salvador Pellicer, Marco Primo

Thumbnail PDF


  • Filesize: 0.52 MB
  • 8 pages

Document Identifiers

Author Details

Juan V. Carrillo
  • CIFP Carlos III, Cartagena, Spain
Alberto Sierra
  • CIFP Carlos III, Cartagena, Spain
José Paulo Leal
  • CRACS - INESC-Porto LA & DCC - FCUP, Porto, Portugal
Ricardo Queirós
  • CRACS - INESC-Porto LA & uniMAD, ESMAD/P. Porto, Portugal
Salvador Pellicer
  • Entornos de Formación (EdF), Valencia, Spain
Marco Primo
  • Faculty of Sciences, University of Porto, Portugal

Cite AsGet BibTex

Juan V. Carrillo, Alberto Sierra, José Paulo Leal, Ricardo Queirós, Salvador Pellicer, and Marco Primo. Integration of Computer Science Assessment into Learning Management Systems with JuezLTI. In Third International Computer Programming Education Conference (ICPEC 2022). Open Access Series in Informatics (OASIcs), Volume 102, pp. 9:1-9:8, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2022)


Computer science is a skill that will continue to be in high demand in the foreseeable future. Despite this trend, automated assessment in computer science is often hampered by the lack of systems supporting a wide range of topics. While there is a number of open software systems and programming exercise collections supporting automated assessment, up to this date, there are few systems that offer a diversity of exercises ranging from computer programming exercises to markup and databases languages. At the same time, most of the best-of-breed solutions force teachers and students to alternate between the Learning Management System - a pivotal piece of the e-learning ecosystem - and the tool providing the exercises. This issue is addressed by JuezLTI, an international project whose goal is to create an innovative tool to allow the automatic assessment of exercises in a wide range of computer science topics. These topics include different languages used in computer science for programming, markup, and database management. JuezLTI borrows part of its name from the IMS Learning Tools Interoperability (IMS LTI) standard. With this standard, the tool will interoperate with reference systems such as Moodle, Sakai, Canvas, or Blackboard, among many others. Another contribution of JuezLTI will be a pool of exercises. Interoperability and content are expected to foster the adoption of JuezLTI by many institutions. This paper presents the JuezLTI project, its architecture, and its main components.

Subject Classification

ACM Subject Classification
  • Applied computing → Computer-managed instruction
  • Applied computing → Interactive learning environments
  • Applied computing → E-learning
  • programming
  • interoperability
  • automatic assessment
  • programming exercises


  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    PDF Downloads


  1. Alberto Abelló, M. Elena Rodríguez, Toni Urpí, Xavier Burgués, M. José Casany, Carme Martín, and Carme Quer. LEARN-SQL: Automatic assessment of SQL based on IMS QTI specification. In 2008 Eighth IEEE International Conference on Advanced Learning Technologies, pages 592-593, 2008. URL:
  2. Míriam Antón-Rodríguez, María Ángeles Pérez-Juárez, Francisco Javier Díaz-Pernas, David González-Ortega, Mario Martínez-Zarzuela, and Javier Manuel Aguiar-Pérez. An Experience of Game-Based Learning in Web Applications Development Courses. In Ricardo Queirós, Filipe Portela, Mário Pinto, and Alberto Simões, editors, First International Computer Programming Education Conference (ICPEC 2020), volume 81 of OpenAccess Series in Informatics (OASIcs), pages 3:1-3:11, Dagstuhl, Germany, 2020. Schloss Dagstuhl-Leibniz-Zentrum für Informatik. URL:
  3. European Commission. Coding - the 21st century skill, 2018. accessed on 20 Jan 2020. URL:
  4. IMS Global Learning Consortium. Learning tools interoperability core specification, 2019. accessed on 14 Apr 2022. URL:
  5. Paul Conway and Ann Arbor. Digitization for everybody (Dig4E). Archiving Conference, 2020:12-16, April 2020. URL:
  6. Nikolas Galanis, Marc Alier, María José Casany, Enric Mayol, and Charles Severance. Tsugi: A framework for building PHP-based learning tools. In Proceedings of the Second International Conference on Technological Ecosystems for Enhancing Multiculturality, TEEM '14, pages 409-413, New York, NY, USA, 2014. Association for Computing Machinery. URL:
  7. Ryan Hardt and Esther Gutzmer. Database query analyzer (DBQA): A data-oriented SQL clause visualization tool. In Proceedings of the 18th Annual Conference on Information Technology Education, SIGITE '17, pages 147-152, New York, NY, USA, 2017. Association for Computing Machinery. URL:
  8. JuezLTI Project Consortium. JuezLTI - automatic assessment of computing exercises using LTI standard, 2021. accessed on 12 Apr 2022. URL:
  9. José Paulo Leal and Ricardo Queirós. Using the learning tools interoperability framework for LMS integration in service oriented architectures. Technology Enhanced Learning TECH-EDUCATION'11, 2011. Google Scholar
  10. José Paulo Leal, Ricardo Queirós, and Duarte Ferreira. Specifying a programming exercises evaluation service on the e-framework. In Xiangfeng Luo, Marc Spaniol, Lizhe Wang, Qing Li, Wolfgang Nejdl, and Wu Zhang, editors, Advances in Web-Based Learning - ICWL 2010 - 9th International Conference, Shanghai, China, December 8-10, 2010. Proceedings, volume 6483 of Lecture Notes in Computer Science, pages 141-150. Springer, 2010. URL:
  11. Jeff Merriman, Tom Coppeto, Francesc Santanach Delisau, Cole Shaw, and Xavier Aracil Díaz. Next generation learning architecture. eLearn Center. Universitat Oberta de Catalunya, 2016. Google Scholar
  12. José Carlos Paiva, Ricardo Queirós, José Paulo Leal, and Jakub Swacha. Fgpe authorkit-a tool for authoring gamified programming educational content. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education, pages 564-564, 2020. Google Scholar
  13. José Carlos Paiva, Ricardo Queirós, José Paulo Leal, and Jakub Swacha. Yet Another Programming Exercises Interoperability Language (Short Paper). In Alberto Simões, Pedro Rangel Henriques, and Ricardo Queirós, editors, 9th Symposium on Languages, Applications and Technologies (SLATE 2020), volume 83 of OpenAccess Series in Informatics (OASIcs), pages 14:1-14:8, Dagstuhl, Germany, 2020. Schloss Dagstuhl-Leibniz-Zentrum für Informatik. URL:
  14. Charles Severance, Ted Hanss, and Josepth Hardin. IMS learning tools interoperability: Enabling a mash-up approach to teaching and learning tools. Technology, Instruction, Cognition and Learning, 7(3-4):245-262, 2010. Google Scholar
  15. Anuj Ramesh Shah. Web-cat: A web-based center for automated testing. PhD thesis, Virginia Tech, 2003. Google Scholar
  16. Antonio J. Sierra, Álvaro Martín-Rodríguez, Teresa Ariza, Javier Muñoz-Calle, and Francisco J. Fernández-Jiménez. LTI for interoperating e-assessment tools with LMS. In Mauro Caporuscio, Fernando De la Prieta, Tania Di Mascio, Rosella Gennari, Javier Gutiérrez Rodríguez, and Pierpaolo Vittorini, editors, Methodologies and Intelligent Systems for Technology Enhanced Learning, pages 173-181, Cham, 2016. Springer International Publishing. Google Scholar
  17. Zahid Ullah, Adidah Lajis, Mona Jamjoom, Abdulrahman Altalhi, Abdullah Al-Ghamdi, and Farrukh Saleem. The effect of automatic assessment on novice programming: Strengths and limitations of existing systems. Computer Applications in Engineering Education, 26(6):2328-2341, 2018. URL: