2026-07-26T00:38:23Z https://drops.dagstuhl.de/oai

oai:drops-oai.dagstuhl.de:25272 2025-12-19T09:42:21Z ddc:004 open_access

Conversational Agents: A Framework for Evaluation (CAFE) (Dagstuhl Perspectives Workshop 24352) Bauer, Christine Chen, Li Ferro, Nicola Fuhr, Norbert Anand, Avishek Breuer, Timo Faggioli, Guglielmo Frieder, Ophir Joho, Hideo Karlgren, Jussi Kiesel, Johannes Knijnenburg, Bart P. Lipani, Aldo Michiels, Lien Papenmeier, Andrea Pera, Maria Soledad Sanderson, Mark Sanner, Scott Stein, Benno Trippas, Johanne R. Verspoor, Karin Willemsen, Martijn C. Conversational Agents Evaluation Information Access During the workshop, we deeply discussed what CONversational Information ACcess (CONIAC) is and its unique features, proposing a world model abstracting it, and defined the Conversational Agents Framework for Evaluation (CAFE) for the evaluation of CONIAC systems, consisting of six major components: 1) goals of the system’s stakeholders, 2) user tasks to be studied in the evaluation, 3) aspects of the users carrying out the tasks, 4) evaluation criteria to be considered, 5) evaluation methodology to be applied, and 6) measures for the quantitative criteria chosen. Schloss Dagstuhl – Leibniz-Zentrum für Informatik Christine Bauer and Li Chen and Nicola Ferro and Norbert Fuhr and Avishek Anand and Timo Breuer and Guglielmo Faggioli and Ophir Frieder and Hideo Joho and Jussi Karlgren and Johannes Kiesel and Bart P. Knijnenburg and Aldo Lipani and Lien Michiels and Andrea Papenmeier and Maria Soledad Pera and Mark Sanderson and Scott Sanner and Benno Stein and Johanne R. Trippas and Karin Verspoor and Martijn C. Willemsen 2025 Is Part Of Dagstuhl Manifestos, Volume 11, Issue 1 (2025) Article Text doc-type:ResearchArticle publishedVersion application/pdf doi:10.4230/DagMan.11.1.19 urn:nbn:de:0030-drops-252722 https://drops.dagstuhl.de/entities/document/10.4230/DagMan.11.1.19 eng https://creativecommons.org/licenses/by/4.0/legalcode