2026-07-22T08:26:27Z https://drops.dagstuhl.de/oai

oai:drops-oai.dagstuhl.de:9898 2024-03-06T10:27:50Z ddc:004 open_access

From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442) Ferro, Nicola Fuhr, Norbert Grefenstette, Gregory Konstan, Joseph A. Castells, Pablo Daly, Elizabeth M. Declerck, Thierry Ekstrand, Michael D. Geyer, Werner Gonzalo, Julio Kuflik, Tsvi Lindén, Krister Magnini, Bernardo Nie, Jian-Yun Perego, Raffaele Shapira, Bracha Soboroff, Ian Tintarev, Nava Verspoor, Karin Willemsen, Martijn C. Zobel, Justin Information Systems Formal models Evaluation Simulation User Interaction We describe the state-of-the-art in performance modeling and prediction for Information Retrieval (IR), Natural Language Processing (NLP) and Recommender Systems (RecSys) along with its shortcomings and strengths. We present a framework for further research, identifying five major problem areas: understanding measures, performance analysis, making underlying assumptions explicit, identifying application features determining performance, and the development of prediction models describing the relationship between assumptions, features and resulting performance. Schloss Dagstuhl – Leibniz-Zentrum für Informatik Nicola Ferro and Norbert Fuhr and Gregory Grefenstette and Joseph A. Konstan and Pablo Castells and Elizabeth M. Daly and Thierry Declerck and Michael D. Ekstrand and Werner Geyer and Julio Gonzalo and Tsvi Kuflik and Krister Lindén and Bernardo Magnini and Jian-Yun Nie and Raffaele Perego and Bracha Shapira and Ian Soboroff and Nava Tintarev and Karin Verspoor and Martijn C. Willemsen and Justin Zobel 2018 Is Part Of Dagstuhl Manifestos, Volume 7, Issue 1 (2018) Article Text doc-type:ResearchArticle publishedVersion application/pdf doi:10.4230/DagMan.7.1.96 urn:nbn:de:0030-drops-98987 https://drops.dagstuhl.de/entities/document/10.4230/DagMan.7.1.96 eng https://creativecommons.org/licenses/by/3.0/legalcode