Ferro, Nicola ;
Fuhr, Norbert ;
Grefenstette, Gregory ;
Konstan, Joseph A. ;
Castells, Pablo ;
Daly, Elizabeth M. ;
Declerck, Thierry ;
Ekstrand, Michael D. ;
Geyer, Werner ;
Gonzalo, Julio ;
Kuflik, Tsvi ;
Lindén, Krister ;
Magnini, Bernardo ;
Nie, Jian-Yun ;
Perego, Raffaele ;
Shapira, Bracha ;
Soboroff, Ian ;
Tintarev, Nava ;
Verspoor, Karin ;
Willemsen, Martijn C. ;
Zobel, Justin
Weitere Beteiligte (Hrsg. etc.): Nicola Ferro et al.
From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442)
Abstract
We describe the state-of-the-art in performance modeling and prediction for Information Retrieval (IR), Natural Language Processing (NLP) and Recommender Systems (RecSys) along with its shortcomings and strengths. We present a framework for further research, identifying five major problem areas: understanding measures, performance analysis, making underlying assumptions explicit, identifying application features determining performance, and the development of prediction models describing the relationship between assumptions, features and resulting performance.
BibTeX - Entry
@Article{ferro_et_al:DM:2018:9898,
author = {Nicola Ferro and Norbert Fuhr and Gregory Grefenstette and Joseph A. Konstan and Pablo Castells and Elizabeth M. Daly and Thierry Declerck and Michael D. Ekstrand and Werner Geyer and Julio Gonzalo and Tsvi Kuflik and Krister Lind{\'e}n and Bernardo Magnini and Jian-Yun Nie and Raffaele Perego and Bracha Shapira and Ian Soboroff and Nava Tintarev and Karin Verspoor and Martijn C. Willemsen and Justin Zobel},
title = {{From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442)}},
pages = {96--139},
journal = {Dagstuhl Manifestos},
ISSN = {2193-2433},
year = {2018},
volume = {7},
number = {1},
editor = {Nicola Ferro et al.},
publisher = {Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik},
address = {Dagstuhl, Germany},
URL = {http://drops.dagstuhl.de/opus/volltexte/2018/9898},
URN = {urn:nbn:de:0030-drops-98987},
doi = {10.4230/DagMan.7.1.96},
annote = {Keywords: Information Systems, Formal models, Evaluation, Simulation, User Interaction}
}
|
Keywords: |
|
Information Systems, Formal models, Evaluation, Simulation, User Interaction |
|
Seminar: |
|
Dagstuhl Manifestos, Volume 7, Issue 1
|
|
Issue date: |
|
2018 |
|
Date of publication: |
|
21.11.2018 |
21.11.2018
