DROPS

Document

Evaluation Methodologies in Information Retrieval (Dagstuhl Seminar 13441)

Authors: Maristella Agosti, Norbert Fuhr, Elaine Toms, and Pertti Vakkari

Published in: Dagstuhl Reports, Volume 3, Issue 10 (2014)

Abstract

This report documents the program and the outcome of Dagstuhl Seminar 13441 "Evaluation Methodologies in Information Retrieval", which brought together 42 participants from 11 countries. The seminar was motivated by the fact that today's information retrieval (IR) applications can hardly be evaluated based on the classic test collection paradigm, thus there is a need for new evaluation approaches. The event started with five introductory talks on evaluation frameworks, user modeling for evaluation, evaluation criteria, measures, evaluation methodology, and new trends in IR evaluation. The seminar participants then formed working groups addressing specific aspects of IR evaluation, such as reliability and validity, task-based IR, learning as search outcome, searching for fun, IR and social media, graph search, domain-specific IR, interaction measures and models, and searcher-aware information access systems.

Cite as

Maristella Agosti, Norbert Fuhr, Elaine Toms, and Pertti Vakkari. Evaluation Methodologies in Information Retrieval (Dagstuhl Seminar 13441). In Dagstuhl Reports, Volume 3, Issue 10, pp. 92-126, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2014)

Copy BibTex To Clipboard

@Article{agosti_et_al:DagRep.3.10.92,
  author =	{Agosti, Maristella and Fuhr, Norbert and Toms, Elaine and Vakkari, Pertti},
  title =	{{Evaluation Methodologies in Information Retrieval (Dagstuhl Seminar 13441)}},
  pages =	{92--126},
  journal =	{Dagstuhl Reports},
  ISSN =	{2192-5283},
  year =	{2014},
  volume =	{3},
  number =	{10},
  editor =	{Agosti, Maristella and Fuhr, Norbert and Toms, Elaine and Vakkari, Pertti},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/DagRep.3.10.92},
  URN =		{urn:nbn:de:0030-drops-44335},
  doi =		{10.4230/DagRep.3.10.92},
  annote =	{Keywords: evaluation, testbeds, user studies, measures}
}

Search Results

Documents authored by Toms, Elaine

Evaluation Methodologies in Information Retrieval (Dagstuhl Seminar 13441)

Abstract

Cite as

Thanks for your feedback!

Could not send message