Reproducibility of Data-Oriented Experiments in e-Science (Dagstuhl Seminar 16041)

Authors Juliana Freire, Norbert Fuhr, Andreas Rauber and all authors of the abstracts in this report



PDF
Thumbnail PDF

File

DagRep.6.1.108.pdf
  • Filesize: 1.33 MB
  • 52 pages

Document Identifiers

Author Details

Juliana Freire
Norbert Fuhr
Andreas Rauber
and all authors of the abstracts in this report

Cite AsGet BibTex

Juliana Freire, Norbert Fuhr, and Andreas Rauber. Reproducibility of Data-Oriented Experiments in e-Science (Dagstuhl Seminar 16041). In Dagstuhl Reports, Volume 6, Issue 1, pp. 108-159, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2016)
https://doi.org/10.4230/DagRep.6.1.108

Abstract

This report documents the program and the outcomes of Dagstuhl Seminar 16041 "Reproducibility of Data-Oriented Experiments in e-Science". In many subfields of computer science, experiments play an important role. Besides theoretic properties of algorithms or methods, their effectiveness and performance often can only be validated via experimentation. In most of these cases, the experimental results depend on the input data, settings for input parameters, and potentially on characteristics of the computational environment where the experiments were designed and run. Unfortunately, most computational experiments are specified only informally in papers, where experimental results are briefly described in figure captions; the code that produced the results is seldom available. This has serious implications. Scientific discoveries do not happen in isolation. Important advances are often the result of sequences of smaller, less significant steps. In the absence of results that are fully documented, reproducible, and generalizable, it becomes hard to re-use and extend these results. Besides hindering the ability of others to leverage our work, and consequently limiting the impact of our field, the absence of reproducibility experiments also puts our reputation at stake, since reliability and validity of empiric results are basic scientific principles. This seminar brought together experts from various sub-fields of computer science to create a joint understanding of the problems of reproducibility of experiments, discussing existing solutions and impediments, and proposing ways to overcome current limitations.
Keywords
  • Documentation
  • Reliability
  • Repeatibility
  • Replicability
  • reproducibility
  • Software

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail