1 Search Results for "Shahab, Anas"


Document
Complications for Computational Experiments from Modern Processors

Authors: Johannes K. Fichte, Markus Hecher, Ciaran McCreesh, and Anas Shahab

Published in: LIPIcs, Volume 210, 27th International Conference on Principles and Practice of Constraint Programming (CP 2021)


Abstract
In this paper, we revisit the approach to empirical experiments for combinatorial solvers. We provide a brief survey on tools that can help to make empirical work easier. We illustrate origins of uncertainty in modern hardware and show how strong the influence of certain aspects of modern hardware and its experimental setup can be in an actual experimental evaluation. More specifically, there can be situations where (i) two different researchers run a reasonable-looking experiment comparing the same solvers and come to different conclusions and (ii) one researcher runs the same experiment twice on the same hardware and reaches different conclusions based upon how the hardware is configured and used. We investigate these situations from a hardware perspective. Furthermore, we provide an overview on standard measures, detailed explanations on effects, potential errors, and biased suggestions for useful tools. Alongside the tools, we discuss their feasibility as experiments often run on clusters to which the experimentalist has only limited access. Our work sheds light on a number of benchmarking-related issues which could be considered to be folklore or even myths.

Cite as

Johannes K. Fichte, Markus Hecher, Ciaran McCreesh, and Anas Shahab. Complications for Computational Experiments from Modern Processors. In 27th International Conference on Principles and Practice of Constraint Programming (CP 2021). Leibniz International Proceedings in Informatics (LIPIcs), Volume 210, pp. 25:1-25:21, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2021)


Copy BibTex To Clipboard

@InProceedings{fichte_et_al:LIPIcs.CP.2021.25,
  author =	{Fichte, Johannes K. and Hecher, Markus and McCreesh, Ciaran and Shahab, Anas},
  title =	{{Complications for Computational Experiments from Modern Processors}},
  booktitle =	{27th International Conference on Principles and Practice of Constraint Programming (CP 2021)},
  pages =	{25:1--25:21},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-211-2},
  ISSN =	{1868-8969},
  year =	{2021},
  volume =	{210},
  editor =	{Michel, Laurent D.},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.CP.2021.25},
  URN =		{urn:nbn:de:0030-drops-153165},
  doi =		{10.4230/LIPIcs.CP.2021.25},
  annote =	{Keywords: Experimenting, Combinatorial Solving, Empirical Work}
}
  • Refine by Author
  • 1 Fichte, Johannes K.
  • 1 Hecher, Markus
  • 1 McCreesh, Ciaran
  • 1 Shahab, Anas

  • Refine by Classification
  • 1 Computer systems organization → Multicore architectures
  • 1 Hardware → Impact on the environment
  • 1 Hardware → Platform power issues
  • 1 Hardware → Temperature monitoring
  • 1 Theory of computation → Design and analysis of algorithms

  • Refine by Keyword
  • 1 Combinatorial Solving
  • 1 Empirical Work
  • 1 Experimenting

  • Refine by Type
  • 1 document

  • Refine by Publication Year
  • 1 2021

Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail