Document Open Access Logo

Challenges in Benchmarking Optimization Heuristics (Dagstuhl Seminar 23251)

Authors Anne Auger, Peter A. N. Bosman, Pascal Kerschke, Darrell Whitley, Lennart Schäpermeier and all authors of the abstracts in this report



PDF
Thumbnail PDF

File

DagRep.13.6.55.pdf
  • Filesize: 7.76 MB
  • 26 pages

Document Identifiers

Author Details

Anne Auger
  • INRIA Saclay - Palaiseau, FR
Peter A. N. Bosman
  • CWI - Amsterdam, NL
Pascal Kerschke
  • TU Dresden, DE
Darrell Whitley
  • Colorado State University - Fort Collins, US
Lennart Schäpermeier
  • TU Dresden, DE
and all authors of the abstracts in this report

Cite AsGet BibTex

Anne Auger, Peter A. N. Bosman, Pascal Kerschke, Darrell Whitley, and Lennart Schäpermeier. Challenges in Benchmarking Optimization Heuristics (Dagstuhl Seminar 23251). In Dagstuhl Reports, Volume 13, Issue 6, pp. 55-80, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2024)
https://doi.org/10.4230/DagRep.13.6.55

Abstract

This report documents the program and outcomes of the Dagstuhl Seminar 23251 "Challenges in Benchmarking Optimization Heuristics". In the domain of optimization heuristics, a stable basis for fairly evaluating the performance of optimization algorithms and other solution approaches - commonly referred to as "benchmarking" - is fundamental to ensuring steady scientific progress. Although many pitfalls are well known in the community, the development of sound benchmarking protocols is slow, and the adoption of community-wide recognized and implementable standards requires lasting and joint efforts among research groups. This seminar brought together people from diverse backgrounds and fostered discussions among different optimization communities, focusing on how to cope with "horse racing papers", landscape analysis techniques for understanding problem instances, and discussions about the overarching goals of benchmarking.

Subject Classification

ACM Subject Classification
  • General and reference → Empirical studies
  • Computing methodologies → Search methodologies
Keywords
  • benchmarking
  • design of search heuristics
  • optimization
  • real-world applications
  • understanding problem complexity

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail