2026-08-18T14:41:28Z https://drops.dagstuhl.de/oai

oai:drops-oai.dagstuhl.de:19638 2026-04-20T13:10:09Z ddc:004 open_access

Challenges in Benchmarking Optimization Heuristics (Dagstuhl Seminar 23251) Auger, Anne Bosman, Peter A. N. Kerschke, Pascal Whitley, Darrell Schäpermeier, Lennart benchmarking design of search heuristics optimization real-world applications understanding problem complexity This report documents the program and outcomes of the Dagstuhl Seminar 23251 "Challenges in Benchmarking Optimization Heuristics". In the domain of optimization heuristics, a stable basis for fairly evaluating the performance of optimization algorithms and other solution approaches - commonly referred to as "benchmarking" - is fundamental to ensuring steady scientific progress. Although many pitfalls are well known in the community, the development of sound benchmarking protocols is slow, and the adoption of community-wide recognized and implementable standards requires lasting and joint efforts among research groups. This seminar brought together people from diverse backgrounds and fostered discussions among different optimization communities, focusing on how to cope with "horse racing papers", landscape analysis techniques for understanding problem instances, and discussions about the overarching goals of benchmarking. Schloss Dagstuhl – Leibniz-Zentrum für Informatik Anne Auger and Peter A. N. Bosman and Pascal Kerschke and Darrell Whitley and Lennart Schäpermeier 2024 Is Part Of Dagstuhl Reports, Volume 13, Issue 6 (2024) Article Text doc-type:ResearchArticle publishedVersion application/pdf doi:10.4230/DagRep.13.6.55 urn:nbn:de:0030-drops-196383 https://drops.dagstuhl.de/entities/document/10.4230/DagRep.13.6.55 eng https://creativecommons.org/licenses/by/4.0/legalcode