4 Search Results for "Hoos, Holger"


Document
Statistical Comparison of Algorithm Performance Through Instance Selection

Authors: Théo Matricon, Marie Anastacio, Nathanaël Fijalkow, Laurent Simon, and Holger H. Hoos

Published in: LIPIcs, Volume 210, 27th International Conference on Principles and Practice of Constraint Programming (CP 2021)


Abstract
Empirical performance evaluations, in competitions and scientific publications, play a major role in improving the state of the art in solving many automated reasoning problems, including SAT, CSP and Bayesian network structure learning (BNSL). To empirically demonstrate the merit of a new solver usually requires extensive experiments, with computational costs of CPU years. This not only makes it difficult for researchers with limited access to computational resources to test their ideas and publish their work, but also consumes large amounts of energy. We propose an approach for comparing the performance of two algorithms: by performing runs on carefully chosen instances, we obtain a probabilistic statement on which algorithm performs best, trading off between the computational cost of running algorithms and the confidence in the result. We describe a set of methods for this purpose and evaluate their efficacy on diverse datasets from SAT, CSP and BNSL. On all these datasets, most of our approaches were able to choose the correct algorithm with about 95% accuracy, while using less than a third of the CPU time required for a full comparison; the best methods reach this level of accuracy within less than 15% of the CPU time for a full comparison.

Cite as

Théo Matricon, Marie Anastacio, Nathanaël Fijalkow, Laurent Simon, and Holger H. Hoos. Statistical Comparison of Algorithm Performance Through Instance Selection. In 27th International Conference on Principles and Practice of Constraint Programming (CP 2021). Leibniz International Proceedings in Informatics (LIPIcs), Volume 210, pp. 43:1-43:21, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2021)


Copy BibTex To Clipboard

@InProceedings{matricon_et_al:LIPIcs.CP.2021.43,
  author =	{Matricon, Th\'{e}o and Anastacio, Marie and Fijalkow, Nathana\"{e}l and Simon, Laurent and Hoos, Holger H.},
  title =	{{Statistical Comparison of Algorithm Performance Through Instance Selection}},
  booktitle =	{27th International Conference on Principles and Practice of Constraint Programming (CP 2021)},
  pages =	{43:1--43:21},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-211-2},
  ISSN =	{1868-8969},
  year =	{2021},
  volume =	{210},
  editor =	{Michel, Laurent D.},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops-dev.dagstuhl.de/entities/document/10.4230/LIPIcs.CP.2021.43},
  URN =		{urn:nbn:de:0030-drops-153346},
  doi =		{10.4230/LIPIcs.CP.2021.43},
  annote =	{Keywords: Performance assessment, early stopping, automated reasoning solvers}
}
Document
Automating Data Science (Dagstuhl Seminar 18401)

Authors: Tijl De Bie, Luc De Raedt, Holger H. Hoos, and Padhraic Smyth

Published in: Dagstuhl Reports, Volume 8, Issue 9 (2019)


Abstract
Data science is concerned with the extraction of knowledge and insight, and ultimately societal or economic value, from data. It complements traditional statistics in that its object is data as it presents itself in the wild (often complex and heterogeneous, noisy, loosely structured, biased, etc.), rather than well-structured data sampled in carefully designed studies. It also has a strong computer science focus, and is related to popular areas such as big data, machine learning, data mining and knowledge discovery. Data science is becoming increasingly important with the abundance of big data, while the number of skilled data scientists is lagging. This has raised the question as to whether it is possible to automate data science in several contexts. First, from an artificial intelligence perspective, it is interesting to investigate whether (data) science (or portions of it) can be automated, as it is an activity currently requiring high levels of human expertise. Second, the field of machine learning has a long-standing interest in applying machine learning at the meta-level, in order to obtain better machine learning algorithms, yielding recent successes in automated parameter tuning, algorithm configuration and algorithm selection. Third, there is an interest in automating not only the model building process itself (cf. the Automated Statistician) but also in automating the preprocessing steps (data wrangling). This Dagstuhl seminar brought together researchers from all areas concerned with data science in order to study whether, to what extent, and how data science can be automated.

Cite as

Tijl De Bie, Luc De Raedt, Holger H. Hoos, and Padhraic Smyth. Automating Data Science (Dagstuhl Seminar 18401). In Dagstuhl Reports, Volume 8, Issue 9, pp. 154-181, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2019)


Copy BibTex To Clipboard

@Article{debie_et_al:DagRep.8.9.154,
  author =	{De Bie, Tijl and De Raedt, Luc and Hoos, Holger H. and Smyth, Padhraic},
  title =	{{Automating Data Science  (Dagstuhl Seminar 18401)}},
  pages =	{154--181},
  journal =	{Dagstuhl Reports},
  ISSN =	{2192-5283},
  year =	{2019},
  volume =	{8},
  number =	{9},
  editor =	{De Bie, Tijl and De Raedt, Luc and Hoos, Holger H. and Smyth, Padhraic},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops-dev.dagstuhl.de/entities/document/10.4230/DagRep.8.9.154},
  URN =		{urn:nbn:de:0030-drops-103443},
  doi =		{10.4230/DagRep.8.9.154},
  annote =	{Keywords: artificial intelligence, automated machine learning, automated scientific discovery, data science, inductive programming}
}
Document
Automated Algorithm Selection and Configuration (Dagstuhl Seminar 16412)

Authors: Holger H. Hoos, Frank Neumann, and Heike Trautmann

Published in: Dagstuhl Reports, Volume 6, Issue 10 (2017)


Abstract
This report documents the programme and the outcomes of Dagstuhl Seminar 16412 "Automated Algorithm Selection and Configuration", which was held October 9--14, 2016 and attended by 34 experts from 10 countries. Research on automated algorithm selection and configuration has lead to some of the most impressive successes within the broader area of empirical algorithmics, and has proven to be highly relevant to industrial applications. Specifically, high-performance algorithms for cnp-hard problems, such as propositional satisfiability (SAT) and mixed integer programming (MIP), are known to have a huge impact on sectors such as manufacturing, logistics, healthcare, finance, agriculture and energy systems, and algorithm selection and configuration techniques have been demonstrated to achieve substantial improvements in the performance of solvers for these problems. Apart from creating synergy through close interaction between the world's leading groups in the area, the seminar pursued two major goals: to promote and develop deeper understanding of the behaviour of algorithm selection and configuration techniques and to lay the groundwork for further improving their efficacy. Towards these ends, the organisation team brought together a group of carefully chosen researchers with strong expertise in computer science, statistics, mathematics, economics and engineering; a particular emphasis was placed on bringing together theorists, empiricists and experts from various application areas, with the goal of closing the gap between theory and practice.

Cite as

Holger H. Hoos, Frank Neumann, and Heike Trautmann. Automated Algorithm Selection and Configuration (Dagstuhl Seminar 16412). In Dagstuhl Reports, Volume 6, Issue 10, pp. 33-74, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2017)


Copy BibTex To Clipboard

@Article{hoos_et_al:DagRep.6.10.33,
  author =	{Hoos, Holger H. and Neumann, Frank and Trautmann, Heike},
  title =	{{Automated Algorithm Selection and Configuration (Dagstuhl Seminar 16412)}},
  pages =	{33--74},
  journal =	{Dagstuhl Reports},
  ISSN =	{2192-5283},
  year =	{2017},
  volume =	{6},
  number =	{10},
  editor =	{Hoos, Holger H. and Neumann, Frank and Trautmann, Heike},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops-dev.dagstuhl.de/entities/document/10.4230/DagRep.6.10.33},
  URN =		{urn:nbn:de:0030-drops-69569},
  doi =		{10.4230/DagRep.6.10.33},
  annote =	{Keywords: algorithm configuration, algorithm selection, features, machine learning, optimisation, performance prediction}
}
Document
aspeed: ASP-based Solver Scheduling

Authors: Holger Hoos, Roland Kaminski, Torsten Schaub, and Marius Schneider

Published in: LIPIcs, Volume 17, Technical Communications of the 28th International Conference on Logic Programming (ICLP'12) (2012)


Abstract
Although Boolean Constraint Technology has made tremendous progress over the last decade, it suffers from a great sensitivity to search configuration. This problem was impressively counterbalanced at the 2011 SAT Competition by the rather simple approach of ppfolio relying on a handmade, uniform and unordered solver schedule. Inspired by this, we take advantage of the modeling and solving capacities of ASP to automatically determine more refined, that is, non-uniform and ordered solver schedules from existing benchmarking data. We begin by formulating the determination of such schedules as multi-criteria optimization problems and provide corresponding ASP encodings. The resulting encodings are easily customizable for different settings and the computation of optimum schedules can mostly be done in the blink of an eye, even when dealing with large runtime data sets stemming from many solvers on hundreds to thousands of instances. Also, its high customizability made it easy to generate even parallel schedules for multi-core machines.

Cite as

Holger Hoos, Roland Kaminski, Torsten Schaub, and Marius Schneider. aspeed: ASP-based Solver Scheduling. In Technical Communications of the 28th International Conference on Logic Programming (ICLP'12). Leibniz International Proceedings in Informatics (LIPIcs), Volume 17, pp. 176-187, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2012)


Copy BibTex To Clipboard

@InProceedings{hoos_et_al:LIPIcs.ICLP.2012.176,
  author =	{Hoos, Holger and Kaminski, Roland and Schaub, Torsten and Schneider, Marius},
  title =	{{aspeed: ASP-based Solver Scheduling}},
  booktitle =	{Technical Communications of the 28th International Conference on Logic Programming (ICLP'12)},
  pages =	{176--187},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-939897-43-9},
  ISSN =	{1868-8969},
  year =	{2012},
  volume =	{17},
  editor =	{Dovier, Agostino and Santos Costa, V{\'\i}tor},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops-dev.dagstuhl.de/entities/document/10.4230/LIPIcs.ICLP.2012.176},
  URN =		{urn:nbn:de:0030-drops-36208},
  doi =		{10.4230/LIPIcs.ICLP.2012.176},
  annote =	{Keywords: Algorithm Schedule, Portfolio-based Solving, Answer Set Programming}
}
  • Refine by Author
  • 3 Hoos, Holger H.
  • 1 Anastacio, Marie
  • 1 De Bie, Tijl
  • 1 De Raedt, Luc
  • 1 Fijalkow, Nathanaël
  • Show More...

  • Refine by Classification
  • 1 General and reference → Evaluation
  • 1 Theory of computation → Automated reasoning
  • 1 Theory of computation → Constraint and logic programming

  • Refine by Keyword
  • 1 Algorithm Schedule
  • 1 Answer Set Programming
  • 1 Performance assessment
  • 1 Portfolio-based Solving
  • 1 algorithm configuration
  • Show More...

  • Refine by Type
  • 4 document

  • Refine by Publication Year
  • 1 2012
  • 1 2017
  • 1 2019
  • 1 2021

Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail