Document Open Access Logo

Global Benchmark Database

Authors Markus Iser , Christoph Jabs

PDF
Thumbnail PDF

File

LIPIcs.SAT.2024.18.pdf
  • Filesize: 0.68 MB
  • 10 pages

Document Identifiers

Author Details

Markus Iser
  • Karlsruhe Institute of Technology, Germany
Christoph Jabs
  • HIIT, University of Helsinki, Finland

Funding

  • Jabs, Christoph: Work financially supported by Research Council of Finland grant 356046.

Acknowledgements

Many thanks to all the people who have supported GBD, either by contributing code or data, or by providing infrastructure. Special thanks also go to the early adopters whose successes and feedback have been and continue to be inspiring and motivating.

Cite AsGet BibTex

Markus Iser and Christoph Jabs. Global Benchmark Database. In 27th International Conference on Theory and Applications of Satisfiability Testing (SAT 2024). Leibniz International Proceedings in Informatics (LIPIcs), Volume 305, pp. 18:1-18:10, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2024)
https://doi.org/10.4230/LIPIcs.SAT.2024.18

Abstract

This paper presents Global Benchmark Database (GBD), a comprehensive suite of tools for provisioning and sustainably maintaining benchmark instances and their metadata. The availability of benchmark metadata is essential for many tasks in empirical research, e.g., for the data-driven compilation of benchmarks, the domain-specific analysis of runtime experiments, or the instance-specific selection of solvers. In this paper, we introduce the data model of GBD as well as its interfaces and provide examples of how to interact with them. We also demonstrate the integration of custom data sources and explain how to extend GBD with additional problem domains, instance formats and feature extractors.

Subject Classification

ACM Subject Classification
  • Theory of computation → Logic
  • Theory of computation → Design and analysis of algorithms
  • Information systems → Information integration
Keywords
  • Maintenance and Distribution of Benchmark Instances and their Features

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Supplementary Materials

References

  1. Fahiem Bacchus. MaxSAT Formulas Library, 2006. URL: http://www.cs.toronto.edu/maxsat-lib/maxsat-instances/.
  2. Jakob Bach, Markus Iser, and Klemens Böhm. A Comprehensive Study of k-Portfolios of Recent SAT Solvers. In Intl. Conf. on Theo. and Appl. of Satisf. Test., SAT, pages 1-18, 2022. URL: https://doi.org/10.4230/LIPIcs.SAT.2022.2.
  3. Adrian Balint, Daniel Gall, Gregor Kapler, and Robert Retz. Experiment Design and Administration for Computer Clusters for SAT Solvers, EDACC. J. Satisf. Bool. Model. Comput., pages 77-82, 2010. URL: https://doi.org/10.3233/SAT190078.
  4. Clark Barrett, Pascal Fontaine, and Cesare Tinelli. The Satisfiability Modulo Theories Library, SMT-LIB, 2016. URL: www.smt-lib.org.
  5. Bernd Bischl, Pascal Kerschke, Lars Kotthoff, et al. ASlib: A Benchmark Library for Algorithm Selection. Artif. Intel., pages 41-58, 2016. URL: https://doi.org/10.1016/j.artint.2016.04.003.
  6. Jan Elffers, Jesús Giráldez-Cru, Stephan Gocht, Jakob Nordström, and Laurent Simon. Seeking Practical CDCL Insights from Theoretical SAT Benchmarks. In Intl. Joint Conf. on Artif. Intel., IJCAI, pages 1300-1308, 2018. URL: https://doi.org/10.24963/ijcai.2018/181.
  7. Roy Thomas Fielding. REST: Architectural Styles and the Design of Network-based Software Architectures. PhD thesis, University of California, 2000. URL: http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm.
  8. Nils Froleyks, Marijn Heule, Markus Iser, Matti Järvisalo, and Martin Suda. SAT Competition 2020. Artif. Intel., 2021. URL: https://doi.org/10.1016/j.artint.2021.103572.
  9. E. Giunchiglia, M. Narizzano, L. Pulina, and A. Tacchella. Quantified Boolean Formulas Library, QBFLIB, 2005. URL: www.qbflib.org.
  10. Ambros Gleixner, Gregor Hendel, Gerald Gamrath, et al. MIPLIB 2017: Data-driven Compilation of the 6th Mixed-Integer Programming Library. Math. Program. Comput., pages 443-490, 2021. URL: https://doi.org/10.1007/s12532-020-00194-3.
  11. Andrew Haberlandt, Harrison Green, and Marijn J. H. Heule. Effective Auxiliary Variables via Structured Reencoding. In Intl. Conf. on Theo. and Appl. of Satisf. Test., SAT, pages 1-19, 2023. URL: https://doi.org/10.4230/LIPICS.SAT.2023.11.
  12. Holger Hoos and Thomas Stützle. SATLIB: An Online Resource for Research on SAT. SAT 2000, pages 283-292, 2000. URL: https://api.semanticscholar.org/CorpusID:17486963.
  13. Markus Iser. Benchmark Compilation for SAT Competition 2023. In Proc. of SAT Comp. 2023: Solver, Benchmark and Proof Checker Descriptions, 2023. URL: http://hdl.handle.net/10138/563824.
  14. Markus Iser and Christoph Jabs. GBD Data Repository. Dataset, version 1.0., swhId: https://archive.softwareheritage.org/swh:1:dir:1764eef19455a0bfdb6389a04a055977338304ff;origin=https://github.com/Udopia/gbd-data;visit=swh:1:snp:c94e9ddcf3108d147a440520e71697d5a7526095;anchor=swh:1:rev:609f64f7d8335d50aca9f4713afcce3e81bf09b2 (visited on 2024-08-02). URL: https://github.com/Udopia/gbd-data.
  15. Markus Iser and Christoph Jabs. GBD Evaluation Scripts. Software, version 1.0., swhId: https://archive.softwareheritage.org/swh:1:dir:159fae7ac54d9f678ee38ed882dae693cd5f14cf;origin=https://github.com/Udopia/gbdeval;visit=swh:1:snp:0ca8e899579d511367cf2981a5da9ff656f6577d;anchor=swh:1:rev:1db61d77d84d5e54ad5cedd66cd15f9334212875 (visited on 2024-08-02). URL: https://github.com/Udopia/gbdeval.
  16. Markus Iser and Christoph Jabs. Global Benchmark Database (Extension Module). Software, version 1.0., swhId: https://archive.softwareheritage.org/swh:1:dir:705258b1bd5c3415069fbd156759945a8ad725c0;origin=https://github.com/Udopia/gbdc;visit=swh:1:snp:23f6abd3b9d7389152ed319b2671f16df2071c42;anchor=swh:1:rev:7b6183a29cca2316e7b5b6801d9cc0c86b3ed404 (visited on 2024-08-02). URL: https://github.com/Udopia/gbdc.
  17. Markus Iser and Christoph Jabs. Udopia/gbd. Software, version 4.8.5., swhId: https://archive.softwareheritage.org/swh:1:dir:37fb54c7420fd71b6c4930bcf7e78d5129325a6d;origin=https://github.com/Udopia/gbd;visit=swh:1:snp:2ac08f4b21c50b3e1afa06cdf1c124f23032a3ee;anchor=swh:1:rev:d046347f76eab17038205b3485b0dec83d90860f (visited on 2024-08-02). URL: https://github.com/Udopia/gbd.
  18. Markus Iser and Christoph Jabs. GBD Tools, 2024. URL: https://doi.org/10.5281/zenodo.11093597.
  19. Markus Iser and Carsten Sinz. A Problem Metadata Library for Research in SAT. In Proceedings of Pragmatics of SAT, POS, pages 144-152, 2018. URL: https://doi.org/10.29007/gdbb.
  20. Wes McKinney et al. Data Structures for Statistical Computing in Python. In Python in Science, pages 51-56, 2010. URL: https://doi.org/10.25080/majora-92bf1922-00a.
  21. Dirk Merkel. Docker: lightweight linux containers for consistent development and deployment. Linux J., 2014. Google Scholar
  22. Sibylle Möhle and Norbert Manthey. Better Evaluations by Analyzing Benchmark Structure. In Pragmatics of SAT workshop, POS, 2016. URL: http://www.pragmaticsofsat.org/2016/reg/POS-16_paper_4.pdf.
  23. Christos H. Papadimitriou. Computational Complexity. Addison-Wesley, 1994. Google Scholar
  24. Will Reese. Nginx: the high-performance web server and reverse proxy. Linux J., 2008. Google Scholar
  25. Dominik Pascal Schreiber. Scalable SAT Solving and its Application. PhD thesis, Karlsruhe Institute of Technology, KIT, 2023. URL: https://doi.org/10.5445/IR/1000165224.
  26. Laurent Simon and Philippe Chatalic. SatEx: A Web-based Framework for SAT Experimentation. Electron. Notes Discrete Math., pages 129-149, 2001. URL: https://doi.org/10.1016/S1571-0653(04)00318-X.
  27. Lin Xu, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. SATzilla: Portfolio-based Algorithm Selection for SAT. J. Artif. Intel. Res., JAIR, pages 565-606, 2008. URL: https://doi.org/10.5555/1622673.1622687.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact