Document Open Access Logo

Complications for Computational Experiments from Modern Processors

Authors Johannes K. Fichte , Markus Hecher , Ciaran McCreesh, Anas Shahab

PDF
Thumbnail PDF

File

LIPIcs.CP.2021.25.pdf
  • Filesize: 1.1 MB
  • 21 pages

Document Identifiers

Author Details

Johannes K. Fichte
  • TU Dresden, Germany
Markus Hecher
  • TU Wien, Austria
  • Universität Potsdam, Germany
Ciaran McCreesh
  • University of Glasgow, UK
Anas Shahab
  • TU Dresden, Germany

Funding

  • Fichte, Johannes K.: Google Fellowship at the Simons Institute, UC Berkeley.
  • Hecher, Markus: FWF Grants Y698 and P32830 and Grant WWTF ICT19-065.

Acknowledgements

Authors are given in alphabetical order. The work has been carried out while the first three authors were visiting the Simons Institute for the Theory of Computing.

Cite As Get BibTex

Johannes K. Fichte, Markus Hecher, Ciaran McCreesh, and Anas Shahab. Complications for Computational Experiments from Modern Processors. In 27th International Conference on Principles and Practice of Constraint Programming (CP 2021). Leibniz International Proceedings in Informatics (LIPIcs), Volume 210, pp. 25:1-25:21, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2021) https://doi.org/10.4230/LIPIcs.CP.2021.25

Abstract

In this paper, we revisit the approach to empirical experiments for combinatorial solvers. We provide a brief survey on tools that can help to make empirical work easier. We illustrate origins of uncertainty in modern hardware and show how strong the influence of certain aspects of modern hardware and its experimental setup can be in an actual experimental evaluation. More specifically, there can be situations where (i) two different researchers run a reasonable-looking experiment comparing the same solvers and come to different conclusions and (ii) one researcher runs the same experiment twice on the same hardware and reaches different conclusions based upon how the hardware is configured and used. We investigate these situations from a hardware perspective. Furthermore, we provide an overview on standard measures, detailed explanations on effects, potential errors, and biased suggestions for useful tools. Alongside the tools, we discuss their feasibility as experiments often run on clusters to which the experimentalist has only limited access. Our work sheds light on a number of benchmarking-related issues which could be considered to be folklore or even myths.

Subject Classification

ACM Subject Classification
  • Computer systems organization → Multicore architectures
  • Hardware → Temperature monitoring
  • Hardware → Impact on the environment
  • Hardware → Platform power issues
  • Theory of computation → Design and analysis of algorithms
Keywords
  • Experimenting
  • Combinatorial Solving
  • Empirical Work

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Supplementary Materials

References

  1. Altuna Akalin. Scientific data analysis pipelines and reproducibility. https://towardsdatascience.com/scientific-data-analysis-pipelines-and-reproducibility-75ff9df5b4c5, 2018. Retrieved 2021-05-29.
  2. Chris Angelini. Intel’s second-gen core CPUs: The Sandy bridge review. https://www.tomshardware.com/reviews/sandy-bridge-core-i7-2600k-core-i5-2500k,2833-3.html, 2011.
  3. Gilles Audemard, Loïc Paulevé, and Laurent Simon. SAT heritage: A community-driven effort for archiving, building and running more than thousand sat solvers. In Luca Pulina and Martina Seidl, editors, Proceedings of the 23rd International Conference on Theory and Applications of Satisfiability Testing (SAT'20), volume 12178 of Lecture Notes in Computer Science, pages 107-113, Alghero, Italy, July 2020. Springer Verlag. Google Scholar
  4. David F. Bacon, Susan L. Graham, and Oliver J. Sharp. Compiler transformations for high-performance computing. ACM Comput. Surv., 26(4):345-420, 1994. URL: https://doi.org/10.1145/197405.197406.
  5. Daniel Le Berre, Matti Järvisalo, Armin Biere, and Kuldeep S. Meel. The SAT practitioner’s manifesto v1.0. https://github.com/danielleberre/satpractitionermanisfesto, 2020.
  6. Donald Berry. Avoiding clock drift on VMs. https://www.redhat.com/en/blog/avoiding-clock-drift-vms, 2017.
  7. Dirk Beyer, Stefan Löwe, and Philipp Wendler. Benchmarking and resource measurement. In Bernd Fischer and Jaco Geldenhuys, editors, Proceedings of the 22nd International Symposium on Model Checking of Software (SPIN'15), volume 9232 of Lecture Notes in Computer Science, pages 160-178. Springer Verlag, 2015. URL: https://doi.org/10.1007/978-3-319-23404-5_12.
  8. Dirk Beyer, Stefan Löwe, and Philipp Wendler. Reliable benchmarking: requirements and solutions. International Journal on Software Tools for Technology Transfer, 21(1):1-29, 2019. URL: https://doi.org/10.1007/s10009-017-0469-y.
  9. Steve Blank. What the GlobalFoundries' retreat really means. IEEE Spectrum, 2018. URL: https://spectrum.ieee.org/nanoclast/semiconductors/devices/what-globalfoundries-retreat-really-means.
  10. Terrehon Bowden, Bodo Bauer, Jorge Nerin, Shen Feng, and Stefani Seibold. The /proc filesystem. https://www.kernel.org/doc/Documentation/filesystems/proc.txt, 2009.
  11. Padraig Brady. Timeout(1). https://www.gnu.org/software/coreutils/timeout, 2019.
  12. Dominik Brodowski, Nico Golde, Rafael J. Wysocki, and Viresh Kumar. CPU frequency and voltage scaling code in the Linux(tm) kernel. https://www.kernel.org/doc/Documentation/cpu-freq/governors.txt, 2016.
  13. Davidlohr Bueso. Util-linux 2.37. https://www.kernel.org/pub/linux/utils/util-linux/, 2021.
  14. Alejandro Colomar et al. Ulimit(3). https://www.kernel.org/doc/man-pages/, 2017.
  15. Roberto Di Cosmo, Morane Gruenpeter, and Stefano Zacchiroli. Identifiers for digital objects: the case of software source code preservation. In iPRES 2018 - 15th International Conference on Digital Preservation, 2018. URL: https://doi.org/10.17605/OSF.IO/KDE56.
  16. Roberto Di Cosmo and Stefano Zacchiroli. Software heritage: Why and how to preserve software source code. In iPRES 2017: 14th International Conference on Digital Preservation, Kyoto, Japan, 2017. URL: https://hal.archives-ouvertes.fr/hal-01590958.
  17. Roberto Di Cosmo, Stefano Zacchiroli, Gérard Berry, Jean-François Abramatic, Julia Lawall, and Serge Abiteboul. Software heritage. https://www.softwareheritage.org/mission/heritage/, 2020.
  18. Chris Drummond. Replicability is not reproducibility: Nor is it good science. In Proceedings of the Evaluation Methods for Machine Learning Workshop at the 26th ICML, Montreal, Canada, 2009. Google Scholar
  19. Jan Elffers and Jakob Nordström. Divide and conquer: Towards faster pseudo-boolean solving. In Jérôme Lang, editor, Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pages 1291-1299. International Joint Conferences on Artificial Intelligence Organization, July 2018. URL: https://doi.org/10.24963/ijcai.2018/180.
  20. Johannes K. Fichte and Markus Hecher. The model counting competition 2021. https://mccompetition.org/past_iterations, 2021.
  21. Johannes K. Fichte, Markus Hecher, and Florim Hamiti. The model counting competition 2020. ACM Journal of Experimental Algorithmics, 2021. In press. Google Scholar
  22. Johannes K. Fichte, Markus Hecher, and Stefan Szeider. A time leap challenge for SAT-solving. In Helmut Simonis, editor, Proceedings of the 26th International Conference on Principles and Practice of Constraint Programming (CP'20), pages 267-285, Louvain-la-Neuve, Belgium, September 2020. Springer Verlag. URL: https://doi.org/10.1007/978-3-030-58475-7_16.
  23. Johannes K. Fichte, Markus Hecher, Stefan Woltran, and Markus Zisser. Weighted model counting on the GPU by exploiting small treewidth. In Yossi Azar, Hannah Bast, and Grzegorz Herman, editors, Proceedings of the 26th Annual European Symposium on Algorithms (ESA'18), volume 112 of Leibniz International Proceedings in Informatics (LIPIcs), pages 28:1-28:16. Dagstuhl Publishing, 2018. URL: https://doi.org/10.4230/LIPIcs.ESA.2018.28.
  24. Johannes K. Fichte, Norbert Manthey, André Schidler, and Julian Stecklina. Towards faster reasoners by using transparent huge pages. In Helmut Simonis, editor, Proceedings of the 26th International Conference on Principles and Practice of Constraint Programming (CP'20), volume 12333 of Lecture Notes in Computer Science, pages 304-322, Louvain-la-Neuve, Belgium, September 2020. Springer Verlag. URL: https://doi.org/10.1007/978-3-030-58475-7_18.
  25. Ronald Fisher. Arrangement of field experiments. Journal of the Ministry of Agriculture of Great Britain, pages 503-513, 1926. Google Scholar
  26. Ronald Fisher. The Design of Experiments. Oliver and Boyd, 1935. Google Scholar
  27. Vijay Ganesh and Moshe Y. Vardi. On the unreasonable effectiveness of sat solvers, 2021. Google Scholar
  28. Ian P. Gent. The recomputation manifesto. CoRR, abs/1304.3674, 2013. URL: http://arxiv.org/abs/1304.3674.
  29. Ian P Gent and Toby Walsh. The SAT phase transition. In Anthony G. Cohn, editor, Proceedings of the 11th European Conference on Artificial Intelligence (ECAI'94), volume 94, pages 105-109, Amsterdam, The Netherlands, 1994. PITMAN. Google Scholar
  30. Stephan Gocht, Jakob Nordström, and Amir Yehudayoff. On division versus saturation in pseudo-boolean solving. In Sarit Kraus, editor, Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI'19), pages 1711-1718. International Joint Conferences on Artificial Intelligence Organization, July 2019. URL: https://doi.org/10.24963/ijcai.2019/237.
  31. Odd Erik Gundersen. The reproducibility crisis is real. AI Magazine, 2020. URL: https://doi.org/10.1609/aimag.v41i3.5318.
  32. Odd Erik Gundersen. AAAI'21 reproducibility checklist. https://aaai.org/Conferences/AAAI-21/reproducibility-checklist/, 2021.
  33. Odd Erik Gundersen and Sigbjørn Kjensmo. State of the art: Reproducibility in artificial intelligence. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI'18), 2018. Google Scholar
  34. Daniel Hackenberg, Robert Schöne, Thomas Ilsche, Daniel Molka, Joseph Schuchart, and Robin Geyer. An energy efficiency feature survey of the Intel Haswell processor. In Jean-Francois Lalande and Teng Moh, editors, Proceedings of the 17th International Conference on High Performance Computing & Simulation (HPCS'19), 2019. Google Scholar
  35. Youssef Hamadi and Christoph M. Wintersteiger. Seven challenges in parallel SAT solving. In Jörg Hoffmann and Bart Selman, editors, Proceedings of the 26th AAAI Conference on Artificial Intelligence (AAAI'12), pages 2120-2125, Toronto, Ontario, Canada, 2012. The AAAI Press. Google Scholar
  36. Tejun Heo. Control group v2. https://www.kernel.org/doc/Documentation/cgroup-v1/cgroups.txt, 2015.
  37. Christopher Hollowell, Costin Caramarcu, William Strecker-Kellogg, Antonio Wong, and Alexandr Zaytsev. The effect of numa tunings on cpu performance. Journal of Physics: Conference Series, 2015. Google Scholar
  38. Christopher Hollowell, Costin Caramarcu, William Strecker-Kellogg, Antonio Wong, and Alexandr Zaytsev. The effect of NUMA tunings on CPU performance. Journal of Physics: Conference Series, 664(9):092010, December 2015. URL: https://doi.org/10.1088/1742-6596/664/9/092010.
  39. John N. Hooker. Testing heuristics: We have it all wrong. Journal of Heuristics, 1:33-42, 1995. URL: https://doi.org/10.1007/BF02430364.
  40. Holger H. Hoos, Benjamin Kaufmann, Torsten Schaub, and Marius Schneider. Robust benchmark set selection for Boolean constraint solvers. In Proceedings of the 7th International Conference on Learning and Intelligent Optimization (LION'13), volume 7997 of Lecture Notes in Computer Science, pages 138-152, Catania, Italy, January 2013. Springer Verlag. Revised Selected Papers. Google Scholar
  41. Neil Horman, PJ Waskiewicz, and Anton Arapov. Irqbalance. http://irqbalance.github.io/irqbalance/, 2020.
  42. Sascha Hunold, Alexandra Carpen-Amarie, and Jesper Larsson Träff. Reproducible MPI micro-benchmarking isn't as easy as you think. In Proceedings of the 21st European MPI Users' Group Meeting, EuroMPI/ASIA '14, pages 69-76, New York, NY, USA, 2014. Association for Computing Machinery, New York. URL: https://doi.org/10.1145/2642769.2642785.
  43. Satoshi Imamura, Keitaro Oka, Yuichiro Yasui, Yuichi Inadomi, Katsuki Fujisawa, Toshio Endo, Koji Ueno, Keiichiro Fukazawa, Nozomi Hata, Yuta Kakibuka, Koji Inoue, and Takatsugu Ono. Evaluating the impacts of code-level performance tunings on power efficiency. In 2016 IEEE International Conference on Big Data (Big Data), pages 362-369, 2016. URL: https://doi.org/10.1109/BigData.2016.7840624.
  44. VmWare Inc. Timekeeping in VMware virtual machines. http://www.vmware.com/pdf/vmware_timekeeping.pdf, 2008.
  45. Intel. Intel product specifications: Processors. https://ark.intel.com/content/www/us/en/ark.html#@Processors, 2021.
  46. Paul Jackson and Christoph Lameter. cgroups - Linux control groups. https://www.kernel.org/doc/Documentation/cgroup-v1/cgroups.txt, 2006.
  47. Matti Järvisalo, Daniel Le Berre, Olivier Roussel, and Laurent Simon. The international SAT solver competitions. AI Magazin, 33(1), 2012. URL: http://www.aaai.org/ojs/index.php/aimagazine/article/view/2395, URL: https://doi.org/10.1609/aimag.v33i1.2395.
  48. D. A. Jimenez and C. Lin. Dynamic branch prediction with perceptrons. In Proceedings of 7th International Symposium on High-Performance Computer Architecture (HPCA'01), pages 197-206, 2001. Google Scholar
  49. Armin Biere Katalin Fazekas, Daniela Kaufmann. Ranking robustness under sub-sampling for the SAT competition 2018. In Matti Järvisalo and Daniel Le Berre, editors, Proceedings of the 10th Workshop on Pragmatics of SAT (POS'19), 2019. Google Scholar
  50. Michael Kerrisk. The Linux Programming Interface. No Starch Press, 2010. URL: https://man7.org/tlpi/.
  51. Andi Kleen, Cliff Wickman, Christoph Lameter, and Lee Schermerhorn. numactl. https://github.com/numactl/numactl, 2014.
  52. Donald E. Knuth. The Art of Computer Programming, Volume 4, Fascicle 6: Satisfiability. Addison-Wesley, 2015. Google Scholar
  53. Paul Kocher, Jann Horn, Anders Fogh, , Daniel Genkin, Daniel Gruss, Werner Haas, Mike Hamburg, Moritz Lipp, Stefan Mangard, Thomas Prescher, Michael Schwarz, and Yuval Yarom. Spectre attacks: Exploiting speculative execution. In Proceedings of the 40th IEEE Symposium on Security and Privacy (S&P'19), 2019. Google Scholar
  54. Michael Kohlhase. The theorem prover museum - conserving the system heritage of automated reasoning. CoRR, abs/1904.10414, 2019. URL: http://arxiv.org/abs/1904.10414.
  55. Divya Kiran Kumar. Installing and using Perf in Ubuntu and CentOS. https://www.fosslinux.com/7069/installing-and-using-perf-in-ubuntu-and-centos.htm, 2019.
  56. G. M. Kurtzer, V. Sochat, and Michael W Bauer. Singularity: Scientific containers for mobility of compute. PLoS ONE, 12, 2017. URL: https://doi.org/10.1371/journal.pone.0177459.
  57. Christoph Lameter. Numa (non-uniform memory access): An overview: Numa becomes more common because memory controllers get close to execution units on microprocessors. Queue, 11(7):40-51, 2013. Google Scholar
  58. Moritz Lipp, Michael Schwarz, Daniel Gruss, Thomas Prescher, Werner Haas, Anders Fogh, Jann Horn, Stefan Mangard, Paul Kocher, Daniel Genkin, Yuval Yarom, and Mike Hamburg. Meltdown: Reading kernel memory from user space. In 27th USENIX Security Symposium (USENIX Security 18), 2018. Google Scholar
  59. Zhiyuan Liu and Jian Tang. IJCAI 2021 reproducibility guidelines. http://ijcai-21.org/wp-content/uploads/2020/12/20201226-IJCAI-Reproducibility.pdf, 2020.
  60. Eric Ma. How to statically link C and C++ programs on Linux with gcc. https://www.systutorials.com/how-to-statically-link-c-and-c-programs-on-linux-with-gcc/, 2020. Accessed 20-April-2021.
  61. Kamil Maciorowski and meuh. Run perf without root-rights. https://superuser.com/questions/980632/run-perf-without-root-rights, 2015.
  62. Ruben Martins, Vasco Manquinho, and Inês Lynce. An overview of parallel sat solving. Constraints, 17(3):304-347, 2012. URL: https://doi.org/10.1007/s10601-012-9121-3.
  63. Catherine C. McGeoch. A Guide to Experimental Algorithmics. Cambridge University Press, 2012. Google Scholar
  64. Doug McIlroy. Unix time-sharing system: Foreword. The Bell System Technical Journal, pages 1902-1903, 1978. Google Scholar
  65. Marcin Miłkowski, Witold M. Hensel, and Mateusz Hohol. Replicability or reproducibility? on the replication crisis in computational neuroscience and sharing only relevant detail. Journal of Computational Neuroscience, 2018. Google Scholar
  66. Daniel Molka, Daniel Hackenberg, Robert Schone, and Matthias S. Muller. Memory performance and cache coherency effects on an Intel Nehalem multiprocessor system. In 2009 18th International Conference on Parallel Architectures and Compilation Techniques, pages 261-270, 2009. URL: https://doi.org/10.1109/PACT.2009.22.
  67. Ingo Molnar and Max Krasnyansky. Smp irq affinity. https://www.kernel.org/doc/Documentation/IRQ-affinity.txt, 2012.
  68. Matteo Monchiero and Gianluca Palermo. The combined perceptron branch predictor. In José C. Cunha and Pedro D. Medeiros, editors, Euro-Par 2005 Parallel Processing, pages 487-496, Berlin, Heidelberg, 2005. Springer Berlin Heidelberg. Google Scholar
  69. Gordon E. Moore. Cramming more components onto integrated circuits. Electronics, 38(8), April 1965. Google Scholar
  70. Hassan Mujtaba. Intel Xeon E7 'Ivy Bridge-EX' lineup detailed - Xeon E7-8890 V2 'Ivy Town' chip with 15 cores and 37.5 mb LLC. https://Wccftech.com, 2014.
  71. Matthias Müller-Hannemann and Stefan Schirra, editors. Algorithm Engineering, volume 5971 of Theoretical Computer Science and General Issues. Springer Verlag, 2010. Bridging the Gap Between Algorithm Theory and Practice. URL: https://doi.org/10.1007/978-3-642-14866-8.
  72. Steven J. Murdoch. Hot or not: Revealing hidden services by their clock skew. In Proceedings of the 13th ACM Conference on Computer and Communications Security, CCS '06, pages 27-36, New York, NY, USA, 2006. Association for Computing Machinery. URL: https://doi.org/10.1145/1180405.1180410.
  73. Gonzalo Navarro. RCR: Replicated computational results initiative. https://dl.acm.org/journal/jea/rcr-initiative, 2021.
  74. Naomi Oreskes. Why Trust Science? The University Center for Human Values Series, 2021. Google Scholar
  75. Jeffrey M. Perkel. Challenge to scientists: does your ten-year-old code still run? Nature, 584:656-658, 2020. URL: https://doi.org/10.1038/d41586-020-02462-7.
  76. Red Had Developers. Optimizing Red Hat Enterprise Linux performance by tuning IRQ affinity. https://access.redhat.com/articles/216733, 2011.
  77. Benoit Rostykus and Gabriel Hartmann. Predictive CPU isolation of containers at Netflix. https://netflixtechblog.com/predictive-cpu-isolation-of-containers-at-netflix-91f014d856c7, 2019. accessed 20-May-2021.
  78. David Rotman. We're not prepared for the end of moore’s law. MIT Technology Review, 2020. Google Scholar
  79. Olivier Roussel. Controlling a solver execution with the runsolver tool. J. on Satisfiability, Boolean Modeling and Computation, 7:139-144, 2011. Google Scholar
  80. Olivier Roussel and Christophe Lecoutre. XCSP3 competition 2019. http://xcsp.org/competition, July 2019. A joint event with the 25th International Conference on Principles and Practice of Constraint Programming (CP'19).
  81. Marta Serra-Garcia and Uri Gneezy. Nonreplicable publications are cited more than replicable ones. Science Advances, 7(21), 2021. URL: https://doi.org/10.1126/sciadv.abd1705.
  82. Laurent Simon, Daniel Le Berre, and Edward A. Hirsch. The SAT2002 competition (preliminary draft). http://www.satcompetition.org/2002/onlinereport.pdf, 2002.
  83. Helmut Simonis, George Katsirelos, Matt Streeter, and Emmanuel Hebrard. CSP 2009 competition (CSC'2009). http://www.cril.univ-artois.fr/CSC09/, July 2009.
  84. Aaron Stump, Geoff Sutcliffe, and Cesare Tinelli. StarExec: A cross-community infrastructure for logic solving. In Stéphane Demri, Deepak Kapur, and Christoph Weidenbach, editors, Proceedings of the 7th International Joint Conference on Automated Reasoning (IJCAR'14), volume 8562 of Lecture Notes in Computer Science, pages 367-373, Vienna, Austria, July 2014. Springer Verlag. Held as Part of the Vienna Summer of Logic, VSL 2014. URL: https://doi.org/10.1007/978-3-319-08587-6_28.
  85. SUSE Support. Clock drifts in KVM virtual machines. https://www.suse.com/support/kb/doc/?id=000017652, 2020.
  86. Guido Tack and Peter J. Stuckey. The MiniZinc challenge 2019. https://www.minizinc.org/challenge.html, 2019. A joint event with the 25th International Conference on Principles and Practice of Constraint Programming (CP'19).
  87. Andrew S. Tanenbaum and James R. Goodman. Structured Computer Organization. Prentice Hall PTR, USA, 4th edition, 1998. Google Scholar
  88. The Open Group. The open group base specifications issue 7, 2018 edition ieee std 1003.1-2017 (revision of ieee std 1003.1-2008). https://pubs.opengroup.org/onlinepubs/9699919799/functions/ulimit.html, 2018.
  89. Linus Torvalds. LKML archive on lore.kernel.org: Very slow clang kernel config. https://lore.kernel.org/lkml/CAHk-=whs8QZf3YnifdLv57+FhBi5_WeNTG1B-suOES=RcUSmQg@mail.gmail.com/, 2021.
  90. A. M. Turing. On computable numbers, with an application to the Entscheidungsproblem. Proceedings of the London Mathematical Society, s2-42(1):230-265, 1937. URL: https://doi.org/10.1112/plms/s2-42.1.230.
  91. Moshe Y. Vardi. Boolean satisfiability: Theory and engineering. Communications of the ACM, 57(3):5, 2014. URL: https://doi.org/10.1145/2578043.
  92. Felix von Leitner. Source code optimization. Linux Congress 2009. http://www.linux-kongress.org/2009/slides/compiler_survey_felix_von_leitner.pdf, 2009.
  93. J. von Neumann. First draft of a report on the EDVAC. IEEE Annals of the History of Computing, 15(4):27-75, 1993. URL: https://doi.org/10.1109/85.238389.
  94. Szymon Wasik, Maciej Antczak, Jan Badura, Artur Laskowski, and Tomasz Sternal. Cscw '16 companion. In Eric Gilbert and Karrie Karahalios, editors, Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion, Association for Computing Machinery, New York, pages 433-436, New York, NY, USA, Optil.Io: Cloud Based Platform For Solving Optimization Problems Using Crowdsourcing Approach. acm. URL: https://doi.org/10.1145/2818052.2869098.
  95. Ulrich Windl, David Dalton, Marc Martinec, and Dale R. Worley. The NTP FAQ and HOWTO: Understanding and using the network time protocol. http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm#AEN1220, 2006.
  96. Ricardo Wurmus, Bora Uyar, Brendan Osberg, Vedran Franke, Alexander Gosdschan, Katarzyna Wreczycka, Jonathan Ronen, and Altuna Akalin. PiGx: reproducible genomics analysis pipelines with GNU Guix. GigaScience, 7(12), October 2018. giy123. URL: https://doi.org/10.1093/gigascience/giy123.
  97. Rafael J. Wysocki. intel_pstate CPU performance scaling driver. https://www.kernel.org/doc/html/v5.12/admin-guide/pm/intel_pstate.html, 2017.
  98. Rengan Xu, Frank Han, and Nishanth Dandapanthula. Containerizing HPC applications with singularity. https://downloads.dell.com/manuals/all-products/esuprt_solutions_int/esuprt_solutions_int_solutions_resources/high-computing-solution-resources_white-papers10_en-us.pdf, 2017.
  99. Andy B. Yoo, Morris A. Jette, and Mark Grondona. Slurm: Simple Linux utility for resource management. In Dror Feitelson, Larry Rudolph, and Uwe Schwiegelshohn, editors, 9th International Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP'03), volume 2862 of Lecture Notes in Computer Science, pages 44-60, Seattle, WA, USA, 2003. Springer Verlag. Google Scholar
  100. Zhenyun Zhuang, Cuong Tran, and Jerry Weng. Don't let Linux control groups run uncontrolled: Addressing memory-related performance pitfalls of cgroups. https://engineering.linkedin.com/blog/2016/08/don_t-let-linux-control-groups-uncontrolled, 2016. Accessed Apr-28-2020.
  101. Peter Zijlstra, Ingo Molnar, and Arnaldo Carvalho de Melo. Performance events subsystem. https://github.com/torvalds/linux/tree/master/tools/perf, 2009.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact