2026-07-21T14:08:23Z https://drops.dagstuhl.de/oai

oai:drops-oai.dagstuhl.de:22099 2024-11-26T15:16:36Z ddc:004 open_access

Design Principles for Falsifiable, Replicable and Reproducible Empirical Machine Learning Research Vranješ, Daniel Ehrhardt, Jonas Heesch, René Moddemann, Lukas Steude, Henrik Sebastian Niggemann, Oliver machine learning hypothesis design research design experimental research statistical testing diagnosis planning Machine learning is becoming increasingly important in the diagnosis and planning fields, where data-driven models and algorithms are being employed as alternatives to traditional first-principle approaches. Empirical research plays a fundamental role in the machine learning domain. At the heart of impactful empirical research lies the development of clear research hypotheses, which then shape the design of experiments. The execution of experiments must be carried out with precision to ensure reliable results, followed by statistical analysis to interpret these outcomes. This process is key to either supporting or refuting initial hypotheses. Despite its importance, there is a high variability in research practices across the machine learning community and no uniform understanding of quality criteria for empirical research. To address this gap, we propose a model for the empirical research process, accompanied by guidelines to uphold the validity of empirical research. By embracing these recommendations, greater consistency, enhanced reliability and increased impact can be achieved. Schloss Dagstuhl – Leibniz-Zentrum für Informatik Daniel Vranješ and Jonas Ehrhardt and René Heesch and Lukas Moddemann and Henrik Sebastian Steude and Oliver Niggemann 2024 Is Part Of OASIcs, Volume 125, 35th International Conference on Principles of Diagnosis and Resilient Systems (DX 2024) InProceedings Text doc-type:ResearchArticle publishedVersion application/pdf doi:10.4230/OASIcs.DX.2024.7 urn:nbn:de:0030-drops-220991 https://drops.dagstuhl.de/entities/document/10.4230/OASIcs.DX.2024.7 eng https://creativecommons.org/licenses/by/4.0/legalcode