Document Open Access Logo

SafeAI-Kit: A Software Toolbox to Evaluate AI Systems with a Focus on Uncertainty Quantification (Practitioner Track)

Authors Dominik Eisl, Bastian Bernhardt, Lukas Höhndorf, Rafal Kulaga

PDF

File

Thumbnail PDF
PDF
OASIcs.SAIA.2024.6.pdf
  • Filesize: 301 kB
  • 3 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Computing methodologies → Artificial intelligence
Keywords
  • safeAI-kit
  • Evaluation of AI Systems
  • Uncertainty Quantification

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

In the course of the practitioner track, the IABG toolbox safeAI-kit is presented with a focus on uncertainty quantification in machine learning. The safeAI-kit consists of five sub-modules that provide analyses for performance, robustness, dataset, explainability, and uncertainty. The development of these sub-modules take ongoing standardization activities into account.

Cite As Get BibTex

Dominik Eisl, Bastian Bernhardt, Lukas Höhndorf, and Rafal Kulaga. SafeAI-Kit: A Software Toolbox to Evaluate AI Systems with a Focus on Uncertainty Quantification (Practitioner Track). In Symposium on Scaling AI Assessments (SAIA 2024). Open Access Series in Informatics (OASIcs), Volume 126, pp. 6:1-6:3, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/OASIcs.SAIA.2024.6

Author Details

Dominik Eisl
  • Industrieanlagen-Betriebsgesellschaft mbH, Ottobrunn, Germany
Bastian Bernhardt
  • Industrieanlagen-Betriebsgesellschaft mbH, Ottobrunn, Germany
Lukas Höhndorf
  • Industrieanlagen-Betriebsgesellschaft mbH, Ottobrunn, Germany
Rafal Kulaga
  • Industrieanlagen-Betriebsgesellschaft mbH, Ottobrunn, Germany

References

  1. "DIN SPEC 92005:2024-03, Künstliche Intelligenz - Quantifizierung von Unsicherheiten im Maschinellen Lernen; Text Englisch". Technical report, DIN Media GmbH, Berlin, 2024. URL: https://www.dinmedia.de/en/technical-rule/din-spec-92005/376619718.
  2. Minliang He, Xuming Wang, and Yijun Zhao. A calibrated deep learning ensemble for abnormality detection in musculoskeletal radiographs. Scientific Reports, 11, 2021. URL: https://api.semanticscholar.org/CorpusID:233427277.
  3. The European Parliament and the Council of the European Union. "REGULATION (EU) 2024/1689 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 13 june 2024 laying down harmonised rules on artificial intelligence and amending regulations (ec) no 300/2008, (eu) no 167/2013, (eu) no 168/2013, (eu) 2018/858, (eu) 2018/1139 and (eu) 2019/2144 and directives 2014/90/eu, (eu) 2016/797 and (eu) 2020/1828 (Artificial Intelligence Act)", 2024. URL: https://eur-lex.europa.eu/eli/reg/2024/1689/oj.
  4. Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data, 2024. https://arxiv.org/abs/2401.10891, URL: https://doi.org/10.48550/arXiv.2401.10891.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact