SafeAI-Kit: A Software Toolbox to Evaluate AI Systems with a Focus on Uncertainty Quantification (Practitioner Track)

Eisl, Dominik; Bernhardt, Bastian; Höhndorf, Lukas; Kulaga, Rafal

doi:10.4230/OASIcs.SAIA.2024.6

Subject Classification

ACM Subject Classification

Computing methodologies → Artificial intelligence

Keywords

safeAI-kit
Evaluation of AI Systems
Uncertainty Quantification

Metrics

Access Statistics
Total Accesses (updated on a weekly basis)

0

Document

0

Metadata

Abstract

In the course of the practitioner track, the IABG toolbox safeAI-kit is presented with a focus on uncertainty quantification in machine learning. The safeAI-kit consists of five sub-modules that provide analyses for performance, robustness, dataset, explainability, and uncertainty. The development of these sub-modules take ongoing standardization activities into account.

Cite As Get BibTex

Dominik Eisl, Bastian Bernhardt, Lukas Höhndorf, and Rafal Kulaga. SafeAI-Kit: A Software Toolbox to Evaluate AI Systems with a Focus on Uncertainty Quantification (Practitioner Track). In Symposium on Scaling AI Assessments (SAIA 2024). Open Access Series in Informatics (OASIcs), Volume 126, pp. 6:1-6:3, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/OASIcs.SAIA.2024.6

Author Details

Dominik Eisl

Industrieanlagen-Betriebsgesellschaft mbH, Ottobrunn, Germany

Bastian Bernhardt

Industrieanlagen-Betriebsgesellschaft mbH, Ottobrunn, Germany

Lukas Höhndorf

Industrieanlagen-Betriebsgesellschaft mbH, Ottobrunn, Germany

Rafal Kulaga

Industrieanlagen-Betriebsgesellschaft mbH, Ottobrunn, Germany

References

"DIN SPEC 92005:2024-03, Künstliche Intelligenz - Quantifizierung von Unsicherheiten im Maschinellen Lernen; Text Englisch". Technical report, DIN Media GmbH, Berlin, 2024. URL: https://www.dinmedia.de/en/technical-rule/din-spec-92005/376619718.
Minliang He, Xuming Wang, and Yijun Zhao. A calibrated deep learning ensemble for abnormality detection in musculoskeletal radiographs. Scientific Reports, 11, 2021. URL: https://api.semanticscholar.org/CorpusID:233427277.
The European Parliament and the Council of the European Union. "REGULATION (EU) 2024/1689 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 13 june 2024 laying down harmonised rules on artificial intelligence and amending regulations (ec) no 300/2008, (eu) no 167/2013, (eu) no 168/2013, (eu) 2018/858, (eu) 2018/1139 and (eu) 2019/2144 and directives 2014/90/eu, (eu) 2016/797 and (eu) 2020/1828 (Artificial Intelligence Act)", 2024. URL: https://eur-lex.europa.eu/eli/reg/2024/1689/oj.
Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data, 2024. https://arxiv.org/abs/2401.10891, URL: https://doi.org/10.48550/arXiv.2401.10891.

SafeAI-Kit: A Software Toolbox to Evaluate AI Systems with a Focus on Uncertainty Quantification (Practitioner Track)

Authors Dominik Eisl, Bastian Bernhardt, Lukas Höhndorf, Rafal Kulaga

Files

Document Identifiers

Subject Classification

ACM Subject Classification

Keywords

Metrics

Abstract

Cite As Get BibTex

Author Details

References

Thanks for your feedback!

Could not send message