AI Assessment in Practice: Implementing a Certification Scheme for AI Trustworthiness (Academic Track)

Authors Carmen Frischknecht-Gruber , Philipp Denzel , Monika Reif , Yann Billeter , Stefan Brunner, Oliver Forster , Frank-Peter Schilling , Joanna Weng , Ricardo Chavarriaga



PDF
Thumbnail PDF

File

OASIcs.SAIA.2024.15.pdf
  • Filesize: 2.15 MB
  • 18 pages

Document Identifiers

Author Details

Carmen Frischknecht-Gruber
  • Zurich University of Applied Sciences ZHAW, Winterthur, Switzerland
Philipp Denzel
  • Zurich University of Applied Sciences ZHAW, Winterthur, Switzerland
Monika Reif
  • Zurich University of Applied Sciences ZHAW, Winterthur, Switzerland
Yann Billeter
  • Zurich University of Applied Sciences ZHAW, Winterthur, Switzerland
Stefan Brunner
  • Zurich University of Applied Sciences ZHAW, Winterthur, Switzerland
Oliver Forster
  • Zurich University of Applied Sciences ZHAW, Winterthur, Switzerland
Frank-Peter Schilling
  • Zurich University of Applied Sciences ZHAW, Winterthur, Switzerland
Joanna Weng
  • Zurich University of Applied Sciences ZHAW, Winterthur, Switzerland
Ricardo Chavarriaga
  • Zurich University of Applied Sciences ZHAW, Winterthur, Switzerland

Acknowledgements

We would like to acknowledge the support and collaboration of CertX AG in the development of the certification scheme discussed in this paper.

Cite As Get BibTex

Carmen Frischknecht-Gruber, Philipp Denzel, Monika Reif, Yann Billeter, Stefan Brunner, Oliver Forster, Frank-Peter Schilling, Joanna Weng, and Ricardo Chavarriaga. AI Assessment in Practice: Implementing a Certification Scheme for AI Trustworthiness (Academic Track). In Symposium on Scaling AI Assessments (SAIA 2024). Open Access Series in Informatics (OASIcs), Volume 126, pp. 15:1-15:18, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/OASIcs.SAIA.2024.15

Abstract

The trustworthiness of artificial intelligence systems is crucial for their widespread adoption and for avoiding negative impacts on society and the environment. This paper focuses on implementing a comprehensive certification scheme developed through a collaborative academic-industry project. The scheme provides practical guidelines for assessing and certifying the trustworthiness of AI-based systems. The implementation of the scheme leverages aspects from Machine Learning Operations and the requirements management tool Jira to ensure continuous compliance and efficient lifecycle management. The integration of various high-level frameworks, scientific methods, and metrics supports the systematic evaluation of key aspects of trustworthiness, such as reliability, transparency, safety and security, and human oversight. These methods and metrics were tested and assessed on real-world use cases to dependably verify means of compliance with regulatory requirements and evaluate criteria and detailed objectives for each of these key aspects. Thus, this certification framework bridges the gap between ethical guidelines and practical application, ensuring the safe and effective deployment of AI technologies.

Subject Classification

ACM Subject Classification
  • Computing methodologies → Artificial intelligence
  • Social and professional topics → Computing / technology policy
  • Information systems → Information systems applications
Keywords
  • AI Assessment
  • Certification Scheme
  • Artificial Intelligence
  • Trustworthiness of AI systems
  • AI Standards
  • AI Safety

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads

References

  1. IEEE 7001-2021 - IEEE Standard for Transparency of Autonomous Systems. Technical report, Institute of Electrical and Electronics Engineers, 2021. URL: https://standards.ieee.org/standard/7001-2021.html.
  2. Apache Software Foundation. Airflow. URL: https://airflow.apache.org/.
  3. Atlassian. Jira, 2002. URL: https://www.atlassian.com/software/jira.
  4. Atlassian. Confluence, 2004. URL: https://www.atlassian.com/software/confluence.
  5. Richard Benjamins, Alberto Barbado, and Daniel Sierra. Responsible AI by design in practice. arXiv preprint arXiv:1909.12838, 2019. Google Scholar
  6. Yann Billeter, Philipp Denzel, Ricardo Chavarriaga, Oliver Forster, Frank-Peter Schilling, Stefan Brunner, Carmen Frischknecht-Gruber, Monika Ulrike Reif, and Joanna Weng. MLOps as enabler of trustworthy AI. In 11th IEEE Swiss Conference on Data Science (SDS), Zurich, Switzerland, 30-31 May 2024, 2024. URL: https://doi.org/10.21256/zhaw-30443.
  7. Jace Browning and Robert Adams. Doorstop: Text-based requirements management using version control, 2014. URL: https://doi.org/10.4236/jsea.2014.73020.
  8. Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen Toner, Ruth Fong, et al. Toward trustworthy AI development: mechanisms for supporting verifiable claims. arXiv preprint arXiv:2004.07213, 2020. Google Scholar
  9. Captum. Model interpretability for PyTorch, 2023. URL: https://captum.ai/.
  10. CEN-CENELEC. Artificial Intelligence. URL: https://www.cencenelec.eu/areas-of-work/cen-cenelec-topics/artificial-intelligence/.
  11. CertX. CertX: First Swiss Functional Safety and Cyber Security Certification Body. URL: https://certx.com/.
  12. Chun Sik Chan, Huanqi Kong, and Guanqing Liang. A comparative study of faithfulness metrics for model interpretability methods. arXiv preprint arXiv:2204.05514, 2022. URL: https://doi.org/10.48550/arXiv.2204.05514.
  13. Council of European Union. Artificial Intelligence Act: Council and Parliament Strike a Deal on the First Rules for AI in the World, 2023. URL: https://www.consilium.europa.eu/en/press/press-releases/2023/12/09/artificial-intelligence-act-council-and-parliament-strike-a-deal-on-the-first-worldwide-rules-for-ai/.
  14. Philipp Denzel, Stefan Brunner, Yann Billeter, Oliver Forster, Carmen Frischknecht-Gruber, Monika Ulrike Reif, Frank-Peter Schilling, Joanna Weng, Ricardo Chavarriaga, Amin Amini, et al. Towards the certification of AI-based systems. In 11th IEEE Swiss Conference on Data Science (SDS), Zurich, Switzerland, 30-31 May 2024, 2024. URL: https://doi.org/10.21256/zhaw-30439.
  15. DIN, DKE. Artificial intelligence standardization roadmap, 2023. URL: https://www.dke.de/en/areas-of-work/core-safety/standardization-roadmap-ai.
  16. EASA and Daedalean. Concepts of Design Assurance for Neural Networks (CoDANN) II. Technical report, May 2021. URL: https://www.easa.europa.eu/en/downloads/128161/en.
  17. European Parliament and Council of the European Union. Regulation (eu) 2024/1689 of the european parliament and of the council of 9 october 2024 on harmonized rules for artificial intelligence (artificial intelligence act) and amending certain union legislative acts, 2024. URL: https://eur-lex.europa.eu/eli/reg/2024/1689/oj.
  18. International Organization for Standardization. ISO/IEC 27001:2013 information technology – security techniques – information security management systems – requirements. Technical report, ISO, 2013. Google Scholar
  19. International Organization for Standardization. ISO/IEC 24029-1:2021 Artificial intelligence (AI) — Assessment of the robustness of neural networks — Part 1: Overview. Technical report, International Organization for Standardization, 2021. URL: https://www.iso.org/standard/77609.html.
  20. Leilani H Gilpin, David Bau, Ben Z Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA), pages 80-89. IEEE, 2018. URL: https://doi.org/10.1109/DSAA.2018.00018.
  21. GitHub, Inc. GitHub . URL: https://github.com/.
  22. GitHub, Inc. GitHub Actions. URL: https://github.com/features/actions.
  23. Karan Goel, Nazneen Fatema Rajani, Jesse Vig, Zachary Taschdjian, Mohit Bansal, and Christopher Ré. Robustness gym: Unifying the NLP evaluation landscape. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Demonstrations, pages 42-55, Online, June 2021. Association for Computational Linguistics. URL: https://www.aclweb.org/anthology/2021.naacl-demos.6, URL: https://doi.org/10.18653/V1/2021.NAACL-DEMOS.6.
  24. Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adversarial Examples. 2014. URL: https://doi.org/10.48550/arXiv.1412.6572.
  25. Anna Hedström, Leander Weber, Daniel Krakowczyk, Dilyara Bareeva, Franz Motzkus, Wojciech Samek, Sebastian Lapuschkin, and Marina Marina M.-C. Höhne. Quantus: An explainable AI toolkit for responsible evaluation of neural network explanations and beyond. Journal of Machine Learning Research, 24(34):1-11, 2023. URL: http://jmlr.org/papers/v24/22-0142.html.
  26. IEEE. IEEE CertifAIEd: the mark of AI ethics, 2022. URL: https://engagestandards.ieee.org/ieeecertifaied.html.
  27. IEEE Standards Association. IEEE Autonomous and Intelligent Systems Standards. URL: https://standards.ieee.org/initiatives/autonomous-intelligence-systems/standards/.
  28. IEEE Standards Association. IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems, 2023. URL: https://standards.ieee.org/industry-connections/ec/autonomous-systems/.
  29. International Organization for Standardization. ISO/IEC 25024:2015 Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Measurement of data quality. Technical report, 2015. URL: https://www.iso.org/standard/35746.html.
  30. International Organization for Standardization. ISO/IEC 24028:2020 Information technology — Artificial intelligence (AI) — Overview of trustworthiness in AI. Technical report, 2020. URL: https://www.iso.org/standard/77608.html.
  31. ISO. ISO/IEC JTC 1/SC 42 Artificial Intelligence, 2023. URL: https://www.iso.org/committee/6794475.html.
  32. Anna Jobin, Marcello Ienca, and Effy Vayena. The Global Landscape of AI Ethics Guidelines. Nature Machine Intelligence, 1(9):389-399, 2019. URL: https://doi.org/10.1038/s42256-019-0088-2.
  33. Dominik Kreuzberger, Niklas Kühl, and Sebastian Hirschl. Machine Learning Operations (MLOps): Overview, Definition, and Architecture. IEEE Access, 11:31866-31879, 2023. URL: https://doi.org/10.1109/ACCESS.2023.3262138.
  34. LF Projects, LLC. MLFlow. URL: https://mlflow.org/.
  35. LNE. Certification of processes for AI, 2023. URL: https://www.lne.fr/en/service/certification/certification-processes-ai.
  36. Beatriz M. A. Matsui and Denise H. Goya. Mlops: A guide to its adoption in the context of responsible ai. In 2022 IEEE/ACM 1st International Workshop on Software Engineering for Responsible Artificial Intelligence (SE4RAI), pages 45-49, 2022. URL: https://doi.org/10.1145/3526073.3527591.
  37. Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6):1-35, 2021. URL: https://doi.org/10.1145/3457607.
  38. Microsoft. InterpretML. URL: https://github.com/interpretml/interpret.
  39. Microsoft. Azure devops, 2005. URL: https://azure.microsoft.com/en-us/products/devops/#overview.
  40. NIST. NIST Technical AI Standards, 2023. URL: https://www.nist.gov/artificial-intelligence/technical-ai-standards.
  41. NIST. AI Risk Management Framework (AI RMF) Knowledge Base, 2024. URL: https://airc.nist.gov/AI_RMF_Knowledge_Base/AI_RMF.
  42. Office of the United Nations High Commissioner for Human Rights (OHCHR). A taxonomy of AI and human rights harms. Technical report, United Nations Human Rights Office of the High Commissioner, 2023. Accessed: 2024-11-06. URL: https://www.ohchr.org/sites/default/files/documents/issues/business/b-tech/taxonomy-GenAI-Human-Rights-Harms.pdf.
  43. High-Level Expert Group on Artificial Intelligence. Assessment list for trustworthy artificial intelligence (ALTAI), 2020. URL: https://altai.insight-centre.org.
  44. Independent High-Level Expert Group on Artificial Intelligence. Ethics guidelines for trustworthy AI. Technical report, European Commission, 2019. URL: https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai.
  45. Oxen.ai. oxen. URL: https://www.oxen.ai/.
  46. Nicolas Papernot, Fartash Faghri, Nicholas Carlini, Ian Goodfellow, Reuben Feinman, Alexey Kurakin, Cihang Xie, Yash Sharma, Tom Brown, Aurko Roy, Alexander Matyasko, Vahid Behzadan, Karen Hambardzumyan, Zhishuai Zhang, Yi-Lin Juang, Zhi Li, Ryan Sheatsley, Abhibhav Garg, Jonathan Uesato, Willi Gierke, Yinpeng Dong, David Berthelot, Paul Hendricks, Jonas Rauber, and Rujun Long. Technical report on the cleverhans v2.1.0 adversarial examples library. arXiv preprint arXiv:1610.00768, 2018. Google Scholar
  47. Maximilian Poretschkin et al. Leitfaden zur Gestaltung vertrauenswürdiger Künstlicher Intelligenz (KI-Prüfkatalog), 2021. URL: https://www.iais.fraunhofer.de/de/forschung/kuenstliche-intelligenz/ki-pruefkatalog.html.
  48. Rational Software. IBM doors, 2018. URL: https://www.ibm.com/docs/en/engineering-lifecycle-management-suite/doors.
  49. Jonas Rauber, Roland Zimmermann, Matthias Bethge, and Wieland Brendel. Foolbox native: Fast adversarial attacks to benchmark the robustness of machine learning models in PyTorch, TensorFlow, and JAX. Journal of Open Source Software, 5(53):2607, 2020. URL: https://doi.org/10.21105/joss.02607.
  50. SAE. Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. SAE J3016, 2021. URL: https://www.sae.org/standards/content/j3016_202104/.
  51. Wojciech Samek et al. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Springer, 2019. URL: https://doi.org/10.1007/978-3-030-28954-6.
  52. Wojciech Samek, Grégoire Montavon, Andrea Vedaldi, Lars Kai Hansen, and Klaus-Robert Müller. Explainable AI: interpreting, explaining and visualizing deep learning, volume 11700. Springer Nature, 2019. Google Scholar
  53. Anna Schmitz, Michael Mock, Rebekka Görge, Armin B Cremers, and Maximilian Poretschkin. A global scale comparison of risk aggregation in AI assessment frameworks. AI and Ethics, pages 1-26, 2024. Google Scholar
  54. Seldon. Alibi Explain. URL: https://github.com/SeldonIO/alibi.
  55. sicara. TF-Explain: Interpretability Methods for tf.keras Models with TensorFlow 2.x. URL: https://github.com/sicara/tf-explain.
  56. Siemens. Polarion, 2004. URL: https://polarion.plm.automation.siemens.com/.
  57. Ryan Soklaski, Justin Goodwin, Olivia Brown, Michael Yee, and Jason Matterer. Tools and practices for responsible AI engineering. arXiv preprint arXiv:2201.05647, 2022. URL: https://arxiv.org/abs/2201.05647.
  58. Guillaume Soudain. First usable guidance for Level 1 machine learning applications: A deliverable of the EASA AI Roadmap, 2021. URL: https://www.easa.europa.eu/en/downloads/134357/en.
  59. The White House. Fact Sheet: President Biden Issues Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence, 2023. URL: https://www.whitehouse.gov/briefing-room/statements-releases/2023/10/30/fact-sheet-president-biden-issues-executive-order-on-safe-secure-and-trustworthy-artificial-intelligence/.
  60. Trusted-AI LF AI Foundation. Adversarial Robustness Toolbox (ART). URL: https://github.com/Trusted-AI/adversarial-robustness-toolbox.
  61. Trusted-AI LF AI Foundation. AI Explainability 360 (AIX360). URL: https://github.com/Trusted-AI/AIX360.
  62. Trusted-AI LF AI Foundation. AI Uncertainty Quantification 360 (UQ360). URL: https://github.com/Trusted-AI/UQ360.
  63. Arnaud Van Looveren et al. Alibi detect: Algorithms for outlier, adversarial and drift detection, 2019. URL: https://github.com/SeldonIO/alibi-detect.
  64. Larysa Visengeriyeva, Anja Kammer, Isabel Bär, Alexander Kniesz, and Michael Plöd. MLOps Principles, 2020. URL: https://ml-ops.org/content/mlops-principles.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail