2026-10-05T21:32:54Z https://drops.dagstuhl.de/oai

oai:drops-oai.dagstuhl.de:28055 2026-10-05T06:44:06Z ddc:004 open_access

AI Ethics to Requirements Practice: Building and Evaluating the HealthAI Ethics Assistant Huang, Yutan Chen, Jingfan Wang, Fanyu Arora, Chetan Grundy, John Zhang, Cheng Requirements Engineering Software Engineering AI ethics Healthcare Large Language Models Regulatory Compliance. Healthcare AI software teams face growing pressure to translate ethical principles and regulatory obligations into concrete requirements, design decisions, and assurance evidence. Day-to-day requirements engineering (RE) practice still lacks lightweight tool support for connecting abstract frameworks, such as the EU AI Act and the NIST AI Risk Management Framework, to actionable requirements work. This paper builds on a two-phase research project. In the first phase, we conducted an empirical interview study with healthcare AI practitioners spanning clinicians as end users of HealthAI systems and software engineers as developers of HealthAI software systems, which surfaced recurring concerns around transparency, accountability, and the difficulty of mapping regulatory obligations to concrete engineering tasks. Informed by those findings, in the second phase we designed, implemented, and conducted an initial evaluation of the HealthAI Ethics Assistant, an AI-enabled RE tool for healthcare AI software systems. The tool supports practitioners in generating, validating, and comparing ethical requirements through a structured Create-Validate-Compare workflow. It was implemented as a full-stack web application and grounded in a structured knowledge base combining regulatory guidance (EU AI Act, NIST AI RMF) with practitioner concerns surfaced by the interview study. The work was conducted in close collaboration with an R&D engineer at a medical device manufacturer, who contributed to industrial problem framing and participated as the first practitioner evaluator in the case study reported here. Our initial evaluation is exploratory, involving a single case study, and used the Technology Acceptance Model and the System Usability Scale. Traceable regulatory references were rated as the most valuable and trustworthy feature, highlighting the importance of explainability and evidence support in compliance-oriented requirements work. The main adoption challenge was not basic usability, but fitting the tool into existing engineering and regulatory workflows. We derive practice-oriented lessons for designing AI-enabled requirements tools in regulated domains: ground LLM outputs in explicit compliance sources, support multiple practitioner perspectives, make generated requirements reviewable rather than authoritative, and align tool use with existing assurance processes. These early results suggest that AI-enabled assistants can help bridge ethical AI principles and practical requirements engineering when designed as auditable, human-in-the-loop support tools rather than autonomous compliance solutions. Schloss Dagstuhl – Leibniz-Zentrum für Informatik Yutan Huang and Jingfan Chen and Fanyu Wang and Chetan Arora and John Grundy and Cheng Zhang 2026 Is Part Of LIPIcs, Volume 394, 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026) InProceedings Text doc-type:ResearchArticle publishedVersion application/pdf doi:10.4230/LIPIcs.ESEM.2026.87 urn:nbn:de:0030-drops-280555 https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.ESEM.2026.87 eng https://creativecommons.org/licenses/by/4.0/legalcode