AI Readiness of Standards: Bridging Traditional Norms with Modern Technologies

AI as an emerging technology will impact most areas of industry and society. The project „AI Readiness of Standards“ was set to find out what this means for the German national body of standards, which consists of roughly 30,000 DIN, DIN EN and DIN EN ISO documents.

The approach was a mixture of individual human assessment and a machine-assisted process.

During the 2.5 years of the project, answers to the following research questions were sought:

• $\mathrm{■}$

  How can a complex, 100-year-old regulatory framework be adapted for AI technologies?

• $\mathrm{■}$

  Is a comprehensive review of all 30,000 documents for AI readiness necessary or possible?

• $\mathrm{■}$

  How crucial is the application context in adapting standards for AI?

• $\mathrm{■}$

  How might software technologies help with this?

This paper gives an overview of the findings.

Section 2 gives an overview of definitions and previous research, setting out the fundamental principles for this project.

These principles were applied by human experts to a large number of existing standards documents, resulting in section 3, which provides a high-level overview of the challenges and results, including a brief discussion.

To be able to apply the principles to the entirety of the body of standards in an economically feasible way, automation technology has to be used. Section 4 describes the development and use of three primary software tools.

Finally, section 5 concludes the paper with the projects’ achievements and possible future challenges.

What actually is AI readiness? In current literature, there are several understandings of AI readiness, mostly revolving around organization theory or innovation adoption frameworks. AI readiness generally refers to an organization’s capacity to deploy and utilize AI technologies effectively. The concept encompasses various dimensions including technological infrastructure, organizational preparedness, and the external environment.

For instance, one study describes AI readiness as part of the broader process of AI adoption, emphasizing the need for ongoing assessment and development rather than a single preliminary evaluation [4]. It posits that AI readiness and adoption are highly interdependent and must be integrated throughout the entire adoption process to ensure successful implementation. Another work offers an AI readiness framework which assesses an organization’s capabilities in four key dimensions: technologies, activities, boundaries, and goals, providing a pragmatic tool for facilitating digital transformation within organizations [3].

[2] highlight technological, organizational, and environmental factors, while [1] define AI readiness as an organization’s capacity to implement and utilize AI using cultural aspects and necessary resources, emphasizing the importance of top management support, resource availability, and organizational infrastructure.

Regarding standardization there was no known precedence as to how a standards document that is AI ready should look like, so we defined the term ourselves as follows (summarized):

The AI readiness of a standard refers to whether it meets conditions that ensure an AI system compliant with this standard is secure and remains so according to its intended purpose. A standard is considered AI ready if it is specific enough to cover the use of AI or AI-specific requirements and measures, while being unambiguous for users.

For more information see [5].

The initial definition and the evaluation method were developed through lengthy discussions and a wide stakeholder participation process, engaging AI experts, DIN standards committees, business associations as well as the public. This generally worked well. But already at this stage there were often more questions than answers.

While later applying the evaluation method, feedback loops provided input for 20 subsequent evolutions. Validation and testing in real conditions were identified as key for success, but not trivial.

Different industry experts for the application of standardization documents in their respective field proved rare, leading to a higher level of human resources needed to generate expert knowledge.

Additionally, the lack of legal expertise in the project led to the deferral of that aspect, addressing it in a dedicated step within the evaluation method.

The high complexity of the task “detailed human expert analysis of approximately 1100 standardization documents” made it challenging to ensure consistency in evaluation across different experts. The evaluation of normative documents was heavily influenced by the evaluators’ background knowledge, personal biases, and their interpretation of the content e.g., resulting in a different understanding of concepts such as AI, practicability of AI applications in a specific domain. Efforts such as verifying the plausibility of classifications and resolving conflicts through expert reviews were taken. This process required an additional layer of review and revision, adding complexity to the workflow. A certain level of bias and interpretive flexibility remained.

The national body of standards has traditionally been dominated by technical drawings, specifications, and historically developed document families. However, in contemporary applications, there has been a significant shift towards digitization, with increasing demands for “smart” standards, automated integration of standard contents, and even automated conformity checks.

Given the vast number of approximately 30 000 standards, a meticulous manual review is deemed impractical. Consequently, this project explored the utilization of software for the large-scale and scalable analysis of these standards.

Specifically, the research focused on the conceptualization, development, and investigation of three primary tools: a tool for semantic similarity search, a tool for linking standardization documents to a knowledge database, and an AI tool designed for the automated classification of standards into AI readiness classes.

The project aimed to develop a proof-of-concept and provide initial concrete indications and points of connection for AI in established standards. Both aspects were clearly answered positively and supported with examples and results.

However, as is often the case, complex issues require complex solutions. A systematic integration of AI into the standards framework requires deepening the developed content and building competencies. Software and data-driven approaches can help, but they also come with non-negligible additional organizational and technical efforts. We are curious to see how the advancing digitalization of our society will impact standardization and look forward to future challenges.