Software Architecture and Machine Learning

The seminar on software architecture and machine learning was structured to foster an interactive and productive idea exchange environment for all participants. Participants were initially asked to prepare a single-slide introduction about themselves and their work in software architecture and machine learning, which helped to establish a foundational understanding among attendees. Prior to the workshop, organizers developed an initial version of a key concepts map and a research challenges map using a grounded theory approach, which was represented visually on miro boards. During the seminar, participants delivered 10-minute lightning talks about their SA&ML work, which were instrumental in refining the research and concept maps with keywords and challenges mentioned by the speakers. At the end of the first day, organizers summarized the discussions and identified key emerging topics and research areas, leading to an initial set of key quality attributes critical to ML-enabled system development. The second day continued with discussions and a working group formation session, where participants prioritized five critical quality attributes for ML-enabled systems: Evolvability, Uncertainty and Observability, Trust and Trustworthiness, and Data Centricity. Four working groups were formed around these quality attributes, with appointed editors documenting the discussions. Each group was provided with a draft template to guide their discussions for consistency, and organizers rotated among groups to offer diverse insights. Plenary meetings on the second and third days allowed groups to share progress and discuss challenges, complemented by end-of-day meetings among organizers to consolidate learning and monitor progress. The seminar concluded on the fourth day with presentations from each group, open for feedback from other participants. This collaborative review process led to the integration of feedback into the final discussions, resulting in a well-defined set of challenges and research directions, marking the seminar’s successful completion. In addition, this format ensured a thorough exploration of the seminar’s themes and fostered active participation and collaborative learning among attendees.

There is growing uncertainty about the environment of software systems. Therefore, how the system should behave under different contexts cannot be fully predicted at design time. It is considerations such as these that have led to the development of self-adaptive systems (SAS), which can dynamically and autonomously reconfigure their behavior to respond to changing external conditions.

The scope of the talk is in the area of Requirements Engineering (RE) and the development of techniques to quantify uncertainty to improve decision-making. The explicit treatment of uncertainty by the running system improves its judgment to make decisions supported by evaluating evidence found during runtime, possibly including the human-in-the-loop. I will also discuss how quantification of uncertainty can be used to improve requirements elicitation (using simulations, for example).

The talk will cover different approaches to quantifying uncertainty and its role in Human-Machine Teaming.

More and more software systems incorporate techniques from machine learning (ML) to support decision-making or automate information processing. Such ML systems still incorporate many traditional components but nonetheless require specialized practices in certain areas. From a software architecture perspective, one question is if existing practices and frameworks are suitable for the effective modeling and documentation of ML systems. While there are many methods for architecture documentation based on concepts like viewpoints, views, and modeling notations, it is still unclear how ML professionals can best apply these practices to model the architecture of ML systems. The data dependency and uncertainty of ML components, ML stakeholder diversity, plus new quality concerns like explainability or model observability might require new types of views and diagrams. While a few specialized approaches have been proposed, other software architecture publications use vastly different notations and diagrams to model ML systems.

In this short talk, I want to lay the foundation for a discussion about this topic. I will briefly talk about challenges in this space and present a few existing approaches. My goal is to discuss if these challenges are different from architecture modeling for traditional systems and if new approaches are needed. Ideally, the discussion might lead to research collaborations in this space to a) better understand challenges and b) create or adapt methods to overcome them.

Digitalization is concerned with software, data, and AI as enabling technologies. It changes the company and business ecosystem in which it operates from transactional to continuous, i.e., XOps. This requires fundamental changes in how we work with technology from offline and centralized to online and federated. The talk includes examples from Software Center (www.software-center.se) companies, including automated experimentation, federated learning, reinforcement learning, semi-supervised learning, and related topics.

Predicting the performance of highly configurable software systems is the foundation for performance testing and quality assurance. To that end, recent work has been relying on machine/deep learning to model software performance. However, a crucial yet unaddressed challenge is how to cater for the sparsity inherited from the configuration landscape: the influence of configuration options (features) and the distribution of data samples are highly sparse. This talk is based on the paper in which we propose an approach based on the concept of “divide-and-learn”, dubbed DaL. The basic idea is that to handle sample sparsity, we divide the samples from the configuration landscape into distant divisions, for each of which we build a regularized Deep Neural Network as the local model to deal with the feature sparsity. A newly given configuration would then be assigned to the right model of division for the final prediction.

Experiment results show that using strong domain knowledge to specialize AI performs significantly better in configuration performance learning.

This presentation introduces the Spanish GAISSA project ("Towards Green AI-based Software Systems: An Architecture-centric Approach") run by the GESSI research group at UPC. The main hypothesis of GAISSA is that the impact of architectural decisions on environmental sustainability shall be understood, defined, reported, and managed in order to model, develop, and deploy green(er) AI-based systems. After presenting the objectives and vision, the presentation details the work done so far, which is basically a series of empirical studies on the effect of model and system architectures on environmental sustainability, exploring trade-offs with other attributes such as accuracy. Emerging challenges are the consolidation of the results of such studies, and how to boost the attention on sustainability in the community. In this respect, a result of the project is the so-called Energy Label, which categorizes the efficiency level (from A to E) of a trained model, defined in terms of several attributes. The concept has been proven using the HuggingFace repository. As future work, the talk mentions the link between requirements and architectures.

Many organizations, especially enterprises, use a data-lake-based infrastructure to prepare their data for machine learning modeling. Data-lake-based systems ingest raw data from a wide variety of data sources such as databases, legacy information systems, automation systems, and more. The processed data is intended to serve the needs of different users and use cases. A data-lake-based system consists of a complex technology stack that implements distributed data processing and data storage. Typical transformation steps include ingestion, cleansing, enrichment, transformation, and finally, serving.

Typically, a centralized team of data experts runs the infrastructure and implements the data processing. The thinking is centered around different data pipelines.

The monolithic architecture of the data lake and the centralized organization result in a siloed way of working that puts the data team in a difficult position, often leading to unsatisfactory machine learning results and limiting the data culture of the data team. The data team suffers from

• $\mathrm{■}$

  Operational data that is not fit for analytics, and the team lacks a deep understanding of the data

• $\mathrm{■}$

  Frequent and unexpected downstream changes that require the team to patch data pipelines in response

• $\mathrm{■}$

  Time spent searching for and identifying the right data for use cases

• $\mathrm{■}$

  Lack of understanding of business requirements

The resulting monolithic architecture and centralized team setup ignore common software engineering best practices, such as reasonably sized teams of 5-8 people with clear ownership, lack of mechanisms to offload cognitive load, minimizing hand-offs through loose coupling, and clear interfaces between components and teams.

One architectural approach that tries to facilitate the software engineering best practices are data products introduced by Dehgani. They ingest data only from a few source systems or other data products, encapsulate the data transformation and data storage, provide an easy-to-use business-oriented API, and contain a human-readable description and code for observability. Data products group elements of data transformation so that they can be owned by a software team and consumed as a service by other teams.

Essential to the adoption of a data product is a data platform that makes it easy for data product developers to follow the governance policies of the surrounding organization. A key goal of the data platform is developer experience (DevEx). There are several points to learn from a data-product-centric architecture:

• $\mathrm{■}$

  Design software and organizational architecture for ML-enabled systems together.

• $\mathrm{■}$

  Defined components or products should be independently deployable and a team of 5-8 developers should be sufficient to develop them.

• $\mathrm{■}$

  There needs to be some kind of platform to reduce the cognitive load.

• $\mathrm{■}$

  The platform should be built in an evolutionary fashion starting from the thinnest viable platform (TVP).

Our group at the University of São Paulo has 23 years of experience teaching an advanced software development course called “Agile Software Development Lab.” It is extremely popular with students and has been very successful in teaching advanced software development concepts with a ’learn by doing’, project-based approach with real external customers. However, many recent projects have dealt with machine learning and AI-based systems, which sometimes significantly increase the complexity of projects. We are then currently remodeling the course to incorporate pedagogical aspects related to AI-based systems development.

In this short talk, I provide an experience report about the development of a complex system for respiratory insufficiency detection that has been developed at the University of São Paulo in a collaboration between data scientists, physicians, linguists, and computer science professors and students. We discuss the challenges and lessons learned in architecting and building the system.

Developing software systems that contain machine learning (ML) components requires an end-to-end perspective that considers the unique life cycle of these components – from data acquisition to model training to model deployment and evolution. While there is an understanding that ML components, in the end, are software components, there are some characteristics of ML components that bring challenges to software architecture and design activities, such as data-dependent behavior, the need to detect and respond to drift over time, and timely capture of logs, metrics, user input and labeled data to inform retraining. In this presentation, I talk about some of these challenges and propose a set of practices to address these challenges, such as co-architecting, the recognition of monitorability as a driving quality attribute, and system-level architecture patterns and tactics. The presentation concludes with a list of opportunities to make progress in software architecture for ML systems, which includes adapting and growing the software architecture body of knowledge for application to ML systems, bringing the software architecture body of knowledge and discipline to model development activities, and positioning software architecture as a unifying activity for system stakeholders (model developers, software engineers, and operations).

BeT (Behavior-enabled IoT)[1] is a project that aims to provide a reference architecture, conceptual framework, and related techniques to design behavior-enabled IoT systems and applications. The presentation introduces the concept of the Internet of Behaviors (IoB) and analyzes the human-system bi-causal quality connection effects. BeT analyzes the emerging problem of understanding the mutual influence between QoS (Quality of Service) and QoE (Quality of Experience) in IoB systems, where dynamic changes in the system and surroundings can give rise to nontrivial unforeseen cause-effect mutual relations between system and human behaviors.

The boundary between SA4ML and ML4SA is diffusing as more general-purpose AI models are incorporated into systems. As more capable development tools enter the developer’s ecosystem the role and responsibilities of the architect will shift [1]. As generative AI and foundation models increase in capabilities, they will help reveal the criticality of architecture knowledge and how it must guide development. As capabilities of large language models (LLM) evolve, however, there is a potential increasing gap between what tools supported by LLM can accomplish realizing narrowly scoped implementation tasks versus how they can do so while being cognizant of software architectural concerns [2]. This talk will emphasize some of the open questions and challenges that software architecture researchers need to address as LLM-supported development tools evolve rapidly. These include, but are not limited to, the following:

• $\mathrm{■}$

  What are the specific design and architectural concerns that need to be addressed when using generative AI?

• $\mathrm{■}$

  Can generative AI tools be used to improve the design and architecture of systems?

• $\mathrm{■}$

  Can these tools provide new features and capabilities that can be used to support the architectural design process?

• $\mathrm{■}$

  Could generative AI tools be used to generate design patterns and tactics, which could then be used to guide the development process?

• $\mathrm{■}$

  Can these tools accelerate the generation of alternative designs and their comparison?

• $\mathrm{■}$

  Can they be used to provide feedback on designs, such as identifying potential risks and issues?

The main purpose of the WG1 group on “Data Centricity” was to explore and address the pivotal role of architectural elements that extract, transform, load, store, and share data in the architectures of ML and analytics systems. The group focused on examining the entire data life cycle – from acquisition to transformation to runtime consumption – and understanding how data is integrated and managed by an ML-enabled system throughout this data life cycle.

The discussions emphasized that effective data management throughout the data life cycle is crucial for the success of ML-enabled systems. This includes recognizing the challenges associated with the complexities of data architectures. The WG1 group also identified the need for a distinct architectural view that focuses specifically on data, given its critical importance in ML-enabled systems. They acknowledged that while current software architecture practices and architectures of non-ML systems address data processing and storage, there is a need for a more focused approach to identify strategies for addressing data-centricity to cater to the unique requirements and challenges posed by ML systems.

In summary, the WG1 group’s discussions emphasize the centrality of data in ML systems and the need for specialized and clearly communicated architectural approaches to manage and utilize data in these systems effectively.

The primary objective of the WG2 group on "Evolvability" was to address the unique challenges in developing ML-enabled systems that can evolve efficiently in response to changing requirements and environmental factors. The group’s primary focus was on understanding the dynamics of ML components within these systems, particularly how they interact with non-ML components and Machine Learning Operations (MLOps) infrastructures and practices.

A key finding of WG2 was the critical role of MLOps in ensuring the evolvability of ML components within ML-enabled systems. MLOps, supporting the entire life cycle of ML components from development to deployment and maintenance, emerged as a pivotal factor in managing and automating the continuous evolution of these systems. The group identified that the integration of MLOps within existing DevOps practices is not only essential but also challenging, given the unique characteristics of ML components.

Another significant aspect highlighted by WG2 was the concept of technical debt in ML systems. ML-enabled systems bring forth new classes of design and architectural challenges related to degradation in data or models as technical debt, which can impede the maintenance and evolution of ML-enabled systems. Addressing these forms of technical debt is crucial for sustaining the long-term viability and evolvability of these systems.

WG2 also emphasized the importance of the skill sets and roles of architects designing and maintaining ML-enabled systems. Architects need to possess a comprehensive understanding of aspects such as data engineering, model engineering, and ethical considerations to effectively manage the evolution of these complex systems.

In essence, the WG2 group underscores the complexities involved in creating evolvable ML-enabled systems and highlights the importance of MLOps, technical debt management, and specialized architectural knowledge in addressing these challenges.

The working group on “Observability and Uncertainty” primarily focused on how uncertainty and observability were intertwined and how they impact the design, implementation, and operation of ML-enabled systems.

A key finding of the group was the often inherent non-deterministic and statistical nature of ML techniques, which introduces a fundamental level of uncertainty in these systems. This uncertainty is not just a challenge but a necessary aspect to consider in the architecture of ML-enabled systems. The group discussed two main ways to handle uncertainty: (1) reducing or eliminating uncertainty through strategies similar to using redundancy in reliability engineering and (2) managing uncertainty, which includes strategies for dealing with incomplete information and uncertain components.

The group’s discussions underscored the close relationship between uncertainty and observability. Observability – the ability to infer the internal state of a system based on its output – is crucial for understanding and managing uncertainty in ML systems. The degree of uncertainty dictates the requirements for observability, impacting various other aspects of software development, including trustworthiness and evolvability.

In summary, WG3’s outcomes emphasize the importance of treating uncertainty as a primary concern in the design and implementation of ML-enabled systems. In addition, the discussions highlight the need for effective observability mechanisms to identify and manage the inherent uncertainties of ML-enabled systems to ensure their reliability and effectiveness.

The main purpose of WG4 was to explore the key aspects and challenges in designing trustworthy ML systems, such as in safety-critical and autonomous contexts.

The outputs of the discussions in this group emphasize the distinction between trust and trustworthiness and various aspects of these concepts in ML-enabled systems, including state-of-the-art trustworthy AI, the role of trust as an element of overall responsible AI practices and their relevance to ML-enabled system development, and the engineering of trustworthy ML-enabled systems. It also examines the importance of providing evidence of trustworthiness tailored to different stakeholders for calibrated trust, such as through certification labels and empirical studies, and discusses enabling the design of trustworthy ML-enabled systems through architectural models and toolkits.

A significant portion of the discussions were dedicated to addressing the challenges in evaluating the trustworthiness of ML-enabled systems and the role of human-in-the-loop in ensuring trustworthiness. The discussions in this group focused on the trust and trustworthiness of ML-enabled systems and had a close relationship with considerations that apply to AI systems in general, such as responsible AI practices. The discussions, in particular, emphasized the need for explainable AI as a key component in building trust in ML-enabled systems.

In summary, WG4’s observations emphasize the critical need for defining trust and trustworthiness clearly in ML-enabled systems, exploring the complexities of achieving the several qualities essential in building trust and trustworthiness and developing and extending frameworks and strategies to guide the development of trustworthy applications with ML components.

Data-centricity emphasizes the significance of data quality and management for ML-enabled systems. High-quality, well-managed data is crucial for the development of reliable ML models and for maintaining the trustworthiness of the system.

Data-centricity also impacts evolvability and adaptability, as systems that can adapt to changes in data or requirements are more sustainable over time.

Uncertainty is a fundamental aspect of ML-enabled systems due to the often non-deterministic and statistical nature of ML models. Uncertainty can arise from the quality of training data, the model design, interactions between ML and other components, and the environment in which the system operates.

Observability involves the ability to monitor the internal states and outputs of the system to enable understanding and make informed decisions about its functioning, especially in the face of uncertainty. Observability helps system developers and operators understand and manage uncertainty.

Observability can also contribute to enhancing explainability, as more observable systems provide more information that can be used to explain inference results to users and for engineers to diagnose systems. Observability also contributes to evolvability as runtime data can be used to determine, for example, when a model needs to be retrained in response to drift.

Evolvability was defined by the seminar participants as a system’s ability to adapt to changes in expectations (i.e., requirements), while adaptability refers to the ability of the system to handle (or be modified to deal with) changes in its environment.. Evolvability, therefore, is mostly a design-time concern, whereas adaptability is both a design-time and runtime concern. These attributes are essential for designing ML-enabled systems, as these systems must continuously evolve and adapt to remain effective. Evolvability and adaptability are influenced by observability and uncertainty, and together, they contribute towards maintainability; by addressing these attributes proactively and intentionally, systems can be designed to evolve and adapt more effectively. Evolvability and adaptability are also closely linked to data centricity, as the system’s ability to evolve and adapt is dependent on the quality and relevance of the data it uses and how its evolution and adaptation are influenced by changes in data.

Trust is the subjective confidence stakeholders place in a system’s quality. It is influenced by the system’s trustworthiness.

Trustworthiness involves the system’s objective ability to consistently perform according to quality expectations and ethical standards. It is bolstered by explainability, as stakeholders are more likely to trust a system they can understand. Trust and trustworthiness are impacted by data centricity, as the quality and management of data underpin the reliability and robustness of ML-enabled systems. Related, if the system provides better explainability, it fosters trust in the system. A system that is trustworthy (through evidence of achieving quality attributes such as security, reliability, and fairness) merits the trust placed in it by users.

Several open research areas in SA&ML were identified as a result of the seminar discussions. High-priority research areas requiring near-term focus for advancing the state of the practice include the following:

• $\mathrm{■}$

  Architectural Design for Data-Centricity: Identifying and developing architectural approaches that effectively address the central role of data in ML-enabled systems, including data acquisition, processing, and management, as well as designing systems that can adapt to changes in data characteristics, need attention from researchers as well as practitioners in documenting their lessons learned. Existing architecture patterns, tactics, and quality attributes related to data architecting need to also be shared more consistently, and gaps need to be filled where applicable.

• $\mathrm{■}$

  Evolvability and adaptability of MLOps Architectures: Creating architectural designs that support the evolvability and adaptability of ML-enabled systems, particularly focusing on leveraging MLOps infrastructure, is not common practice. Challenge areas include addressing the lifecycle management of ML models and ensuring systems can evolve and adapt to new and changing requirements and environments.

• $\mathrm{■}$

  Uncertainty as a First-Class Concern: Integrating the management of uncertainty into the architectural design process for ML-enabled systems first requires identifying sources of uncertainty which can effectively be managed with architectural approaches. This involves developing methods for quantifying and mitigating uncertainty that can be leveraged by architecture practices and constructs.

• $\mathrm{■}$

  Observability in ML-enabled Systems: Existing approaches in system observability need to be enhanced to better observe and manage the behavior of ML components. This includes developing metrics and tools for monitoring and understanding ML components and ML-enabled system states.

• $\mathrm{■}$

  Trust, Trustworthiness, and Ethical Considerations: Building trustworthiness into the architecture of ML-enabled systems, including designing for ethical considerations, compliance with regulations, and ensuring transparency and explainability of AI decisions, is a growing need. Identifying what aspects of trustworthiness and ethical considerations can be handled architecturally and clearly communicating the remaining gaps is critical as research in this area progresses.

• $\mathrm{■}$

  Human-in-the-Loop AI Decision Making: Architecting systems that effectively incorporate human oversight and interaction, particularly in critical decision-making processes to ensure balanced autonomy and control in AI systems, will be an area of research that influences many quality attribute concerns, as well as how responsibilities are allocated to architectural elements, potentially resulting in new architecture patterns and tactics.

At the seminar, participants recognized that there is a lack of understanding and common definitions for many concepts related to ML-enabled systems and existing architecture practices and fundamentals. Furthermore, in addition to the set of key quality attributes discussed in detail and shared in this report, other quality attributes that require attention from the software architecture community were identified. Hence, a more detailed description of the concepts identified, extended discussions around the quality attributes discussed, as well as a more comprehensive explanation of the research roadmap, are planned for publication in appropriate venues.