Improving Trust between Humans and Software Robots in Robotic Process Automation

Trustworthiness in RPA is often associated with the system’s technical characteristics, such as robustness and accuracy. Conversely, we believe that end-user perception of trust in RPA must be considered crucial for ensuring adoption and proper use of this technology [23]. The lack of trust arises when specific executions of an RPA system, i.e., the SW robots, are not entirely predictable or controllable during the enactment of specific tasks, fueling a degree of uncertainty about potential outcomes regarding the SW robots’ behaviors [26].

Based on the above considerations, the objective of this working group has been the specification of a framework to conceptualize the notion of trust and its intrinsic and perceived characteristics. We focused on the perceived trust that a human user has concerning to a SW robot executing a specific task. In addition, leveraging our framework, we propose a novel construct to measure such a perceived trust as the willingness of the user to give up control to the SW robot. Finally, we discuss open issues and future works.

The generic trust framework [7] involves three entities: the trustor puts trust in the trustee in relation to a trust subject. Applied to the context of RPA, the trustor is the user of the RPA system, who is supposed to collaborate with a novel software robot, the trustee. The user needs to trust the robot to conduct a given task, the trust subject, in a reliable and valid way. Our discussions revolved around this instantiated framework, as sketched in Figure 1. We then focused on the properties of the three entities that potentially impact the level of trust among them. In the following, we briefly discuss these properties and how we suggest to specify them.

For the user, we see two main properties:

• $\mathrm{■}$

  Inclination to use RPA: This describes the user’s general attitude towards RPA technology and hence their propensity to use it at all (independent of a concrete task or robot). It might be influenced by the personality of the user and their overall attitude towards technology. One way to further assess this inclination is by means of the Technology Acceptance Model [13].

• $\mathrm{■}$

  Role: The other property that influences a user’s trust is their role within the organization. We currently see two main roles: the (process) manager, who decides whether a software robot will be used for a certain task, and the employee, who has to collaborate with the software robot. Another possible role would be that of the developer of the bot, but that role is less relevant when considering the trust in a concrete software robot. As the developer is the one who creates that software robot, their trust needs to be placed in the underlying RPA platform, not the robot itself. Therefore, we decided not to consider it for now.

On the part of the software robot, we see five main properties:

• $\mathrm{■}$

  Adaptability: This describes the ability of the robot to determine its own behavior. It can be assessed by the number of task variants or behaviors that the robot can handle.

• $\mathrm{■}$

  Autonomy: We refer to Group 4 for this discussion.

• $\mathrm{■}$

  Mode of Interaction: This describes the possible ways that the user can communicate with the robot. It can be either conversational, e.g., through a chatbot, or technical, e.g., through an API.

• $\mathrm{■}$

  Controllability: This describes the ability of the user to interfere with the robot’s behavior.

• $\mathrm{■}$

  Trust Signals: This describes potential trust-inducing measures that come from the environment of the robot, e.g., testimonials.

Finally, we see two main properties that are relevant for the task:

• $\mathrm{■}$

  Perceived Uncertainty: This describes the amount of knowledge that the user has about how the robot executes the task. In general, the more complex the task, the higher the perceived uncertainty may be.

• $\mathrm{■}$

  Expected (Negative) Consequences: This describes the negative impact that a wrong or unreliable behavior of the robot can have on the user. In general, the more important the task, the higher the expected negative consequences will be.

From these two properties, we can derive a simple classification of tasks, as outlined in Figure 2. Exemplarily, we have defined four possible classes of tasks. For each task, we have determined the degree of controllability that a user requires for this class. Both perceived uncertainty and expected consequences increase the required controllability, resulting in three different levels (low, medium, high).

To establish an indicative measure of trust between a user and a software robot for performing a given task, we make the following arguments:

We propose a preliminary framework that conceptualizes how trust is formed between humans (users) and RPA (robot systems). The next step is to validate this model. Our aim is to define a comprehensive conceptual framework that precisely outlines what trust in Robotic Process Automation (RPA) entails. This framework should encompass various dimensions of trust tailored to the interactions between users and software robots.

Future efforts should focus on creating standardized trust metrics and validation processes. These metrics need to be adaptable to different RPA contexts and capable of capturing both quantitative and qualitative aspects of trust. Identifying and validating empirical methods for measuring trust between users and software robots remains an ongoing challenge. Robust, scalable, and accurate tools and techniques are required to capture trust levels in real-world scenarios across diverse user groups.

Further exploration is needed to understand the influence of contextual factors and specific use cases on trust in RPA. Research should aim to identify how these factors impact trust and how they can be accounted for in trust assessments and frameworks. Additionally, trust is dynamic; it evolves based on user experiences and interactions over time. Understanding how trust develops, degrades, or is restored in the context of RPA is crucial. Future research should also focus on longitudinal studies to capture these dynamics and develop models that reflect the temporal aspects of trust.

Finally, as RPA increasingly integrates with artificial intelligence (AI) and other emerging technologies, future research should explore how these advancements affect trust. Understanding the interplay between RPA, AI, and user trust will be critical for developing next-generation automation systems.

By addressing the open issues and focusing on these future work areas, this research can contribute to a deeper understanding of trust in RPA and develop practical solutions to enhance trust among human users and these systems.

In recent years, lightweight automation technologies such as RPA have surged in popularity. The creators of these tools have often touted them as a panacea for a myriad of business inefficiencies. With promises of unprecedented efficiency gains, cost reductions, and seamless integration across enterprise systems, RPA has fueled a wave of hype and high expectations. Promises of the eventual arrival of AI-empowered intelligent systems have only strengthened this dynamic.

However, as more and more organizations have had experience with RPA, many of them find themselves grappling with the chasm between promise and reality, as many implementations fail to deliver on the promise. This divergence often leads to mismatched expectations, resulting in disillusionment and a growing sense of mistrust. Therefore, it is critical to reduce the gap between the pre-implementation expectations and post-implementation reality to ensure continued trust in the technology and eventually improve the success rate of RPA initiative.

In this working group we discussed the following research question (RQ): How to effectively calibrate and manage trust in RPA to align pre-implementation expectations with post-implementation realities and ensure successful adoption and use of the technology?

Our discussion focused on three components that we deemed necessary to address RQ, thus developing a successful research project:

• $\mathrm{■}$

  First, we analyzed the features and properties of RPA that contribute to its inherent trustworthiness. This included discussing attributes, such as the improved productivity, cost savings, cycle time reduction, accelerated ROI and error reduction. For a complete list of attributes please refer to the following works [2, 4].

• $\mathrm{■}$

  Then, we examined the appropriate conceptualization of the object of trust. This involved identifying what specifically is being trusted in the context of RPA, e.g., is the object of interest the technology or the software “agent” implemented by the RPA technology?

• $\mathrm{■}$

  Finally, we explored various methods for trust calibration. The discussion focused on identifying different methods for assessing the level of trust users place in the RPA. Since, existing approaches considered qualitative approaches to measure trust [31], we focused to develop a quantitative framework for trust calibration.

To ensure a common and strong foundation on which to further develop the trust calibration tool, we started out with a discussion about how we could use and refined the theoretical perspectives from current trust literature. This discussion lead to two main contributions:

The discussions during the seminar provide the framing and foundation for the future research, however, there are a number open issues and action points that require more substantial work.

First, we need to conduct a comprehensive literature review on approaches to trust calibration, especially in the context of technology; and user and managerial expectations from RPA pre- and post-implementation, how these expectations are related to the inherent properties of RPA, and how these expectations evolve over time.

Second, we aim to map what are the factors that constitute the perceived trustworthiness to construct a scale measuring this construct. This step is aimed to consists of two stages. First, we want to review the literature to identify these factors. Second, we consider to make qualitative research in the home countries to add factors that somehow are missed by earlier researchers. After the factors are being constructed, we planned to make the quantitative research and to check how the scale is overlapping with other scales measuring perceived trust towards RPA.

Third, the group intends to construct a scale that would contribute to business practitioners. The analysis of the literature and business talks with business practitioners suggest that the attitudes, trust, satisfaction with RPA is not measured during annual surveys. It is a substantial shortcoming as the perceived trust influence both adoption and intention to use the system. One of the reasons why managers do not measure the trust in software robots is the lack of a credible and easy tool that can be smoothly used in their teams. We intend to propose such a tool with the aid of both managers and actual users of the system.

To validate trust calibration scale we need to empirically test the scale and refine the scale and the eventual tool based on the empirical findings. The scale should be validated through, first the expert interviews, and then a survey. There is a potential to study trust calibration scale for two groups of human agents (as separate studies): (1) Managers making a decision about the adoption and development of RPA; (2) Users that use and collaborate with the RPA in their day-to-day work. The scale validation will require longitudinal approach surveying users at two different time points, pre-implementation and post-implementation. The trust levels will be then compared across the time points. This will allow for validating which aspects of RPA impact the trust in technology over time, and interpret the changes in attitudes of human agents over time.

In conclusion, while RPA technologies have generated significant excitement and high expectations, the reality of their implementation often falls short, leading to disappointment and mistrust. To bridge this gap, it is crucial to align expectations with actual outcomes to maintain trust and improve the success rate of RPA initiatives. Our working group focused on calibrating trust, understanding trustworthiness, and developing a trust measurement scale. By conducting comprehensive research and creating practical tools for business practitioners, we aim to ensure more reliable and sustainable adoption of RPA technologies.

One critical aspect that often undermines the efficiency and effectiveness of the interactions between humans and software robots (“robot”, for short) is the calibration of trust of human users [32].

That is, when humans work with a (new) robot, their level of trust in that robot is not always appropriately calibrated. This miscalibration manifests in two primary ways: overtrust and undertrust [19]. Overtrust occurs when users place too much confidence in a robot, potentially overlooking its limitations and risks. Undertrust, conversely, happens when users are overly skeptical of a robot’s capabilities, leading them to underutilize it or excessively monitor its operations, leading to degrading performance.

Calibrating trust to an appropriate level is not a straightforward process. It involves a complex interplay of various contextual factors, including user previous experiences, robot performance, and transparency of robot operations. Trust calibration thus requires time and consistent interaction between the user and the robot.

To address the issue of miscalibrated trust, we propose varying the way human and robot interact rather than calibrating the trust of a user ex-ante. This approach involves providing users with appropriate control and observability into the interaction with the robot accepting some inefficiency rather than risking disuse or misuse.

However, in current RPA products, the automation spectrum is typically only viewed as a binary option: attended versus unattended [3]. This simplification limits effectively managing human-robot interactions and establishing warranted trust. Furthermore, it has been shown that not all attended mechanisms are the same and just asking for users validation is not enough to increase trust in the robot and, ultimately, user acceptance [30].

Therefore, the central research question driving this exploration is: How do we design trust-aware human-robot interaction in RPA? By answering this question, we aim to create a workable solution that can eventually enable users to develop the appropriate level of trust in the robot.

To address the discussed problems, we propose the following possible approaches:

In this section, we present open issues arising from the proposed approaches that we foresee should be coped with toward the full realization of our proposal:

The consideration and design of human-robot interaction can be a crucial factor in achieving correct and effective trust calibration, particularly in the realm of RPA. By providing users with appropriate control and observability in working with robots, we have devised a proposal to deal with the issue of miscalibrated trust ex-ante. We have explored this issue using the established lens of automation levels based on classes of activities. In this way, a match between the perceived trust of the user and warranted trust can be established when the user is not yet properly calibrated to the robot to improve acceptance. By accepting a reasonable amount of miscalibrated inefficiency, we envision that user can better calibrate their trust towards the system and prevent misuse and disuse of under- or overtrusted robots.

Although our focus has primarily been on human trust in robots, the discussed concepts are also applicable to exploring the reverse – robot trust in humans. This perspective is valuable when designing machine-in/on-the-loop systems, where robots monitor and support human tasks as co-pilots. This bi-directional trust paradigm could enhance future interaction design, promoting even more effective and trustworthy human-robot collaborations.

This working group focused on the development and evolution of Robotic Process Automation (RPA) and trust in these systems as they become increasingly sophisticated. We began by examining the different types of RPA technology and their characteristics. Additionally, we identified use cases and categorized them within our classification framework. We then explored whether different types of RPA technology have varying trust requirements. Instead of concentrating on specific instantiation of RPA systems, tasks, and users, we focused on different types of RPA systems and their underlying characteristics.

We have mainly focused on intrinsic characteristics at the RPA technology level. Several relevant elements of the relationship between trust and RPA technology are instantiation-dependent, which we discussed only for selected use cases. These elements would need to be addressed more systematically to provide a complete vision of trustworthiness. As mentioned before, this can be done using the AI Trustworthiness Framework, also considering the environment and context of the specific instantiation.

Regarding the initial question of how to create trust for RPA systems, we found an interrelation between the type of RPA technology and its trust requirements. To address this, we first structured the different technology types using the dimensions of data, decision-making, and adaptability. Consequently, we propose the following four types of RPA technology: (1) Traditional RPA, (2) Cognitive RPA, (3) Intelligent RPA, and (4) Autonomous RPA. As the sophistication of RPA increases, so does the associated uncertainty. Assuming an inverse relationship between uncertainty and trust, this rising uncertainty leads to heightened trust requirements. Specific trustworthiness characteristics can address these increased trust requirements for concrete implementations. In summary, we contribute to the overall topic of trust and RPA by providing initial insights into the intrinsic trust requirements associated with the various classes of RPA technologies.

There is great agreement, that the technological capabilities of process automation solutions increase over time. Recent examples include the integration of cognitive automation [11], machine learning [25], and artificial intelligence [14]. As such, the process automation solutions become increasingly sophisticated.

Companies integrate these increasingly intelligent process automation solutions into their business processes and services [14]. Predominant goals of this automation are to improve costs, quality, time, and flexibility of the processes and services [12].

With the increased sophistication of the automation solutions integrated in processes and services, perceived ability, integrity, and benevolence changes. Anecdotal evidence includes fear of wrong decisions through AI, biases within the decisions, or malicious intents of more general AI [12]. These are, e.g., exemplified within media reports ⁵⁵5https://www.ft.com/content/979fe974-2902-4d78-8243-a0cff68e630a⁶⁶6https://www.dw.com/en/ai-germany-discrimination/a-66670854.

As ability, integrity, and benevolence form the basis of trust [22], processes and services require a higher degree of trust than before. These trust needs can be analyzed from three different perspectives:

• $\mathrm{■}$

  Managers and decision makers require trust in the process automation solution to purchase the corresponding solution and allow organizational adoption in the first place.

• $\mathrm{■}$

  Employees within the organization need trust in the process automation solution to integrate the solution into the processes and services they are responsible for.

• $\mathrm{■}$

  Customers of the organization need trust into the automated process/service so that they confidently consume this service.

However, situations emerge, where the current trust level is not enough for the increasingly sophisticated automation solution. The simplified assumption is that increased sophistication increases uncertainties (e.g., inability to explain the system’s working) and vulnerabilities (e.g., potential loss of data that now has to be provided). Here, a trust gap emerges between the current level of trust and the level of trust required by the systems.

However, without dedicated trust building there will be a threshold at the level of the actual trust. In these situations, trust signals are needed to persuade managers, employees, or customers to still adopt, implement, or consume the technology within the service. These trust signals can be interpreted as arguments to alleviate trust. Examples include indirect experiences from peers, the opinions from experts in the field, reputation of the provider of the automation solutions [29].

We propose a framework for associating trust signals with trust concerns. Trust concerns are manager, employee and customer specific. Here, we start by focusing on the trust concerns of the employee by drawing a Maslow-like triangle⁷⁷7Maslow’s hierarchy of needs is a motivational theory in psychology comprising a five-tier model of human needs, often depicted as hierarchical levels within a pyramid. (see Fig. 13) of employee concerns regarding the introduction of RPA as part of their employer’s organization, including major concerns identified by the OECD such as job security and quality⁸⁸8https://www.oecd-ilibrary.org/employment/oecd-employment-outlook-2023_08785bba-en. These trust concerns can be addressed with specific trust signals.

Further research is required on the emergence of the trust gap and its effects within the organization. Similarly, further research is needed on the effects of different trust signals. Trust signals have been studied in prior research, e.g., in e-commerce contexts, where they were shown to change consumer behavior [10]. However, a transfer to trust within advanced process automation solutions is yet needed. Employees, managers, and consumers need to have a deeper understanding of the automated processes and services that support the building of trust [6]. The role of trust signals is to help them build this understanding.