Roadmap for Responsible Robotics

As robotics and automation both matures and stalls due to hard deployment challenges, it looks increasingly likely that fully automated, unsupervised systems will only make up a subset of total robot deployments, for both capability and operational concerns. Instead much robotic deployment will occur in collaborative and semi-supervised environments, where robots and people both play a role and interact in rich and meaningful manners beyond simple supervision and oversight. To maximize both the capability of these autonomous systems as well as their collaborative potential with human operators, both present and remote, these autonomous systems will need to “know when they don”t know” – the power of introspection. Introspection enables graceful performance degradation, but also facilitates handover to human operators. From a pragmatic point of view, for any safety or operationally critical activity, a system with a certain level of performance, say in terms of accuracy, but no introspection capability will often be vastly inferior to a system with slightly lower accuracy but good introspection capability. Whilst introspection and related concepts like verification are mature, well practiced areas in domains like aerospace, their consideration and treatment in robotics is relatively early stage or not done. We propose that a substantial research investment and focus in introspection for robotics and autonomous systems will pay dividends, both in terms of advancing knowledge but particularly in enabling promising robotic technologies to successfully make the transition into trusted, enduringly deployed systems.

This talk unpacks the many ways in which the word “trust” is and has been used in robotics, human-robot interaction, and AI ethics. Between “trustworthy AI” and user trust in specific capabilities of a robot, there is a large gap and diversity of solutions to address the trust-trustworthiness problem.

Building on the current trends in AI ethics (namely, principle-based approaches toward trustworthy AI and framing of ethics issues as model/product-level fairness/transparency/accountability problems) and the temptation by roboticists to borrow much of AI ethics contents directly for “responsible robotics,” I problematize how these trends can fail us in our attempts to build generally good robotic systems.

I describe this as a ’water bottle model of trust’. I argue that responsible robotics is an exercise that should help build trustworthy robotics design norms that focus on considerate forms of design and deployment of all robots, rather than one that narrowly guides the ethical design of a single system/hardware/feature. In this process, we should challenge our existing assumptions/taboos (e.g., those related to anthropomorphization) and think about what our shared vision of the world with robots looks like.

When interacting with other non-competitive decision-making agents, it is critical for an autonomous agent to have inferable behavior: Their actions must convey their intention and strategy. For example, an autonomous car’s strategy must be inferable by the pedestrians interacting with the car. We model the inferability problem using a repeated bimatrix Stackelberg game with observations where a leader and a follower repeatedly interact. During the interactions, the leader uses a fixed, potentially mixed strategy. The follower, on the other hand, does not know the leader’s strategy and dynamically reacts based on observations that are the leader’s previous actions. In the setting with observations, the leader may suffer from an inferability loss, i.e, the performance compared to the setting where the follower has perfect information of the leader’s strategy. We show that the inferability loss is upper-bounded by a function of the number of interactions and the stochasticity level of the leader’s strategy, encouraging the use of inferable strategies with lower stochasticity levels. As a converse result, we also provide a game where the required number of interactions is lower bounded by a function of the desired inferability loss.

In philosophy, fairness has traditionally not been investigated as a moral concept in itself, but instead played second fiddle in relation to political philosophical concerns such as “distributive justice” [1]. In other words “fairness” has been utilised as a concept regarding the allocation of resources, opportunities, and so on across society. As such, it is not a concept easily and clearly defined in the literature. However, in recent years “Fairness” has been heavily deployed as a key term not as part of theoretical discussions on distributive justice, but instead in opposition to the rising concern regarding bias issues in machine learning models [2].

This working group aimed to broaden the terms of the “Fairness” debate, recognising that “Fairness” as a concept is not and should not be reduced to “Algorithmic Fairness”. As such, the group set out to investigate what further role “Fairness” might play with regards to responsible robotics, and as such whether the concept of “Fairness” may be used to identify and illuminate issues regarding contemporary robotics design and deployment. The following inter-related points were identified as potential Fairness concerns relevant to Responsible Robotics design and deployment practices:

Algorithmic Fairness: Algorithmic injustice is an ongoing concern regarding the use of machine learning models. As machine learning can also be used in robots, the ongoing algorithmic injustice debate still relevant to discussion of fairness in relation to robotics.

Fairness and Design: Fairness may play a role even on the lowest level of design. Certain components may have poorer performance for certain demographics, while design choices such as language used can affect for whom the technology is useful for.

Fairness and Accessibility: On a larger scale, the deployment of robots in public and private domains can give rise to concerns regarding financial inaccessibility. As an example, consider a robot developed as a disability aid or as a teaching tool that is prohibitively expensive for the individuals or public schools that are in need of it. As such, societal or systemic structures can give rise to fairness concerns for the deployment of robots.

For completeness, it should be noted that the work group participants throughout the session discussed and expressed concerns on the merit of Fairness as a focal key concept in the Responsible Robotics debate, particularly in comparison to either more established political philosophical concepts such as Justice, or in comparison to other motivating concepts of well-established moral nature. Hence, a conceptual issue emerged regarding the notion of “Fairness”, as the lack of current in-depth definitions results in ambiguity and lack of clarity when attempting to use “Fairness” as a guiding principle. As such, further formal investigation in the definition and dimensions of “Fairness” is needed in order to fully establish its full role in Responsible Robotics.

Responsible robotics requires that robots, as well as the people and organizations who design and deploy them, are trustworthy. This goes beyond getting people to trust robotic systems, including their human elements, which might be accomplished whether or not people have good reason for trust (e.g. through effective advertising). Rather, robotic systems must satisfy certain independent normative standards in order to warrant trust. What are these standards, and what challenges must robotic systems overcome to satisfy them? We think that trust is importantly distinct from transparency, explainability, and predictability. While these qualities can make robotic systems trustworthy, trust seems especially important where robotic systems aren’t, or can’t be, made fully transparent, explainable, and predictable. Like human-human interactions, human-robot interactions can involve unavoidable uncertainty. Also like human-human interactions, human-robot interactions can take place in physical space (as opposed to cyberspace) and therefore make immediate physical harm a real and sometimes visceral possibility. These factors necessitate a trust that’s structurally similar to trust between human actors: people often have to trust each other to behave appropriately without full knowledge of each others’ motives, intentions, capacities, propensities, needs, vulnerabilities, etc. But robots are different from humans in relevant respects too. For instance, they don’t share humans’ basic interests in avoiding pain, injury, and death; they can’t communicate with humans as humans can with each other; and they can be controlled by humans in ways that humans can’t be controlled by each other. These factors pose distinct challenges for trust in robotic systems. A promising approach to clarifying and addressing such challenges would draw on social epistemology and moral psychology, to better understand what undergirds trust(worthiness) in general, as well as engineering and organizational studies, especially human factors, to explore possibilities for designing and deploying robotic systems such that people have good reason to trust them despite, or perhaps even because of, their peculiarities.

There has been some work on defining predictability in the context of robotics, and exploring its connection to other relevant properties, namely understandability. Although not entirely consensual, the common idea of all these definitions is that predictability is about matching the expectations of the user/observer and that predictability lies in a continuum (given a goal, a robot is as predictable as its chosen plan matches the expectations of the user/observer for that goal).

Full predictability might not always be a desirable property for all different users/observers (for a robot operating in public spaces, fully predictable behavior might open opportunities for observers to abuse/bully the robot), so a key responsibility at design time is precisely to identify the level of predictability that is adequate for each stakeholder. Predictability requires the user/observer to know the goal – the design should clarify how this will be achieved, either by designing the robot to also be understandable/legible, building single purpose robots, educating the users (this one a responsibility at deployment time…), etc. Not all users have the same expectations of what is the best plan, so responsible design for predictability should incorporate in the robot some mechanism for the robot to adapt to the individual users, so that (at least) predictability improves over time, or again level everyone by educating at deployment time…

Predictability is also a technical concept: regardless of the user/observer and the robot platform, task and domain, the extent and specificity with which a robot”s actions can be predicted also varies. For example, a large robot moving with substantial inertia through the environment – such as an autonomous truck – has a highly predictable set of next step possibilities – it will continue to move in the current direction at near the current velocity, possibly with the application of acceleration or braking changing its velocity. A human observer does not need to know anything about the algorithms or control systems for the robot in order to have broad predictability for the autonomous truck – it will likely continue on its current trajectory in the next moment, but may increase or decrease its velocity and its heading may change (initially not by very much).

Continuing the autonomous truck example, another key aspect of predictability is predicting the performance of the system. For autonomous vehicles, localization – knowing where the vehicle or robot is located – is a key estimation task that enables safe navigation and higher level behaviours. One aspect of the predictability of a localization system is predictability of how well it is performing – also relating to the concept of introspection. Imagine a choice of two localization systems: one that works well 99 % of the time but is unable to predict its failures that remaining 1% of the time, versus a second system that works well 95% of the time, but is able to predict when it is performing badly 95% of the time. An autonomous vehicle using the first system will unknowingly navigate using incorrect localization information 1% of the time: using the second system, this percentage drops to 0.25%, a major different for such a safety critical application. A side note relates to the research culture of robotics and related fields currently: one key issue with research in this domain currently is that the former system is much more likely to yield a top tier publication, despite the second system having far more utility for many end-user applications.