Normative Reasoning for AI

Machine learning models have revolutionised various fields by providing highly effective solutions to complex problems. However, their success comes at the cost of unexpected behaviours, which might violate known requirements expressing background knowledge about the problem at hand. This can have dramatic consequences, especially in safety critical scenarios (e.g., healthcare/autonomous driving). In this talk, I will first give an overview of the standard performance-driven machine learning development pipeline, and then I will present our proposed requirements-driven machine learning development process, highlighting its advantages. Finally, I will argue that it is desirable to use logic to express requirements, and I will briefly discuss how different neuro-symbolic methods have been developed to incorporate both norms (or soft constraints) and requirements (or hard constraints).

We advance an alternative version of the Chisholm Paradox and we argue that the alternative version (while logically equivalent to the original version), in its manifestation in the natural language, is not intuitively consistent. The alternative version of the paradox suggests some requirements for deontic logics designed for legal reasoning.

The original intent of this talk was to highlight some problems/issues a deontic logic has to address to capture normative (legal) reasoning (including some that might be controversial). However, after some of the previous presentation, the focus shifted to one of the issues, more specifically whether it is possible to use other logic, in particular Temporal Logic to model legal reasoning. I show that using Temporal Logic, more precisely, Linear Temporal Logic, as done by some work in the area of business process compliance, leads to some paradoxical results: either it is not possible to model some deontic aspects, or the outcome of the modelling contradicts expected legal outcome.

When a computer system takes decisions that affect people, they may demand an explanation. When the application involves norms, we need a deontic explanation. An deontic explanation is analyzed here as an answer to a why-question, relative to a normative system. Just like a who-question asks for persons, a why-question asks for reasons. We analyze differences and similarities of three kinds of semantics, that are formulated in terms of a partition of the set of possible worlds: (1) questions and answers, (2) moral dilemmas, and (3) see-to-it-that choice structures. The analysis is built on an analogy between providing an answer to a question, resolving a moral dilemma and choosing an action. The role of the context in these types of semantics can be naturally analysed in a form of update semantics. In future work we hope to find constructions in the object language, to construct or reframe a question, a choice for action, or a dilemma, and ways of answering or resolving them.

A normative reason is a consideration that counts either in favor of or against an action or attitude. A second-order normative reason, then, is a consideration that counts in favor of or against taking another consideration to be a normative reason. While some authors have questioned the existence of such reasons, others assign them very important roles. Thus, exclusionary or negative second-order reasons – that is, reasons against taking other considerations to be reasons – play a crucial role in Joseph Raz’s account of practical reasoning. The primary goal of this talk is to show how second-order reasons and their normative effects can be captured in default logic. Starting with Horty s default logic-based model of the way reasons interact to support ought statements, I explain why one can’t rest content with Horty’s formalization of exclusionary reasons. Most importantly, it assimilates defeat by exclusionary reasons to canceling (or undercutting), doesn’t do justice to the idea that excluded first-order reasons remain valid, and doesn’t account for a distinct sense of “ought” grounded in first-order reasons. I discuss an alternative model, present an account of positive-second order reasons, and explore the model’s predictions regarding the structure of even higher-order reasons and conflicts between them.

The talk shows how non-classical logics with special emphasis on modal logic, epistemic logic and conditional logic  can be used to represent and compare a rich variety of explanations of classifier systems; these include abductive, contrastive, counterfactual,  objective vs subjective, and interactive explanations. The first part of the presentation will be devoted to explaining “white box” classifiers that are assumed to be perfectly known, while the second part will focus on “black box” classifiers about which the external observer has only partial knowledge.  I will present proof-theoretic and complexity results for the involved logics and illustrate their expressiveness through concrete examples.

Norms are indispensable for human communities, and so they will be for robot-human communities. We analyze some of the requirements for a robot to have norms and conform its actions to them. These requirements include both cognitive and social properties that human norms have. We examine which of these properties can be implemented in a robot’s architecture and review some previous computational approaches. We then introduce a new one using behavior trees, argue for its promise to implement properties of norms, and discuss unsolved challenges.

This work lays the groundwork for a (proper) mechanisation of normative reasoning, via the use of a so-called analytic sequent calculi. They are particularly useful for backward reasoning and deontic explanations. To answer a question of the form “why should I do X?”, needed is to retrieve the path (the “proof”) leading to this conclusion. Analytic calculi allow precisely this.

This is part of a bigger project aiming at developing analytic proof systems for deontic logic formalisms.

We consider a SoA formalism, the preference-based system E for conditional obligation due to Aqvist. Its key strength lies in its ability to resolve the CTD paradox. We provide an analytic calculus for it, the first of its kind. We also provide a terminating countermodel generation procedure in case of failure of proof search, and a complexity result (co-NP).

Deontic logic is a family of often propostional modal logics intended to capture certain kinds of moral reasoning, centrally those involving obligation. Deonitc logic has receive a great deal of attention from philosophical logicians and there has been some interest in developing implementations of reasoning procedures for various deontic logics. However, much of the work has rested on axiomatic approaches with the inference rules of necessitation and modus pones. While familiar and convenient for some communities, these not a standard basis for robust implementations.

Description logics are a widespread family of decidable logics, the core of which can be seen as notational variants of propositional modal logics. The description logic $𝒮\mathscr{H}ℛ𝒪ℐ𝒬$ forms the logical foundation of the standardised ontology language OWL 2 DL. OWL 2 has broad and deep infrastructure including production quality reasoners, IDEs, services, repostiories, and so on.

Give the very expressivity and wide range of available tools, description logics are an attractive foundation for a translational approach to implementing reasoning services for deontic logics.

We present various translations of several logics of obligation and agency into OWL 2. While all the translations preserve the meaning (at least in the sense of always being able to recover all entailments) of the original, they have different characteristics which affect the performance of automated reasoning tasks and the utility of the results for users.

We also explore the benefits of an ontology-oriented approach to modeling deontic problems.

AI-based processes are paving the way to fully automated autonomous driving. Up until now, the development of AI solutions has been purely driven by data. This data driven approach requires enormous amounts of data for the training and validation of AI functions, with the collection and processing of this data being very resource-intensive and expensive. In addition to the dependence on extensive amounts of data, data-based AI processes have another weakness: they are still generally black-box models for which the decision making process cannot be directly reconstructed. In the talk I will report about the project “KI Wissen” (https://www.kiwissen.de/) and neuro-symbolic methods for integrating existing knowledge into the data-driven AI functions of autonomous vehicles (AVs) and vice versa for extracting interpretable symbolic knowledge from deep neural network models. In this talk I will specifically present an approach for extracting interpretable hierarchical rules from the learned AVs’ deep neural networks and a neuro-symbolic architecture for a hybrid integration of deep neural networks, modelling the AVs’ behaviour and situation information, with symbolic knowledge models for representing ontological domain and world knowledge for situation interpretation and for representing rule-based legal knowledge and norms for legal reasoning and compliance checks. The goal of the KI Wissen project is to create a comprehensive ecosystem for the integration of knowledge into the training and safeguarding of AI functions. By combining conventional data-based AI methods with the knowledge- or rule-based methods developed in the project, the basis for training and validating of AI functions will be completely redefined: This basis now includes not only data, but information, i.e., data and knowledge. The development from data- to information-based AI carried out in the project addresses the central challenges towards autonomous driving: the generalization of AI to phenomena with small data bases, the increase of the stability of the trained AI to disturbances in the data, the data efficiency, the plausibility check and the validation of AI-supported functions as well as the increase of the functional quality.

One fundamental question lies behind the distinction between justification and explanation in normative reasoning. In fact, we must notice that the tradition of legal logic and legal theory, in modeling legal decision-making, very often elaborate on various types of justification and takes this last concept as central, somehow maintaining that the idea of explanation depends on justification: while the explanation of a legal decision does not necessarily correspond to a justificatory reason for it, the opposite usually holds. The talk will offer some formal insights about this topic.

We show how to combine deep NLP with nonmonotic reasoning in legal domain. We extract legally relevant facts from a case description written in natural language using deep NLP and input these facts into manually encoded articles in our legal logic programming language PROLEG to make legal judgement and produce explanation of the judgement.

What are social norms? There is a surprising level of disagreement about this question in the literature. I compare Bicchieri’s (2003) game-theoretic account with Brennan, Erikson, Goodin, and Southwood’s account of norms as cluster of attitudes. Both accounts agree that real or perceived social practices and normative expectations play a central role. However, examples show that the different accounts can come apart.

The problem of developing explainable AI methods is becoming increasingly central in AI. At the same time, the notions of explanation, explainability, and justification have been extensively investigated in philosophy, law, and social science. As a result, an increasing number of scholars from these disciplines is considering how to apply research in these fields to explainable AI. One problem of this interdisciplinary effort is that, more often than not, researchers from different fields have different understandings of what the task underlying explainable AI is (or should be) or what exactly “explanation” or “justification” mean when applied to AI systems. This working group aimed at identifying some key distinctions that could be used to build a unified conceptual framework for explainable AI. The discussion was split into two parts:

When philosophers talk about normative matters – about what is right, obligatory, permitted, and so on – they tend to rely on the notion of a normative reason. In the practical domain – which includes morality, as well as the domain of practical rationality – normative reasons are understood as considerations that count in favor of or against actions. The notion has become a mainstay of philosophy, where it is very often relied on in answering various normative and metanormative questions. The so-called reasons-first program takes this to the extreme, taking the notion of a normative reason to be basic and holding that all other normative notions are to be analyzed in terms of it [1, 2, 3]. When discussing the interaction between reasons, the philosophical literature uses such phrases as “the action supported on the balance of reasons” and “reasons for outweigh reasons against”, inviting an image of a weighing scale. Philosophers have explored various ideas about the exact workings of this normative weighing scale, with rare exceptions, their investigations have been carried out informally. The overall goal of this breakout session, then, was to model normative reasons and their interaction – roughly, the normative weighing scale for reasons – using methods from mathematical modeling and knowledge representation.

Toward the end of the discussion, the following model emerged:

A structure $T=⟨P,I,F,f,N,D⟩$ is a model of reasons, where:

• $\mathrm{■}$

  $P$ is a set of persons;

• $\mathrm{■}$

  $I$ is a set of issues;

• $\mathrm{■}$

  $F$ is a set of features;

• $\mathrm{■}$

  $f:P\times I\to 2^F$ is a function mapping pairs of persons and issues to a set of features, indicating which features serve as normative reasons in determining the normative status of the issue for the given person;

• $\mathrm{■}$

  $N$ is a set of polarity functions, with each element $n$ of $N$ having the form $n:P\times I\times \{f\}\times F\to \{+,-,0\}$ (as their name suggests, these functions determine the polarity or directedness of features);

• $\mathrm{■}$

  $D$ is a set of deontic functions, with each element $d$ of $D$ having the form $d:P\times I\times f\times N\to \{+,-,0\}$ (as the name suggests, these functions determine the normative status of issues for persons: intuitively, an assignment of $+$ means that the issue (action) is obligatory for the person, that of $-$ means that it is forbidden, and that of $0$ means that it is indifferent).

It was noted that, its simplifying assumptions notwithstanding, this model captures important parts of philosophers’ way of thinking about normative reasons, the interaction between reasons (or weighing reasons), and the relation between reasons and such normative notions as obligations and permissions. It was also noted that this model is only a starting point, and that more structure can be added to it with ease. For instance, one could make deontic functions depend on additional arguments (representing other normatively relevant information), or one could allow for multiple types of deontic functions (representing different types of obligations). One could also substitute the polarity functions with numerical functions, with the result that the features identified as reasons for (against) an issue would be associated not only with polarities, but also with magnitudes, bringing the model even closer to the metaphor of weight scales. The affinities with the field of multi-criteria decision-making [4] were also noted.

The discussion participants agreed that the model can be used to formalize various sorts of methodological questions, making them (more) tractable. Questions prompted by the talks delivered at the seminar served as examples: What is the role of polarity (or “directedness”) in reason-based decisions? Are Raz’s views on reasons in the practical domain equivalent to Pollock’s views in the epistemic domain?

For completeness, it should be added that the breakout session participants also voiced some reservations toward the model. Thus, it was noted that the model does not (yet) represent normatively relevant considerations that are not reasons – including conditions and modifiers – which are widely discussed in the philosophical literature. Another issue that was noted was that the model allows one to represent only situations of binary choice. Still, all participants agreed that these issues can be overcome.

As we assign more roles to RL-based agents it becomes crucial to ensure that they act in ways that are legal, ethics-sensitive, and socially acceptable. This introduces a further challenge: establishing boundaries around the behaviour of these agents, i.e. equipping them with the ability to comply with legal, ethical and social norms, while still enacting pre-learned optimal behaviour.

We have recognized the different approaches that emerge and did thoroughly discuss them. The first approach uses symbolic AI techniques (a.k.a. Logic, Knowledge Representation and Reasoning) and was successfully employed, e.g., in [2] where a theorem proved for a defeasible deontic logic advises the learning agent on the compliant actions; this approach can however be computationally expensive and less suited for dealing with (signal-based) data, sensory input, or stochastic environments. The other approach relies on sub-symbolic AI (a.k.a. Machine Learning), and was applied, e.g., in [3], to constraint the behaviour of AI agents via reward/penalities; this approach excels under these conditions and enables the construction of efficient and adaptable AI systems, which however lack modularity and transparency; moreover, it is not clear how to adapt this approach to deal with complex normative systems.

All participants agreed that the best way to proceed would be to interlace the two approaches, thus providing the best of both worlds. This breakout session consisted of two parts.

This scenario seems to repeat itself: A company creates a service that uses artificial intelligence and/or has some agency. We will refer to this service very generically as “a machine”. In its interaction with the users, this machine inevitably violates a norm. The company responds with a constraints that disables the machine from violating the norm, effectively turning the norm into a constraint. Then a new situation arises in which the machines constrained behaviour violates another norm. This practice does not result with a machines ability to operate in a normative context, but rather with a system whose behaviour is neither desirable, nor predictable. To be able to move away from this trap of update and adjust, we first need to ensure there is an understanding on what instruments are available for adjusting and guiding the behaviour of machines.

The work group discussed the basic concepts in normatively regulating the behaviour of machines and artificial agents, as well as the basic approaches. machine:

• $\mathrm{■}$

  Functional level: the machine is either constrained to operating in an environment in which norm violation cannot happen. Of course unintended norm violations can never be ruled out entirely.

• $\mathrm{■}$

  Normative level: the machine is provided with norms that reduce the action space to those actions that comply with the norms (most likely, most of the time).

• $\mathrm{■}$

  Value level: the machine is provided with values with which it needs do align its choice of norm-guided actions, especially when norm conflicts arise.

What intervention we choose to do depends on many factors. The aim of the group is provide a joint article that can be used as an interface to the state of the art in the field of normative reasoning withing multi-agent systems.