Cognitive Robotics

Robotic agents that can accomplish manipulation tasks with the competence of humans have been the holy grail for AI and robotics research for more than 50 years. However, while the fields made huge progress over the years, this ultimate goal is still out of reach. I believe that this is the case because the knowledge representation and reasoning methods that have been proposed in AI so far are necessary but still too abstract. In this talk I propose to endow robots with the capability to mentally “reason with their eyes and hands,” that is to internally emulate and simulate their perception-action loops based on photo-realistic images and faithful physics simulations, which are made machine-understandable by casting them as virtual symbolic knowledge bases. These capabilities allow robots to generate huge collections of machine-understandable manipulation experiences, which they can then generalize into commonsense and intuitive physics knowledge applicable to open manipulation task domains. The combination of learning, representation, and reasoning will equip robots with an understanding of the relation between their motions and the physical effects they cause at an unprecedented level of realism, depth, and breadth, and enable them to master human-scale manipulation tasks. This breakthrough will be achievable by combining simulation and visual rendering technologies with mechanisms to semantically interpret internal simulation data structures and processes.

We are interested in the problem where a number of robots, in parallel, are trying to solve reaching through clutter problems in a simulated warehouse setting. In such a setting, we investigate the performance increase that can be achieved by using a human-in-the-loop providing guidance to robot planners. These manipulation problems are challenging for autonomous planners as they have to search for a solution in a high- dimensional space. In addition, physics simulators suffer from the uncertainty problem where a valid trajectory in simulation can be invalid when executing the trajectory in the real-world. To tackle these problems, we propose an online-replanning method with a human-in-the-loop. This system enables a robot to plan and execute a trajectory autonomously, but also to seek high- level suggestions from a human operator if required at any point during execution. This method aims to minimize the human effort required, thereby increasing the number of robots that can be guided in parallel by a single human operator. We performed experiments in simulation and on a real robot, using an experienced and a novice operator. Our results show a significant increase in performance when using our approach in a simulated warehouse scenario and six robots.

In this work, the problem of bootstrapping knowledge in language and vision for autonomous robots is addressed through novel techniques in grammar induction and word grounding to the perceptual world. In particular, we demonstrate a system, called OLAV, which is able, for the first time, to (1) learn to form discrete concepts from sensory data; (2) ground language (n-grams) to these concepts; (3) induce a grammar for the language being used to describe the perceptual world; and moreover to do all this incrementally, without storing all previous data. The learning is achieved in a loosely-supervised manner from raw linguistic and visual data. Moreover, the learnt model is transparent, rather than a black-box model and is thus open to human inspection. The visual data is collected using three different robotic platforms deployed in real-world and simulated environments and equipped with different sensing modalities, while the linguistic data is collected using online crowdsourcing tools and volunteers. The analysis performed on these robots demonstrates the effectiveness of the framework in learning visual concepts, language groundings and grammatical structure in these three online settings.

Autonomy represents a step-change in systems development and requires new approaches to system architectures, to systems analysis and to effective usage.

In this presentation, I describe an approach that utilises the modularity and heterogeneity of (robotic) software architectures to provide a hybrid agent architecture. Then, a range of verification techniques can be applied to the different components, from formal verification applied to the core autonomous decision-making through to varieties of testing used in other parts of the system.

Finally, an important component is the use of runtime verification (or runtime monitoring) to check for anomolies and violations. Together, these mechanisms provide a basis for more relaible, transparent, trustworthy and verifiable autonomous systems.

Recent breakthroughs in AI have shown the remarkable power of deep learning and deep reinforcement learning. These developments, however, have been tied to specific tasks, and progress in out-of-distribution generalization has been limited. While it is assumed that these limitations can be overcome by incorporating suitable inductive biases in neural nets, this is left vague and informal, and does not provide meaningful guidance. In this talk, I articulate a different learning approach where representations are learned over domain-independent target languages whose structure and semantics yield a meaningful and strongly biased hypothesis space. The learned representations do not emerge then from biases in a low level architecture but from a general preference for the simplest hypothesis that explain the data. I illustrate this general idea by considering three learning problems in AI planning: learning general actions models, learning general policies, and learning general subgoal structures (“intrinsic rewards”). In all these cases, learning is formulated and solved as a combinatorial optimization problem although nothing prevents the use of deep learning techniques instead. Indeed, learning representations over domain-independent languages with a known structure and semantics provides an account of what is to be learned, while learning representations with neural nets provides a complementary account of how representations can be learned. The challenge and the opportunity is to bring the two approaches together.

Due to the challenges of perception and action, and inevitable inaccuracies in world modelling, the results of a robot’s interactions with its environment are inherently stochastic. To successfully complete extended missions under such conditions it is therefore essential that autonomous robots use techniques from decision-making under uncertainty to plan goal-directed behaviour. In this talk I will give an overview of our recent work on planning under uncertainty for autonomous robots, drawing examples from mobile service robots, underwater vehicles, and quadrupeds.

In this overview talk I address some of the main representation and reasoning techniques that have been used in robotic systems. On the representation side, these include simple databases (logical literals), description logics, and geometric or topological maps with semantic annotations. On the reasoning side, we find methods for temporal, spatial, and uncertainty reasoning as well as automated planning techniques. I also touch upon the need for execution monitoring and failure diagnosis. At the end of my talk I briefly introduce the RoboCup Logistics League, where robots interact with machines in a production logistics scenario and which can serve as a benchmark for applying KR in robotics, both in simulation and on real robots.

Neurosymbolic AI aims to bring together the statistical nature of machine learning and the logical essence of reasoning in AI systems. Recently, leading technology companies and research groups have put forward agendas for the development of the field, as modern AI systems require sound reasoning and improved explainability. In this talk, we highlight Neurosymbolic AI research results that led to applications and novel developments towards building richer AI systems. We summarize how the field evolved over the years and how it can potentially contribute to improved explainability and the effective integration of learning and reasoning in robust AI.

Robots deployed today largely perform a predefined set of tasks in limited, controlled environments. In order to handle the complexity of human-centric spaces, it is necessary to learn about the world and tasks from human end users, and natural language is a key modality for such learning. Two high level approaches to understanding and learning from such language are, first, learning probabilistic grammars describing the perceptual state of the world and, second, learning directly from speech, without any textual intermediary. This talk describes work on using a combination of language and perceptual data to learn about how people describe objects in the world, with the long-term goal of understanding tasks and instructions presented in natural language by non-specialist end users. The importance of using speech directly is discussed, and the effectiveness of using featurized speech is compared to ASR-based approaches. Using speech not only improves performance on the language grounding task, but also reduces performance differences among different demographic groups, leading to more immediately deployable robotic systems.

Reinforcement Learning (RL) is proving to be a powerful technique for building sequential decision-making systems in cases where the complexity of the underlying environment is difficult to model. Two challenges that face RL are reward specification and sample complexity. Specification of a reward function – a mapping from state to numeric value – can be challenging, particularly when reward-worthy behaviour is complex and temporally extended. Further, when reward is sparse, it can require millions of exploratory episodes for an RL agent to converge to a reasonable quality policy. In this talk I’ll show how formal languages and automata can be used to represent complex non-Markovian reward functions. I’ll present the notion of a Reward Machine, an automata-based structure that provides a normal form representation for reward functions, exposing function structure in a manner that greatly expedites learning. Finally, I’ll also show how these machines can be generated via symbolic planning or learned from data, solving (deep) RL problems that otherwise could not be solved.

Model learning can primarily be characterized across three dimensions: (1) the input data format; (2) the output model components; and (3) the priors/partial models that we start with. Here, we explore two settings where model learning for planning has been studied.

First, we detail the Model Acquisition Toolkit (MACQ): a library dedicated to learning action theories from state traces of various forms. Each technique in the library comes with its own priors, but collectively the library provides the most comprehensive treatment to date of extracting action theories from discrete time series data.

The second work explores how strong priors influenced by planning concepts can aid in learning planning models from image pairs alone. By embedding strong notions of action representation into the learning architecture itself, we are able to learn action theories and state representations that can be given to off-the-shelf planners.

These are but two modern examples of how model learning is being explored in the context of planning.

This talk does not report on research results, but rather on perspectives of how hardware acceleration can be exploited for automatic planning. Focusing on RPG-style heuristics, it is sketched how such heuristics estimators can be compiled into sequential circuits for moderately large planning tasks, which opens up the possibility to implement that on standard FPGAs. Since 80-90% of the compute time in planning systems is spent on computing heuristic estimates, this could result in a speedup of one order of magnitude.

Autonomous robots that can assist humans in situations of daily life have been a long standing vision of robotics, artificial intelligence, and cognitive sciences. A first step towards this goal is to create robots that can learn tasks triggered by environmental context or higher level instruction. However, learning techniques have yet to live up to this promise as only few methods manage to scale to high-dimensional manipulator or humanoid robots. In this talk, we investigate the challenges for robot learning from both the symbolic and subsymbolic perspective! We show how symbols can arise in a robot learning system and can used to further the general application of robot learning. We also discuss how classically disjunct approaches from first order insight can be used as inductive biases for faster learning using the simulation based approach. We describe the work in various robotic scenarios ranging from tactile manipulation to robot juggling.

Robot ethics is no different from bioethics, information ethics, environmental ethics, etc. in that as a technology it has impact on humans human societies. It is different from all other technologies in that AI enables the development and deployment of autonomous systems that perceive their environment and determine their actions without human ado. AI/robot ethics thus raises the question of whether these systems can operate in human societies and interact with humans in a way that is ethical and acceptable to humans, not causing any harm. For this, robots need to be able to learn human norms from observations and instructions and follow them. When norm conflicts arise, they need to be able to determine the best course of action and justify their choices by appealing to principles used for their decisions. How to build a robotic architecture capable of all of this is the main challenge of ethical HRI!

Machine learning is a powerhouse in information rich environments. However, machine learning remains challenging when data is sparse, is costly to collect, and is dangerous and complex to acquire. As two examples, ocean exploration and subsea inspection use autonomous vehicles to perform information gathering, to answer questions about the environment. In these applications, communication is limited, vehicles need to be autonomous, environments are risky, and resources are constrained.

Our vision is to create systems that answer information queries by performing active learning in risky environments. These systems 1) generate information gathering plans that bound risk, while maximizing information with respect to a set of questions being asked, 2) continuously adapt plans based on what is observed and what remains unanswered and 3) incorporates informative measures and risk within operational plans, at multiple levels of abstraction.

The talk introduces a series of model-based agent programming paradigms that support this process of active learning in risky environments, starting with state and decision-theoretic programming. The talk then focuses on planning and learning methods that are needed to support two new programming paradigms – information theoretic and risk-aware programming. These approaches are demonstrated in the context of a 2019 ocean campaign, to explore the Columbo volcano in the Mediterranean Sea.

Knowledge Representation and Reasoning (KR) has been a concern in cognitive robotics for many years, beginning with the robot Shakey developed at SRI in the late sixties. While ontological knowledge, formalized using description logics, and automated planning systems, among other things, can be found in many robotic applications, KR has yet to play a central role in building cognitive robots. In this working group, we discussed and collected some of the challenges that remain in order to leverage the true potential of KR for cognitive robotics. The following lists the main findings and recommendations.

Verification and validation of complex cognitive robots is very challenging and existing methods, mainly from formal verification, can only be applied to relatively simple cases. This section summarizes the challenges, directions for future research and provides a roadmap towards verification of cognitive robots.

Human-robot interaction (HRI) with its many facets and interdisciplinary nature is of key importance for cognitive robotics, with ethical concerns playing an important role as well. In this working group challenges for HRI and robot ethics were discussed and collected along several dimensions: humans modeling robots and vice versa, norms, communication and information flow, and proactive behavior. In the following, we summarize our main findings.

Planning and learning have traditionally been two separate research tracks within cognitive robots. Lately, several research groups have started to study the combination and integration of planning and learning. For example learning symbols or primitives from observations. To achieve this, it is important to use the right inductive biases in learning to ground the AI system in the world. The key to complex behavior is being able to compose these into more complex plans or composite behaviors, thus planning based on these learned primitives clearly adds a significant value. This section provides a roadmap to achieve this in the form of three short-term challenges and four longer term challenges.