Generalization by People and Machines

Artificial Intelligence (AI) has been experiencing significant advances in the last 10 years, including image and video understanding and natural language dialog systems. AI promises to disrupt a wide range of applications and industries from self-driving cars and intelligent transportation to drug discovery and healthcare. However, existing AI technology suffer from major limitations in terms of generalization to real-world scenarios and real-world data. In the first part of this talk, I will be discussing several limitations of computer vision systems, as one important modality of AI systems. For example, the performance of computer vision systems trained to classify, detect and/or segment a set of object classes degrades rapidly when these systems are deployed in new environments where the statistical distribution and/or characteristics of the data is different than training data. Meanwhile, computer vision systems with continual learning capabilities struggle to differentiate between outliers of known classes (e. g., unusual fish or bird) and samples from completely new classes (e. g., new biometric face spoofing attack) that should be incorporated into the AI system. Further, continual learning system often suffer from catastrophic forgetting, when learning new classes and/or adapting to new data distributions leads to performance degradation on already learned classes/distributions. In the second part of the talk, I will discuss a hybrid NeuroSymbolic artificial intelligence architecture that mitigates the limitations of existing AI systems and leads to better generalization and reasoning capabilities, when AI systems are deployed in new real-world environments

In this presentation, the term generalization refers to the ability of adaptive systems, for instance, neural networks, to apply a rule that is learned from training examples to novel, unseen data in the working phase. The statistical physics approach to learning theory is outlined very briefly. It complements other theoretical frameworks and has re-gained significant interest due to the growing popularity of neural networks and machine learning in general. The computation of typical learning curves in so-called student teacher model scenarios is exemplified in terms of training layered networks by stochastic optimization of an objective function. Assuming training from randomized data sets, the average generalization ability is computed as a function of the training set size, for instance, the number of available examples. As an important example result, the existence of phase transitions in batch training is discussed: here the generalization ability improves suddenly at a critical data set size. Similarly, the analysis of the training dynamics of stochastic gradient descent reveals the existence of plateau states which can dominate the training process. They are left by means of rapid changes of the generalization ability with time and can lead to cascade-like learning curves.

How does the human ability to generalize work? This talk examined two sources of this capability. The first are qualitative representations, which provide abstract causal and spatial models that are easier to learn than detailed quantitative models. The second is analogy, where the process of analogical matching provides a means of constructing generalizations by identifying what is common across a set of examples. The talk outlined Gentner’s structure-mapping theory, the analogy stack consisting of cognitive models of matching, retrieval, and generalization, and how these have been used in a variety of cognitive simulations and performance-oriented AI systems. The Continuum of Knowledge Hypothesis was discussed, which proposes that knowledge starts out concrete and is incrementally and partially abstracted in stages. Finally, a set of open questions was discussed.

Generalization is defined as transfer of what has been learned in one context to a new one which is similar. Representation and how similarity is assessed are crucial for generalization. Generalisazion can involve abstraction of general characteristics (deleting irrelevant and constructing more general features) for a collection of entities. In the talk I give an introduction to classic theoretical approaches and empirical findings from cognitive science with a focus on concept learning. Open questions, from my perspective, are: (1) The relationship between generalisation and representation: Where does structure come from? What is the human inductive bias which leads to useful generalizations? (2) What is the relation between implicit and explicit learning?

Generalization in KRR is usually defined with respect to a logical framework where background knowledge, also known as inductive biases, is expressed through sentences of a logical language. The adopted logical framework provides an inference mechanism to check logical consequence and a background theory (set of formulas) to explicitly state assumptions (inductive biases). Within this framework, four main processes of generalization can be formally defined. The first process is Predicate Invention: this involves identifying a set of elements in the domain of interest and defining the necessary and sufficient conditions for membership in this set. The second process of generalization occurs during Clustering: given a set of individuals $S$ with associated properties, the goal is to find a partition $S_1,\mathrm{\dots },S_k$ of $S$ based on the similarity of their properties. Subsequently, extend the language with new symbols for each cluster in the partition. Another process of generalization is Subsumption: starting from classes $C_1,\mathrm{\dots },C_k$, introduce a superclass $S$ that subsumes each $C_i$, meaning that every instance of $C_i$ is an instance of $S$, and optionally, every instance of $S$ is an instance of some $C_i$. A further generalization method is called Rule Mining: given a set $F$ of ground facts about a subset of individuals $S$, find a set of lifted rules that hold for a larger set of individuals $S^{\prime }\supset S$. Building Analogies is another generalization process found in the KR literature. In this case, given a base domain $B$ and a target domain $T$, find a mapping $\alpha$ between the objects of $B$ and $T$ that preserves relational structure. Finally, the operation of extending a formal theory is also a generalization process where a theory $T$ is expanded by adding new symbols to the language of $T$, providing a new set of axioms $T^{\prime }$ that relate the new symbols to the existing ones, and then identifying a condition $C$ such that: $T\vDash \varphi$ if and only if $C,T^{\prime }\vDash \varphi$.

Out-of-distribution generalization in natural language processing is the ability of models to solve examples from a different distribution of the training data, based on prior knowledge and similarity to training examples. This includes robustness to prediction when introducing superficial changes to the input; and updating the prediction when introducing semantic changes to the input. Lack of generalization makes models brittle and unsafe to deploy for real-world applications. Current evaluation methods for generalization include cross-dataset evaluations and adversarial examples. In terms of making models more generalizable, there are several model enhancements such as partial model updates, neuro-symbolic and compositional models, and training on fewer examples (such as few-shot learning). From the data perspective, training on more data or specifically on adversarial examples can make models more robust. LLMs are exceptionally general and versatile, given their training on vast amounts of raw text. They are to some extent able to generalize to new concepts and ideas. However, they still over-rely on similar training examples, and are brittle when these examples are manipulated. They are still not robust to changes in the prompt phrasing. There is no evidence that they are capable of causal reasoning. Finally, testing generalization in LLMs is tricky without access to the training data.

The group on methods for generalization identified three families of methods:

1. 1.

   symbolic, for instance, predicate invention, semantic clustering, subsumption, rule mining,

2. 2.

   statistical, for instance, machine learning methods, generative AI, representation learning, and

3. 3.

   combinations of the two, for instance, neuro-symbolic methods, embedding-based methods.

Generalizations can be learned directly from data, or they can be obtained from a combination of data and knowledge. Key considerations about methods include

• $\mathrm{■}$

  provable properties of generalizations including worst-case guarantees,

• $\mathrm{■}$

  context sensitivity of generalization,

• $\mathrm{■}$

  methods for explainability,

• $\mathrm{■}$

  compositionality,

• $\mathrm{■}$

  quantifying the trade-off between compression, memorization and forgetting,

• $\mathrm{■}$

  evolving generalizations over time, and

• $\mathrm{■}$

  choice of appropriate representations.

This group was coordinated by Frank van Harmelen and Barbara Hammer.

Machine-learning based systems and humans both are capable of generalizing from examples. However, generalization capabilities appear to differ significantly, with complementary strengths and weaknesses. For example, humans are generally good at commonsense reasoning, using structured knowledge, and handling out-of-distribution data. Machine learning excels at objectivity (at least based on the data given), at scale, and at high complexity. This complementarity gives opportunities for human-machine teaming, with each side addressing the limitations of the other. For example, some generalization capabilities of LLMs, like the quick production of rhetorically polished texts on any topic, are beyond that of most humans. Yet, they make generalization errors (called “hallucinations”) like the replacing of specific facts with non-factual information; an error easily caught by a knowledgeable human. But such human-machine teaming breaks down if the human is not a topic expert.

The need for teaming arises naturally in complex application scenarios, for instance, automotive driver assistance or complex decision making. For these, it is of central importance that the human can assess machine responses, for example, has access to the rationales (called “explanations”) on the basis of which the machine responded. Future XAI research must prioritize understanding human cognition because effective human-AI collaboration requires explanations that bridge the explanatory gap between human reasoning and AI’s internal workings.

A critical challenge lies in reconciling fundamentally different reasoning paradigms: human causal models versus AI’s deep learning associations. Can these approaches be unified into a common explanatory language? Furthermore, fostering successful human-AI teams necessitates AI’s ability to learn and potentially retain feedback indefinitely. Robust feedback mechanisms are crucial for AI to understand effective communication and align with human cognition, fostering seamless collaboration. Future research should also prioritize the investigation of human generalization and abstraction processes and contrast those with AI-based approaches. Interdisciplinary collaboration between computer and social sciences will be essential to integrate this understanding into AI design, not only enhancing explainability but also mitigating biases in machine learning generalization. This group was coordinated by Pascal Hitzler, with help from John Shawe-Taylor.

The generalization group identified certain challenges with evaluation of generalization, such as:

• $\mathrm{■}$

  inadequate data splitting practices,

• $\mathrm{■}$

  hard to define the bounds of generalization,

• $\mathrm{■}$

  no comprehensive way to evaluate the total phenomena,

• $\mathrm{■}$

  limited metrics,

• $\mathrm{■}$

  consolidation challenges of discriminative and generative evaluation,

• $\mathrm{■}$

  measuring tradeoffs between predictive power and efficiency,

• $\mathrm{■}$

  evaluating long-tail phenomena, and

• $\mathrm{■}$

  selection of the right granularity for generalization.

Emerging practices for evaluation include

• $\mathrm{■}$

  cross-benchmark evaluations,

• $\mathrm{■}$

  testing robustness to perturbations,

• $\mathrm{■}$

  evaluations of over- and under-generalization,

• $\mathrm{■}$

  evaluation with multiple metrics, and

• $\mathrm{■}$

  factoring out memorization.

Many important questions were identified as relevant future works, including

• $\mathrm{■}$

  the design of checklists,

• $\mathrm{■}$

  evaluations of different levels of similarity,

• $\mathrm{■}$

  clear definition of bounds of generalization, and

• $\mathrm{■}$

  quantification of variations in performance.

This group was coordinated by Filip Ilievski and Sascha Saralajew.

There are at least three types of generalization in the broader context of cognitive science and artificial intelligence research:

1. 1.

   Generalization refers to a process by which general concepts and rules are constructed from example data.

2. 2.

   Generalization refers to the product of such a process, meaning the general concepts and rules themselves, in their diverse representations.

3. 3.

   Generalization refers to the application of a product to new data.

The types of Generalization group dove deep into these three types and their sub-types, drawing on prior work from cognitive science, symbolic artificial intelligence, and machine learning. Key theories of generalization deal with abstraction, adaptation, domain extension, composition, analogy/transfer, and in vs. out of distribution. Important representations of generalization are:

• $\mathrm{■}$

  symbolic rules,

• $\mathrm{■}$

  prototypes/exemplars,

• $\mathrm{■}$

  probabilistic distributions, and

• $\mathrm{■}$

  functional mappings.

The coordinator of this working group was Benjamin Paassen.