Strategies for creating new informational primitives in minds and machines

Abstract

Open-endedness is an important goal for designing systems that can autonomously find new and expected solutions to combinatorically-complex and ill-defined problems. Classically, issues of open-ended generation of novelty in the universe have come under the rubric of the problem of emergence. We distinguish two modes of creating novelty: combinatoric (new combinations of existing primitives , “resultants”) and creative (new primitives, “emergents”).  Although combinatoric systems may differ in numbers of possible combinations, their set of possibilities is closed. Creative systems, on the other hand, have open-sets of possibilities because of the partial- or ill-defined nature of the space of possible primitives. The two conceptions provide two modes for describing and understanding change and creativity: as the unfolding consequences of fixed combinatorial rules on bounded sets of pre-defined primitives or as new processes and interactions that come into play over time to define new primitives.
We face several related problems. We want to know how to recognize creative novelty when it occurs (the methodological problem), we want to understand the creative process in humans and other systems (the scientific problem) such that creativity in human-machine collaborations can be facilitated and that creativity in autonomous devices can be usefully enabled (the design problem).
The methodological problem can be solved by the “emergence-relative-to-a-model” approach in which an observer forms a model of the behavior of a system. Novelty and creativity are inherently in the eye of the observer, i.e. relative to some model that specifies possible alternatives. If the behavior changes, but it can still be predicted/tracked in terms of the basic categories/state set of the model, one has combinatorial creativity. If it changes, but requires new categories/observables/states for the observer to regain predictability, then one has emergent creativity (creation of new primitives).
	We argue that pure computation by itself can generate new combinations of symbol primitives, but, absent states or processes that are hidden from the observer, it cannot autonomously generate new primitives. Breakout strategies are therefore required. In order for a computational system to transcend the limitations of its own primitive symbol set, it must be coupled to some other non-symbolic, material system. In order to increase its effective dimensionality, it can couple to the world outside its internal symbol-states by three means: 
1) via human-machine interactions (facilitate novel insights in humans, use humans to create new primitives that expand systems, develop tools for creativity), 
2) via sensors and effectors on an external world (epistemically-autonomous evolutionary robots), and 
3) via internal analog dynamics (adaptive self-organization in mixed analog-digital devices or biological brains). 
When a computational system is augmented and opened up in these ways, it is transformed from a formal system that is informationally isolated from its surrounds to one that is self-organizing, self-complexifying, and in informational interaction with its surrounds. 
We discuss classes of adaptive and self-modifying cybernetic evolutionary robotic devices in terms of combinatoric and creative novelty and in terms of new functionalities that are created (new syntactic states, new semantic observables & actions, new pragmatic goals). If adaptive sensors and effectors are internalized in the form of signal generators and receivers, it is possible to think of neural networks in these terms. What this view of biological brains might look like is sketched out. Adaptively-tuned neuronal assemblies would conceivably function as internal sensors and signal generators, such that new signal types could be produced (i.e. new concepts). Emergence of new signal types would increase the effective dimensionality of internal signal spaces over time in an apparently open-ended manner.