DROPS

Document

DOI: 10.4230/OASIcs.ICCSW.2013.89

Dreaming Machines: On multimodal fusion and information retrieval using neural-symbolic cognitive agents

Authors: Leo de Penning, Artur D'Avila Garcez, and John-Jules C. Meyer

Published in: OASIcs, Volume 35, 2013 Imperial College Computing Student Workshop

Abstract

Deep Boltzmann Machines (DBM) have been used as a computational cognitive model in various AI-related research and applications, notably in computational vision and multimodal fusion. Being regarded as a biological plausible model of the human brain, the DBM is also becoming a popular instrument to investigate various cortical processes in neuroscience. In this paper, we describe how a multimodal DBM is implemented as part of a Neural-Symbolic Cognitive Agent (NSCA) for real-time multimodal fusion and inference of streaming audio and video data. We describe how this agent can be used to simulate certain neurological mechanisms related to hallucinations and dreaming and how these mechanisms are beneficial to the integrity of the DBM. Finally, we will explain how the NSCA is used to extract multimodal information from the DBM and provide a compact and practical iconographic temporal logic formula for complex relations between visual and auditory patterns.

Cite as

Leo de Penning, Artur D'Avila Garcez, and John-Jules C. Meyer. Dreaming Machines: On multimodal fusion and information retrieval using neural-symbolic cognitive agents. In 2013 Imperial College Computing Student Workshop. Open Access Series in Informatics (OASIcs), Volume 35, pp. 89-94, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2013)

Copy BibTex To Clipboard

@InProceedings{depenning_et_al:OASIcs.ICCSW.2013.89,
  author =	{de Penning, Leo and D'Avila Garcez, Artur and Meyer, John-Jules C.},
  title =	{{Dreaming Machines: On multimodal fusion and information retrieval using neural-symbolic cognitive agents}},
  booktitle =	{2013 Imperial College Computing Student Workshop},
  pages =	{89--94},
  series =	{Open Access Series in Informatics (OASIcs)},
  ISBN =	{978-3-939897-63-7},
  ISSN =	{2190-6807},
  year =	{2013},
  volume =	{35},
  editor =	{Jones, Andrew V. and Ng, Nicholas},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/OASIcs.ICCSW.2013.89},
  URN =		{urn:nbn:de:0030-drops-42765},
  doi =		{10.4230/OASIcs.ICCSW.2013.89},
  annote =	{Keywords: Multimodal fusion, Deep Boltzmann Machine, Neural-Symbolic Cognitive Agent, Dreaming, Hallucinations}
}

Search Results

Documents authored by de Penning, Leo

Dreaming Machines: On multimodal fusion and information retrieval using neural-symbolic cognitive agents

Abstract

Cite as

Thanks for your feedback!

Could not send message