Dreaming Machines: On multimodal fusion and information retrieval using neural-symbolic cognitive agents

Authors Leo de Penning, Artur D'Avila Garcez, John-Jules C. Meyer



PDF
Thumbnail PDF

File

OASIcs.ICCSW.2013.89.pdf
  • Filesize: 0.67 MB
  • 6 pages

Document Identifiers

Author Details

Leo de Penning
Artur D'Avila Garcez
John-Jules C. Meyer

Cite AsGet BibTex

Leo de Penning, Artur D'Avila Garcez, and John-Jules C. Meyer. Dreaming Machines: On multimodal fusion and information retrieval using neural-symbolic cognitive agents. In 2013 Imperial College Computing Student Workshop. Open Access Series in Informatics (OASIcs), Volume 35, pp. 89-94, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2013)
https://doi.org/10.4230/OASIcs.ICCSW.2013.89

Abstract

Deep Boltzmann Machines (DBM) have been used as a computational cognitive model in various AI-related research and applications, notably in computational vision and multimodal fusion. Being regarded as a biological plausible model of the human brain, the DBM is also becoming a popular instrument to investigate various cortical processes in neuroscience. In this paper, we describe how a multimodal DBM is implemented as part of a Neural-Symbolic Cognitive Agent (NSCA) for real-time multimodal fusion and inference of streaming audio and video data. We describe how this agent can be used to simulate certain neurological mechanisms related to hallucinations and dreaming and how these mechanisms are beneficial to the integrity of the DBM. Finally, we will explain how the NSCA is used to extract multimodal information from the DBM and provide a compact and practical iconographic temporal logic formula for complex relations between visual and auditory patterns.
Keywords
  • Multimodal fusion
  • Deep Boltzmann Machine
  • Neural-Symbolic Cognitive Agent
  • Dreaming
  • Hallucinations

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail