Document Open Access Logo

Towards Empathic Neurofeedback for Interactive Storytelling

Authors Marc Cavazza, Gabor Aranyi, Fred Charles, Julie Porteous, Stephen Gilroy, Ilana Klovatch, Gilan Jackont, Eyal Soreq, Nimrod Jakob Keynan, Avihay Cohen, Gal Raz, Talma Hendler

PDF

File

Thumbnail PDF
PDF
OASIcs.CMN.2014.42.pdf
  • Filesize: 4.36 MB
  • 19 pages

Document Identifiers

Subject Classification

Keywords
  • brain-computer interfaces
  • neurofeedback
  • interactive narrative
  • affective computing

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

Interactive Narrative is a form of digital entertainment based on AI techniques which support narrative generation and user interaction. Despite recent progress in the field, there is still a lack of unified models integrating narrative generation, user response and interaction. This paper addresses this issue by revisiting existing Interactive Narrative paradigms, granting explicit status to users' disposition towards story characters. We introduce a novel Brain-Computer Interface (BCI) design, which attempts to capture empathy for the main character in a way that is compatible with filmic theories of emotion. Results from two experimental studies with a fully-implemented system demonstrate the effectiveness of a neurofeedback-based approach, showing that subjects can successfully modulate their emotional support for a character who is confronted with challenging situations. A preliminary fMRI analysis also shows activation during user interaction, in regions of the brain associated with emotional control.

Cite As Get BibTex

Marc Cavazza, Gabor Aranyi, Fred Charles, Julie Porteous, Stephen Gilroy, Ilana Klovatch, Gilan Jackont, Eyal Soreq, Nimrod Jakob Keynan, Avihay Cohen, Gal Raz, and Talma Hendler. Towards Empathic Neurofeedback for Interactive Storytelling. In 2014 Workshop on Computational Models of Narrative. Open Access Series in Informatics (OASIcs), Volume 41, pp. 42-60, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2014) https://doi.org/10.4230/OASIcs.CMN.2014.42

Author Details

Marc Cavazza
Gabor Aranyi
Fred Charles
Julie Porteous
Stephen Gilroy
Ilana Klovatch
Gilan Jackont
Eyal Soreq
Nimrod Jakob Keynan
Avihay Cohen
Gal Raz
Talma Hendler

References

  1. John J. B. Allen, James A. Coan, and Maria Nazarian. Issues and assumptions on the road from raw signals to metrics of frontal EEG asymmetry in emotion. Biological Psychology, 67(1):183-218, 2004. Google Scholar
  2. John J. B. Allen, Eddie Harmon-Jones, and James H. Cavender. Manipulation of frontal EEG asymmetry through biofeedback alters self-reported emotional responses and facial EMG. Psychophysiology, 38(4):685-693, June 2001. Google Scholar
  3. Elsa Baehr, J. Peter Rosenfeld, and Rufus Baehr. Clinical use of an alpha asymmetry neurofeedback protocol in the treatment of mood disorders: Follow-up study one to five years post therapy. Journal of Neurotherapy, 4(4):11-18, 2001. Google Scholar
  4. Benny B. Briesemeister, Sascha Tamm, Angela Heine, and Arthur M Jacobs. Approach the good, withdraw from the bad - a review on frontal alpha asymmetry measures in applied psychological research. Psychology, 4:261, 2013. Google Scholar
  5. James A. Coan and John J. B. Allen. Frontal EEG asymmetry as a moderator and mediator of emotion. Biological psychology, 67(1):7-50, 2004. Google Scholar
  6. Richard J. Davidson. Affective style and affective disorders: Perspectives from affective neuroscience. Cognition &Emotion, 12(3):307-330, 1998. Google Scholar
  7. Mark H. Davis. Measuring individual differences in empathy: Evidence for a multidimensional approach. Journal of personality and social psychology, 44(1):113, 1983. Google Scholar
  8. Nico H. Frijda. The laws of emotion. American Psychologist, 43:349-358, 1988. Google Scholar
  9. Stephen W. Gilroy, Julie Porteous, Fred Charles, and Marc Cavazza. Exploring passive user interaction for adaptive narratives. In Proceedings of the 17th International Conference on Intelligent User Interfaces (IUI `12), pages 119-128. ACM, 2012. Google Scholar
  10. Aysegul Gunduz, Peter Brunner, Amy Daitch, Eric C. Leuthardt, Anthony L. Ritaccio, Bijan Pesaran, and Gerwin Schalk. Neural correlates of visual-spatial attention in electrocorticographic signals in humans. Frontiers in human neuroscience, 5:89, 2011. Google Scholar
  11. D. Corydon Hammond and Elsa Baehr. Neurofeedback for the treatment of depression: Current status of theoretical issues and clinical research. In T. H. Budzynski, H. K. Budzynski, J. R. Evans, and A. Abarbanel, editors, Introduction to Quantitative EEG and Neurofeedback: Advanced Theory and Applications, pages 295-314. Academic Press, 2009. Google Scholar
  12. Eddie Harmon-Jones. Clarifying the emotive functions of asymmetrical frontal cortical activity. Psychophysiology, 40(6):838-848, 2003. Google Scholar
  13. Jeffrey B. Henriques and Richard J. Davidson. Left frontal hypoactivation in depression. Journal of Abnormal Psychology, 100(4):535-545, 1991. Google Scholar
  14. S.J. Johnston, S.G. Boehm, D. Healy, R. Goebel, and D. Linden. Neurofeedback: a promising tool for the self-regulation of emotion networks. Neuroimage, 49(1):1066-1072, 2010. Google Scholar
  15. Silvia E. Kober, Matthias Witte, Manuel Ninaus, Christa Neuper, and Guilherme Wood. Learning to modulate one’s own brain activity: the effect of spontaneous mental strategies. Frontiers in human neuroscience, 7, 2013. Google Scholar
  16. Sharee N. Light, James A. Coan, C. Zahn-Waxler, C. Frye, H. Hill Goldsmith, and R. J. Davidson. Empathy is associated with dynamic change in prefrontal brain electrical activity during positive emotion in children. Child development, 80(4):1210-1231, 2009. Google Scholar
  17. Fabien Lotte. On the need for alternative feedback training approaches for BCI. In Berlin BCI Workshop on Advances in Neurotechnology, 2012. Google Scholar
  18. David Marshall, Damien Coyle, Shane Wilson, and Michael Callaghan. Games, gameplay, and BCI: The state of the art. Computational Intelligence and AI in Games, IEEE Transactions on, 5(2):82-99, 2013. Google Scholar
  19. A. Nijholt, D. Plass-Oude Bos, and B. Reuderink. Turning shortcomings into challenges: Brain-computer interfaces for games. Entertainment Computing, 1(2):85-94, April 2009. Google Scholar
  20. Kevin N. Ochsner, Kyle Knierim, David H. Ludlow, Josh Hanelin, Tara Ramachandran, Gary Glover, and Sean C. Mackey. Reflecting upon feelings: an fMRI study of neural systems supporting the attribution of emotion to self and other. Journal of Cognitive Neuroscience, 16(10):1746-1772, 2004. Google Scholar
  21. D. Plass-Oude Bos, B. Reuderink, B. van de Laar, Hayrettin Gürkök, Christiam Mühl, Mannes Poel, Anton Nijholt, and Dirk Heylen. Brain-computer interfacing and games. In D. Tan and A. Nijholt, editors, Brain-Computer Interfaces, pages 149-178. Springer, 2010. Google Scholar
  22. Julie Porteous, Marc Cavazza, and Fred Charles. Applying Planning to Interactive Storytelling: Narrative Control using State Constraints. ACM Transactions on Intelligent Systems and Technology (ACM TIST), 1(2):1-21, 2010. Google Scholar
  23. Ruth E. Propper, Jenna Pierce, Mark W. Geisler, Stephen D. Christman, and Nathan Bellorado. Asymmetry in resting alpha activity: Effects of handedness. Open Journal of Medical Psychology, 1:86, 2012. Google Scholar
  24. Ander Ramos-Murguialday, Markus Schürholz, Vittorio Caggiano, Moritz Wildgruber, Andrea Caria, Eva M. Hammer, S. Halder, and N. Birbaumer. Proprioceptive feedback and brain computer interface (BCI) based neuroprostheses. PloS One, 7(10):e47048, 2012. Google Scholar
  25. G. Raz, Y. Winetraub, Y. Jacob, S. Kinreich, A. Maron-Katz, G. Shaham, I. Podlipsky, G. Gilam, E. Soreq, and T. Hendler. Portraying emotions at their unfolding: A multilayered approach for probing dynamics of neural networks. NeuroImage, 60:1448-1461, 2012. Google Scholar
  26. Boris Reuderink, Christian Mühl, and Manne Poel. Valence, arousal and dominance in the EEG during game play. International Journal of Autonomous and Adaptive Communications Systems, 6(1/2013):45-62, 2013. Google Scholar
  27. J. Peter Rosenfeld. An EEG biofeedback protocol for affective disorders. Clinical EEG electroencephalography, 31(1):7-12, 2000. Google Scholar
  28. J. Peter Rosenfeld, Grace Cha, Tad Blair, and Ian H. Gotlib. Operant (biofeedback) control of left-right frontal alpha power differences: Potential neurotherapy for affective disorders. Biofeedback and Self-Regulation, 20(3):241-258, 1995. Google Scholar
  29. S. K. Sutton and R. J. Davidson. Prefrontal brain asymmetry: A biological substrate of the behavioral approach and inhibition systems. Psychological Science, 8(3):204-210, 1997. Google Scholar
  30. Ed S. Tan. Emotion and the Structure of Narrative Film: Film as an Emotion Machine. Mahwah, N.J: L. Erlbaum Associates, 1996. Google Scholar
  31. P. Tikka, A. Valjamae, Aline W. de Borst, Roberto Pugliese, Niklas Ravaja, M. Kaipainen, and T. Takala. Enactive cinema paves way for understanding complex real-time social interaction in neuroimaging experiment. Frontiers in Human Neuroscience, 6, 2012. Google Scholar
  32. Alexa M. Tullett, Eddie Harmon-Jones, and Michael Inzlicht. Right frontal cortical asymmetry predicts empathic reactions: Support for a link between withdrawal motivation and empathy. Psychophysiology, 49(8):1145-1153, 2012. Google Scholar
  33. Carmen Vidaurre and Benjamin Blankertz. Towards a cure for BCI illiteracy. Brain Topography, 23(2):194-198, 2010. Google Scholar
  34. A. Vuckovic. Motor imagery questionnaire as a method to detect BCI illiteracy. In Proceedings of the 3rd International Symposium on Applied Sciences in Biomedical and Communication Technologies (ISABEL `10), pages 1-5. IEEE, 2010. Google Scholar
  35. R.E. Wheeler, R.J. Davidson, and A.J. Tomarken. Frontal brain asymmetry and emotional reactivity: A biological substrate of affective style. Psychophysiology, 30(1):82-89, 1993. Google Scholar
  36. Tom Ziemke. Empathy with computer game characters: A cognitive neuroscience perspective. Virtual Social Agents, page 73, 2005. Google Scholar
  37. Vadim Zotev, Raquel Phillips, Han Yuan, Masaya Misaki, and Jerzy Bodurka. Self-regulation of human brain activity using simultaneous real-time fMRI and EEG neurofeedback. NeuroImage, 85:985-995, 2014. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact