Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik GmbH Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik GmbH scholarly article en Gerhard, Michael License
when quoting this document, please refer to the following
URN: urn:nbn:de:0030-drops-4609

Evaluating Embodied Conversational Agents in Collaborative Virtual Environments



There are currently no evaluation methods specific to ECAs in CVEs and traditional evaluation methods are limited in their applicability and consequently unlikely to address the full range of aspects now inherent in such systems. We argue that a combination of controlled experimentation, quasi-experiments, review-based evaluation and heuristic expert reviews is needed. To operationalise these traditional evaluation methods the concept of presence was deployed, and it was argued that presence as a cognitive variable can be measured and that such a measure can be a key indicator of the usability of ECAs in CVEs. Presence measures can be administered within controlled experiments and quasi-experiments to test certain aspects of the system. Such experiments might turn out particularly useful as a means of selecting between two or more design options and it is argued that issues concerning ECAs in CVEs can be meaningfully evaluated by comparing subjects’ experience of presence. Further, although implementation issues were not the primary concern of this study, the strength and shortcomings of the prototype agent were evaluated as secondary variables within that experiment. A set of criteria developed for the evaluation of the strengths and shortcomings of the current prototype agent are partly based on Nielsen’s general usability guidelines and partly on a set of heuristics proposed for non-embodied conversational systems.

BibTeX - Entry

  author =	{Michael Gerhard},
  title =	{Evaluating Embodied Conversational Agents in Collaborative Virtual Environments},
  booktitle =	{Evaluating Embodied Conversational Agents},
  year =	{2006},
  editor =	{Zsofia Ruttkay and Elisabeth Andr{\'e} and W. Lewis Johnson and Catherine Pelachaud},
  number =	{04121},
  series =	{Dagstuhl Seminar Proceedings},
  ISSN =	{1862-4405},
  publisher =	{Internationales Begegnungs- und Forschungszentrum f{\"u}r Informatik (IBFI), Schloss Dagstuhl, Germany},
  address =	{Dagstuhl, Germany},
  URL =		{},
  annote =	{Keywords: Embodied Conversational Agents,Collaborative Virtual Environments, Presence}

Keywords: Embodied Conversational Agents,Collaborative Virtual Environments, Presence
Seminar: 04121 - Evaluating Embodied Conversational Agents
Issue date: 2006
Date of publication: 2006

DROPS-Home | Fulltext Search | Imprint Published by LZI