Multimodality in Pervasive Environment

Abstract

Future pervasive environments are expected to immerse users in a consistent
world of probes, sensors and actuators. Multimodal interfaces combined
with social computing interactions and high-performance networking can foster a
new generation of pervasive environments. However, much work is still needed to
harness the full potential of multimodal interaction. In this paper we discuss some
short-term research goals, including advanced techniques for joining and correlating
multiple data flows, each with its own approximations and uncertainty models.
Also, we discuss some longer term objectives, like providing users with a mental
model of their own multimodal "aura", enabling them to collaborate with the network
infrastructure toward inter-modal correlation of multimodal inputs, much in
the same way as the human brain extracts a single self-conscious experience from
multiple sensorial data flows.