Ontology driven voice-based interaction in mobile environment

Abstract

The paper deals with a new approach for spoken dialogue handling in mobile environment. The goal of our project is to allow the user to retrieve information from a knowledge base defined by ontology, using speech in a mobile environment. This environment has specific features that should be taken into account when the speech recognition and synthesis is performed. First of all, it limits the size of the language that can be understood by speech recognizers. On the other hand, it allows us to use information about user context. Our approach is to use the knowledge and user context to allow the user to speak freely to the system.

Our research has been performed in the framework of an EU funded project MUMMY. This project is targeted to the use of mobile devices on building sites. This fact determines the approach to the solution of the problem. The main issue is user context in which the interaction takes place. As the application (construction site) is rather specific it is possible to use the knowledge related to this particular application during the speech recognition process.

Up-to now the voice based user interfaces are based on various techniques that usually contain various constraints which limit the communication context to strictly predefined application domain. The main idea behind our solution is usage of ontology that represents the knowledge related to our particular application in specific user context. The knowledge acquired from ontology allows the user to communicate in mobile environment as the user input analysis is heavily simplified. The crucial step in our solution was the design of proper system architecture that allows the system to access the knowledge in ontology and use it to enhance the recognition process.

The model of environment in which the recognition process is performed has several parts:
- Domain ontology (construction sites in general) 
- instance of the domain ontology (specific construction site) 
- conversation history + specific user context (location, type of mobile device etc.). 
The key part of the model is the access mechanism that allows to extract particular knowledge in specific context. This access mechanism is controlled by means of dialogue automaton that controls the course of dialogue. The acquired knowledge is used in the speech recognizer for generation of a specific grammar that defines the possible speech inputs in a particular moment of the dialogue - in the next state another access into ontology in different context is done resulting in generation of a grammar that defines new possible inputs. The same access mechanism is also used to produce a response to user's input in natural language.

There exists a pilot implementation of the voice based user interface system, which has been tested in various situations and the results obtained are very encouraging.