Search Results

The increasing power and display resolution of mobile devices allow the user nowadays to work with 3D information in mobile environment. The use of this new technology brings some new problems that need an urgent solution. One of them has its roots in the fact that common users are not trained to work in 3D graphical environment in general. The main obstacle for a common user is the fact that 3D environment offers too much freedom for object manipulation in comparison with situation in 2D environment. There are various solutions to this problem but usually they do not handle efficiently the problem of navigation in 3D environment in context of handling appropriate information density on a small screen on a mobile device. Our approach is based on transformation of a 3D scene in 2D representation in order to decrease freedom of movement during navigation in the scene. The navigation (and other types of interaction) is performed in 2D environment - the information acquired during interaction is transformed back into 3D form (into 3D object space). In such a way the user can manipulate with single objects in various modes (e.g. the user can change focus on various objects what might e.g. result in their temporary elimination from the scene). The work in 2D in mobile environment requires some special features that are not considered in standard environment. Especially critical is zooming. Zooming in raster form could change the picture on a small screen into completely unusable representation (pixels will be transformed in tiles that will be difficult to perceive). Solution to this (and to other related problems) is the use of SVG format to handle 2D scene representation. The main advantage of SVG is that it is object oriented and supports zooming, interaction with objects, scene annotation and manipulations with objects. The process of scene transformation and manipulation with objects is distributed between the server and mobile device. The transformation is performed on the server side. The resulting 2D representation of the scene is sent to mobile device where the visualization is performed. The user can interact with the picture on the mobile device with interaction techniques available on mobile devices. As the SVG is XML based format, the Cascading Style Sheets can be used to style all graphical elements. Therefore each object can be visualized in different mode (solid, transparent, semitransparent). These modes could be changed interactively. This feature (interactive change of modes) substitute in certain sense some functions available for avatars during their walkthroughs when performed in standard 3D environment (e.g. "having look behind corner" etc.). The user interaction in 2D environment is rather limited in comparison with standard 3D interaction but it is less demanding for the user. On the other hand it brings better picture quality due to SVG features applied e.g. during the process of zooming.

The paper deals with a new approach for spoken dialogue handling in mobile environment. The goal of our project is to allow the user to retrieve information from a knowledge base defined by ontology, using speech in a mobile environment. This environment has specific features that should be taken into account when the speech recognition and synthesis is performed. First of all, it limits the size of the language that can be understood by speech recognizers. On the other hand, it allows us to use information about user context. Our approach is to use the knowledge and user context to allow the user to speak freely to the system. Our research has been performed in the framework of an EU funded project MUMMY. This project is targeted to the use of mobile devices on building sites. This fact determines the approach to the solution of the problem. The main issue is user context in which the interaction takes place. As the application (construction site) is rather specific it is possible to use the knowledge related to this particular application during the speech recognition process. Up-to now the voice based user interfaces are based on various techniques that usually contain various constraints which limit the communication context to strictly predefined application domain. The main idea behind our solution is usage of ontology that represents the knowledge related to our particular application in specific user context. The knowledge acquired from ontology allows the user to communicate in mobile environment as the user input analysis is heavily simplified. The crucial step in our solution was the design of proper system architecture that allows the system to access the knowledge in ontology and use it to enhance the recognition process. The model of environment in which the recognition process is performed has several parts: - Domain ontology (construction sites in general) - instance of the domain ontology (specific construction site) - conversation history + specific user context (location, type of mobile device etc.). The key part of the model is the access mechanism that allows to extract particular knowledge in specific context. This access mechanism is controlled by means of dialogue automaton that controls the course of dialogue. The acquired knowledge is used in the speech recognizer for generation of a specific grammar that defines the possible speech inputs in a particular moment of the dialogue - in the next state another access into ontology in different context is done resulting in generation of a grammar that defines new possible inputs. The same access mechanism is also used to produce a response to user's input in natural language. There exists a pilot implementation of the voice based user interface system, which has been tested in various situations and the results obtained are very encouraging.