A Multimodal Interface for Road Design

by Blessing, A., Sezgin, T. M., Arandjelovic, R. and Robinson, P.

Summary:

The authors begin the article by stating that designing with a computer can be made more interactive and natural with speech and sketch-based interfaces. They wanted to show this in a representative interface for road design that will be used either for actual road design or road design for simulators. The common parts for these two include designing the physical aspects of the road such as the road itself, the signs, the place of the signs, etc. It is proposed that these can be specified with the sketch-based system. While, on the other hand, the design of the traffic behavior within a simulation could be achieved through speech.

After conducting necessary interviews with experts on road design, the authors came up with Multimodal Intelligent Road Design Assistant (MIRA) for their intended speech and sketch-based system. The mapping of features of the system to modalities has been done in order to meet the desired affordances, which are put out by the experts. For example, it is suggested that behavior of an object or a group of objects can be defined with speech, together by a traditional WIMP modality. Or, designing the road can be done with a pen-based modality. Furthermore, it is stated that the modalities has been used in a complementary fashion. While some features could both be mapped to speech or sketch-based modalities, the authors have chosen one of the two in order not to overload a given modality.

After dealing with mapping features to modalities, the authors continue with giving information about speech and sketch-based recognition engines. Finally, they provide the pilot testing and evaluation and finish the article by putting out the related and future work.

Discussion:

Overall, the article shows that a system with natural modalities such as speech and sketching is capable of making the design process more interactive. One interesting point about the pilot test indicates that the capabilities of the speech interfaces so-far have a negative effect on the users' part while trying to communicate with the system. A video that shows the usage of speech system has been shown to overcome this negative effect and it succeeded.

The article lacks a proper method for the assignment of modalities to the different capabilities of the system and this proves itself in pilot study. The mapping has been done in an intuitive manner, which lead to some recognition errors during editing operations. Some short words for editing such as "undo", "redo", etc. had been misrecognized due to the speech engines' high sensitivity for noise. The authors needed to change the intended modality for these operations to pen-based in order to prevent some serious results that may frustrate the end-user.

Citation:

Blessing, A., Sezgin, T.M., Arandjelovic, R. and Robinson, P. A Multimodal Interface for Road Design. In IUI '09: Proceedings of the Intelligent User Interfaces Workshop on Sketch Recognition, Sanibel Island, Florida, USA, 2009.