Research project

One of the great challenges in modern information society is at the moment the creation of intelligent, multimodal user interfaces. These interfaces should be able to deal with the natural manner of communication used by `naive´ people; such an interaction will thus be intuitive and adapted to the needs of the prospective users. Computer applications are getting more and more important, and simultaneously, the systems are getting more and more complicated. Moreover, users will have even less time than now to get acquainted with new concepts, and at the same time, the costs for developing intelligent user interfaces have to be cut down to a large extent.

In the SmartKom project, concepts for novel types of human-machine-interaction are being developed. Such concepts make the acceptance of information technology much easier. It is a vision of the SmartKom project to combine the advantages of speech communication with those of graphic user interfaces and tactile interaction; by that, a so-called multimodal user paradigm will be created.

In SmartKom, the Chair for Pattern Recognition (Informatik 5) implements modules for a multimodal processing of speech, facial expression, and gesture. On the one hand, these modules should recognize automatically semantic information encoded in speech and gesture (in speech, accentuation and syntactic-prosodic boundaries, in gesture, deictic gestures); on the other hand, they are intended to detect emotional user states in all three modalities. In co-operation with other SmartKom partners, so-called use cases (representative constellations for applicatons) are being developed; by that, the usefulness of the processing of emotional user states in such systems can be demonstrated.