Audio-visual Analyis of Human Interaction

Rainer Stiefelhagen, University Karlsruhe, Germany

Current Human-Machine Interfaces mostly do not have a perception about their users and cannot perceive their natural communication modalities, which in addition to speech, include facial expression, gestures, body language, eye gaze etc. Such interfaces therefore cannot fully understand their users, their identities, intention and their current context. A number of ongoing research activities focus on developing technologies for perceiving and understanding people, their activities and interactions, in order to build better human-machine interfaces.

In this course we will discuss several technologies for the audio-visual perception of people and their interactions. The course will include the following topics

- Audio-visual detection and tracking of faces and people - Identification of people - Detection and recognition of gestures - Tracking of head pose and focus of attention - Approaches for multimodal information fusion - Audio-visual speech recognition