Department of Computer Science

---

The 12th ELSNET European Summer School

on Language and Speech Communication

Information Fusion
in Natural Language Systems

hosted by the University of Hamburg, Hamburg, Germany

3 - 14 July 2006

Second Announcement

Information Fusion is an everyday necessity in complex speech and language systems although it is rarely noticed as such. Systems in this area are always composed of individual components which need to co-operate towards a common goal. The reason for such modularisation is obvious: due to the many different layers and knowledge sources usually involved, system components need to be developed independently by different people with different areas and degrees of specialization. Moreover, there might be the option or even the necessity to train such components on vastly different kinds of data sets. Often it is the case that a range of solution alternatives exists for one and the same processing task, each of them providing a partial and unreliable but perhaps complementary contribution to the overall behaviour of the system. Here, the problem arises of how to achieve a synergy between such competitive approaches, even for tasks where the desired processing result is no longer a trivial one.

Since all available solutions for speech and language processing are approximations to a conveyed ideal, system design has to account for the inherent uncertainty of processing results on all levels. this makes system integration a problem of information fusion, which can be considered as solved in some cases but it is still an open research issue in others: for speech recognition the contributions of the acoustic models and the language model need to be adjusted properly, whereas in translation a target sentence has to be composed of partial structures produced by e.g. an example-based component and a deep-linguistic one. Other examples can be taken from more ambitious task like the integration of acoustic (speech and noise) and visual data (lip movements, hand gestures and facial expressions) in complex multimodal environments. It is always astonishing to notice, how little effort humans spend to integrate the available information from such a diversity of sources. Even more, multimodal information processing in many cases leads to a facilitatory effect in the sense that using evidence from a range of sensory channels is faster than relying on a single one. This, obviously, is contrary to the behaviour of all the techniques we currently have at our disposal when designing a complex speech and language system. Therefore, the summer school will depart from a survey of phenomena and mechanisms for information fusion. It continues with studying various approaches for sensor-data fusion in technical systems, like robots. Finally it will investigate the issue of information fusion from the perspective of a range of speech and language processing tasks, namely

• speech recognition and spoken language systems
• machine translation
• distributed and multilingual information systems
• parsing
• multimodal speech and language systems

How different these application areas might look, the underlying principles of and the approaches towards information fusion seem to be comparable if not even highly similar. It is the goal of the summer school to highlight such similarities and to inspire the cross-disciplinary transfer of ideas and solutions