|
CROSSMOD - Cross-Modal Perceptual Interaction and Rendering: a New Virtual environments (VEs) play an increasingly important role in our society. Currently two main
sensorial channels are exploited in VEs: visual and auditory, although sound is greatly underused.
The ever-increasing scene complexity of VEs means that it is currently not possible to display
highly realistic scenes in real time despite the availability of modern high-performance graphics and
audio processors. However, the realism and quality of a virtual image/sound needs to be as good as
what the user can perceive: we only need to display what is necessary. Despite recent research in
psychology, little work exists on cross-modal effects, i.e., the effects that each channel (visual and
auditory) has on the other, to improve the efficiency and quality of VEs. CROSSMOD will study
these effects and develop a better understanding of how perceptual issues affect auditory/visual
display; this understanding will lead to the development of novel algorithms for selectively
rendering VEs. The cross-modal effects studied will include the effect of spatial/latency congruence
on quality perception, attention-control, sound-induced changes in visual perception and
foveal/peripheral audiovisual effects. The research will be guided by its applicability to the
improvement of VE display and VE authoring. The solutions developed by CROSSMOD will
enable the display of perceptually highly realistic environments in real time even for very complex
scenes, as well as the use of active cross-modal effects such as attention control in the display and
authoring of VEs. An integrated software cross-modal manager will be developed, using results of
experiments identifying which cross-modal effects are useful for VE display. We will evaluate our
approach on three target applications: computer games, design/architecture and clinical psychiatry,
using platforms adapted to each application: PCs with audio/graphics cards, a projector-enhanced
“personal VE” setup and large-screen immersive VE systems. | Project manager: Prof. Dr.-Ing. Marc Stamminger
Project participants: Dr.-Ing. Michael Schwarz
Keywords: Virtual Reality; Perception
Duration: 1.12.2005 - 31.3.2009
Sponsored by: Europäische Kommission
Mitwirkende Institutionen: INRIA – Institut National de Recherche en Informatique et Automatique, University of Bristol French National Center for Scientific Research, Paris Vienna University of Technology, Wien Institut de Recherche et de Coordination Acoustique/Musique, Paris Consiglio Nazionale delle Ricerche, Pisa
| Publications |
---|
Schwarz, Michael ; Stamminger, Marc: Microquad soft shadow mapping revisited. In: Eurographics Association (Org.) : Eurographics 2008 Annex to the Conference Proceedings: Short Papers (Eurographics 2008 Hersonissos 14.04.-18.04.2008). 2008, pp 295-298. | Schwarz, Michael ; Stamminger, Marc: Quality scalability of soft shadow mapping. In: Bartram, Lyn ; Shaw, Chris (Ed.) : Proceedings of Graphics Interface 2008 (Graphics Interface 2008 Windsor 28.5.-30.5.2008). 2008, pp 147-154. | Drettakis, George ; Bonneel, Nicolas ; Dachsbacher, Carsten ; Lefebvre, Sylvain ; Schwarz, Michael ; Viaud-Delmon, Isabelle: An interactive perceptual rendering pipeline using contrast and spatial masking. In: Kautz, Jan ; Pattanaik, Sumanta (Ed.) : Rendering Techniques 2007 (Eurographics Symposium on Rendering 2007 Grenoble 25.-27.06.2007). 2007, pp 297-308. - ISBN 978-3-905673-52-4 | Schwarz, Michael ; Stamminger, Marc: Bitmask soft shadows. In: Computer Graphics Forum 26 (2007), No. 3, pp 515-524 | Schwarz, Michael ; Staginski, Marco ; Stamminger, Marc: GPU-based rendering of PN triangle meshes with adaptive tessellation. In: Kobbelt, Leif ; Kuhlen, Torsten ; Aach, Til ; Westermann, Rüdiger (Ed.) : Proceedings of Vision, Modeling, and Visualization 2006 (Vision, Modeling, and Visualization 2006 Aachen 22.-24.11.2006). Berlin : Akademische Verlagsgesellschaft Aka GmbH, 2006, pp 161-168. - ISBN 3-89838-081-5 | Schwarz, Michael ; Stamminger, Marc: Fast GPU-based adaptive tessellation with CUDA. In: Computer Graphics Forum 28 (2009), No. 2, pp 365-374 | Schwarz, Michael ; Stamminger, Marc: Multisampled antialiasing of per-pixel geometry. In: Alliez, Pierre ; Magnor, Marcus (Ed.) : Eurographics 2009 Annex: Short Papers (Eurographics 2009 München 30.03.-03.04.2009). 2009, pp 21-24. | Schwarz, Michael ; Stamminger, Marc: On predicting visual popping in dynamic scenes. In: Bodenheimer, Bobby ; O'Sullivan, Carol (Ed.) : Proceedings of Symposium on Applied Perception in Graphics and Visualization 2009 (Symposium on Applied Perception in Graphics and Visualization 2009 Chania 30.09.-02.10.2009). 2009, pp 93-100. |
Institution: Chair of Computer Science 9 (Computer Graphics)
|
|
|