Abstract: Since the times of the Lumière brothers, the way we watch movies hasn’t fundamentally changed: whether in movie theaters, on mobile devices, or on TV at home, we still experience movies as outside observers, watching the action through a “peephole” whose size is defined by the angular extent of the screen. As soon as we look away from the screen or turn around, we are immediately reminded that we are only “voyeurs”. With full field-of-view, head-mounted and tracked displays available now on the consumer market, this outside-observer paradigm of visual entertainment is giving way to a fully immersive experience that encompasses the viewer and is able to draw us in much more than was possible before.

Current endeavors towards immersive visual entertainment, however, are still almost entirely based on 3D graphics-generated content, limiting application scenarios to virtual worlds only. The reason is that in order to provide for stereo vision and ego-motion parallax, which are essential for genuine visual immersion perception, the scene must be rendered in real-time from arbitrary vantage points. While this can be easily accomplished for 3D graphics via standard GPU rendering, it is not at all straight-forward to do the same from conventional video footage acquired of real-world events.

In my talk I will outline avenues of research toward enabling the immersive experience of real-world recordings, enhancing the immersive viewing experience by taking perceptual issues into account, and extending visual immersion beyond a single viewer to create a collectively experienceable immersive real-world environment.