Yinghan Xu

Title: Interactive Volumetric Video for Extended Reality (XR) Applications

Supervision Team: John Dingliana, TCD / Steven Davy, TU Dublin / Gareth W. Young, TCD

Description: In this project we investigate 3D graphics, vision and AI techniques to improve the use of volumetric video for interactive Extended Reality (XR) technologies. An advantage of volumetric video is that it facilitates personalised and photorealistic animations of subjects without need for editing by experienced animators. However, most current applications merely treat volumetric video as a linear sequence of frames with limited possibility for interaction, apart from rudimentary operations such as playback or rigid transformations. We will investigate extensions to volumetric video including: (a) flexible reuse such as retargeting, time-warping or seamlessly transitioning between independently recorded clips, whilst preserving the personalised and realistic appearance of the subject; (b) improving seamless integration in XR, avoiding unrealistic intersections with the real environment, and matching physical events in the volumetric video with viable interaction points in the real-world environment; (c) adaptation of volumetric video in real-time to integrate and improve shared XR experiences.