Priyansh Jalan

Title: Appearance Transfer for Real Objects in Mixed Reality

Supervision Team: John Dingliana, TCD / Cathy Ennis, TU Dublin

Description: Research in mixed reality is largely concerned with rendering virtual objects so that they appear plausibly integrated within a real environment. This project investigates the complementary problem of modifying the appearance of the real environment as viewed through a mixed reality display. For instance, a physical wall might be virtually removed so that objects can be seen through or embedded within it. However, merely removing existing surfaces may appear implausible, instead simulated geometry of a hole could be inserted to create the appearance of sections being cut away, or the object re-rendered as refractive glass so that we see through the surface but retain an understanding of the original geometry. The problem is particularly challenging in the context of modern Optical See-Through (OST) MR displays, such as Microsoft’s Hololens, where the real environment is seen directly through a transmissive screen, limiting the degree to which we can change its appearance.