Hrishikesh Mulay

Title: Annotated lip reading for Augmented Educational Systems

Supervision Team: Eleni Mangina, UCD / Sam Redfern, UoG

Description: This project begins with the hypothesis that emerging technologies (Augmented and Virtual Reality – AR/VR) will influence the pedagogical perspectives and practices for students with literacy problems. A review of the literature has shown that the lip-reading system remains a challenging topic due to the lack of an annotated dataset that is related to this domain. For example, with automatic speech recognition and speech analysis, there is a need a comprehensive dataset, which is expensive as well as time-consuming to gather. This project considers embedding a 3D avatar within an augmented educational system, with the capacity of semi-supervised learning, to generate and manipulate fabricated data from the real input data. The aim is to identify how AR or VR (as an alternative) can assist with the collection of data through an immersive environment.