Seminar: A Window to your Smartphone: Enhancing Social Interaction with Augmented Virtuality Using

Frieda Njien Musa
M.Sc. Candidate
Supervisors: Dr. Edward Brown and Dr. Oscar Meruvia-Pastor

A Window to your Smartphone: Enhancing Social Interaction with Augmented Virtuality Using
Depth Sensing Cameras

Department of Computer Science
Wednesday, November 28, 2018, 1:30 p.m., Room EN-2022


Experiencing Virtual Reality (VR) using Head-Mounted Displays (HMDs) isolates the user into a visual space resulting to full immersion where the user cannot make social interaction with their real-world environment while wearing a VR HMD. This research addresses the issue by using an Augmented Virtuality (AV) design. Augmented Virtuality (AV) incorporates an experience in which the virtual world of the user is merged into the real world in a way it allows both worlds to be visible at the same time.

In this project, the designed AV environment provides nearer-range visibility where the user’s smartphone or portable device (real world) merges into the virtual world while wearing HMD freely when the smartphone is within the field of view (FOV). The camera uses the depth image to calculate the distance of the smartphone. At a specified distance, the camera captures the view of the real-environment and merges it into the virtual world to enhance better readability when operating a smartphone.

In the end, a user study was carried out with 32 participants. Results of the user study indicate that the implementation using the Intel Real Sense camera with depth image proposed in this project allows participants to perform several tasks and is also preferred by a majority of the participants. The results also reveal that some participants still prefer to remove the HMD to accomplish some of the tasks.


Department of Computer Science

230 Elizabeth Ave, St. John's, NL, CANADA, A1B 3X9

Postal Address: P.O. Box 4200, St. John's, NL, CANADA, A1C 5S7

Tel: (709) 864-8000