Seminar: Augmented Reality as a Telemedicine Platform for Remote Procedural Training

Shiyao Wang
M.Sc. Candidate
Co-Supervisors: Drs. Minglun Gong, Oscar Meruvia-Pastor and Andrew Smith

Augmented Reality as a Telemedicine Platform for Remote Procedural Training

Wednesday, April 26, 2017, 11:00 a.m., Room EN 2022


Rural areas have been constrained by a lack of access to health care due to the inherent challenges associated with recruitment and retention of health professionals. Telemedicine, which uses communication technology to deliver medical services over distance, is an economical and effective way to address this problem. In this research, we develop a new telepresence application using an augmented reality (AR) technique. We explore the use of the Microsoft HoloLens to facilitate and enhance remote medical training. Intrinsic advantages of AR systems enable remote learners to perform complex medical procedures such as Point of Care Ultrasound (PoCUS) without visual interference. This research uses the HoloLens to capture the first-person view of a simulated rural emergency room (ER) through mixed reality capture (MRC) and serve as a novel telemedicine platform with remote pointing capabilities. The mentor's hand gestures are captured using a Leap Motion and virtually displayed in the AR space of the HoloLens. Twelve novice trainees are guided by a mentor through the performance of a trauma ultrasound. This user study explores the utility of the system from the trainees, mentor and objective observers perspective and compares the findings to that of a more traditional multi-camera telemedicine solution. The results provide valuable insight and guidance for the development an AR telemedicine platform.
Keywords: Telemedicine; Microsoft HoloLens; Augmented Reality; Ultrasound; Remote Mentoring; Leap Motion.


Department of Computer Science

230 Elizabeth Ave, St. John's, NL, CANADA, A1B 3X9

Postal Address: P.O. Box 4200, St. John's, NL, CANADA, A1C 5S7

Tel: (709) 864-8000