Exploring Mixed Reality in Specialized Surgical Environments

Abstract

Recent technology advances in both Virtual Reality and Augmented Reality are creating an opportunity for a paradigm shift in the design of human-computer interaction systems. Delving into the Reality-Virtuality Continuum, we find Mixed Reality, systems designed to augment the physical world with virtual entities that embody characteristics of real world objects. In the medical field, Mixed Reality systems can overlay real-time and spatially accurate results onto a patient’s body without the need for external screens. The complexity of these systems previously required specialized prototypes, but newly available commercial products like the Microsoft HoloLens make the technology more available. Through a combination of literature review, expert analysis, and prototyping we explore the use of Mixed Reality in healthcare. From the experience of prototyping Patiently and HoloSim, two applications for augmenting medical training, we outline considerations for the future design and development of virtual interfaces grounded in reality.