NephrectomyAR

Using Augmented Reality to Enhance Biomedical Pathology Dissection Guides

In recent years, researchers have begun to explore extended reality’s use in education. Several universities have implemented visualization tablets and virtual reality applications to learn about human anatomy.  Researchers have found that extended reality has a greater impact on spatial learning than traditional 2D textbooks and videos. Although there is plenty of research for medical students, there is a distinct lack of high-quality research regarding Augmented Reality for teaching biomedical and pathology technician students.  Several developments have been made to augment books. Taking this approach, the aim of this thesis was to develop an application to augment the Royal College of Pathologist dissection guide on nephrectomies.

 

A series of 3D models which were segmented from a computerized tomography (CT) scan which were then imported into 3DS Max and ZBrush for retopology, texturing, and animation. The application utilized augmented reality with image targets and a series of model controls which allowed the user to animate the models and provide information in a clear, intuitive format for amateurs following along the dissection guide. Following the application development, user testing was done utilizing a standardized survey called the System Usability Scale.This provided valuable user insight and feedback into the application and demonstrated the application was fully functional and usable.

 

Storyboard & Moodboard

The final storyboard and color scheme used to develop the final look of the application

Application Structure

Structural overview of the final application. Blue boxes indicate the scenes, yellow boxes indicate a button with functions, and grey boxes are content informational boxes with no functions.

color palate

Final color scheme utilized throughout the application (made on Coolors.co)

Simple Nephrectomy

A whole kidney model representing a simple nephrectomy which was created using CT scan data.

Simple Nephrectomy Controls

Screenshot of the application to show the controls available to the user

Simple Nephrectomy Macroscopic Evalutaiton

This panel demonstrated the anatomy to be visualized on each model. This button shows the user where the renal vein is by highlighting it

Initial Incision Controls

The controls the user can access for the model representing the initial incision

Initial Incision

A screenshot from the initial incision animation in the application.

Initial Incision Macroscopic Evaluation

This panel allows the user to revise the internal anatomy of the kidney. The button clicked was for renal pyramids which are highlighted

NephrectomyAR: Application Walkthrough

This is a short video which walk you through the application to show you how it is intended to work.