Skip to content

Latest commit

 

History

History
87 lines (49 loc) · 3.92 KB

README.md

File metadata and controls

87 lines (49 loc) · 3.92 KB
Raw

Back to Projects List

AR in Slicer

Key Investigators

  • Alicia Pose Díez de la Lastra (Universidad Carlos III de Madrid, Madrid, Spain)
  • Javier Pascau (Universidad Carlos III de Madrid, Madrid, Spain)
  • Rafael Moreta Martínez (Universidad Carlos III de Madrid, Madrid, Spain)
  • Gabor Fichtinger (PerkLab, Queen's University , Kingston , Canada)
  • Andras Lasso (PerkLab, Queen's University , Kingston , Canada)
  • Adam Rankin (Robarts Research Institute / Western University, Canada)
  • Csaba Pinter (Ebatinca S.L., Las Palmas de Gran Canaria, Spain)
  • Lucas Gandel (Kitware, France)
  • Jean-Christophe Fillion-Robin (Kitware, USA)
  • Simon Drouin (École de Technologie Supérieure , Montreal , Canada)

Project Description

Augmented Reality has increased its adoption in many areas with exciting benefits. Universidad Carlos III de Madrid (Madrid, Spain) has already worked in several medical projects based on AR (see their progress in https://biig-igt.uc3m.es/augmented-reality/). On these studies, they usually export information from Slicer to an alternative software (Unity).

The ultimate goal of this project is to develop a new 3D Slicer module that will allow to use augmented reality directly in 3D Slicer. With it, it will be possible to centralize the working process, at the time of benefiting from all Slicer tools.

Ebatinca S.L. in Las Palmas de Gran Canaria (Spain), has already started this project in collaboration with Kitware (France) and Universidad Carlos III de Madrid (Spain) and they all are currently developing OpenXR.

Objective

  1. Create OpenXR, a code based on the currently existing OpenVR so that the new mixed reality version is compatible with VTK.

Some links of interest:

  1. Writing a Holographic Remoting remote app using the OpenXR API
  2. Slicer Documentation on Augmented Reality and Virtual Reality support

Approach and Plan

  1. Develop a new Slicer extension that streams AR directly to HoloLens 2.

Progress and Next Steps

  1. OpenXR in VTK:

The WIP branch supporting Holographic remoting to stream VTK rendering inside the Hololens has been submitted here.

  1. Our github latest upload
  2. We confirmed the prototype developed by Lucas Gandel by streaming VTK to the HoloLens 2 via Holographic Remoting.

Illustrations

Here below you can find some AR implementations in health by Universidad Carlos III de Madrid (Madrid, Spain) in the past years:

HoloLens 2 in Orthopedic Oncological Surgeries:

HoloLens 2 in Orthopedic Oncological Surgeries

Smartphone app to communicate with the patient and help him/her understand his/her condition:

Smartphone app to communicate with the patient and help him/her understand his/her condition

Real-time guidance during Open Cranial Vault Remodeling using smartphone:

Smartphone app to guide open cranial vault remodeling

Needle Insertion Guidance for Sacral Nerve Stimulation using smartphone:

Smartphone app to guide needle insertion for sacral nerve stimulation

Background and References