What if medical imaging and visualizing human anatomy were as easy, immersive, and intuitive as playing a video game? The latest project taken on by the Advanced Imaging and Modeling (AIM) program at Jump is working to achieve just that. AIM has nearly perfected the refined, manual process of converting traditional medical images (MRI and CT scans) into digital formats, allowing doctors and surgeons to interact with physical three-dimensional anatomic models for medical decision making, pre-surgical planning, and patient education.
In fact, hospitals from coast to coast are sending us medical images of congenital heart defects and other abnormal conditions for 3D printing. When medical decision making is involved, we provide this service for free in hopes of giving physicians and surgeons improved understanding of each patient’s clinical scenario. The ultimate goal is to utilize intuitive 3D tools to improve the clinical outcomes and standardize interpretations of anatomical images across all clinical specialties.
Now, AIM is taking digital formats of medical scans and plugging them into evolving visual technologies, eliminating the need for a physical model and shrinking the time it takes to view a complete 3D image. We believe virtual and augmented reality will revolutionize how radiologists and clinicians look at anatomy.
Physical 3D Models Vs. Digital Models
Just as there was not an existing process for converting medical images into a digital format, there was also not software to translate imaging data to be compatible with virtual environments like augmented and virtual realities. Our engineers at Jump have created a prototype program to view 3D images of hearts and other parts of the human anatomy for the HTC Vive, a virtual reality headset.
What we’ve found is that physical 3D representations of anatomy (3D printed content) can only be viewed in so many different ways. With augmented and virtual reality, there’s more flexibility to immerse oneself into the entire image, and expand viewing capabilities. It’s also scalable because I can send a video file much quicker than I can ship a physical 3D model.
The AIM team recently ran test case scenarios with several different surgeons to collect feedback on their experiences and whether they found viewing anatomy with the Vive beneficial.
“As one who does not have a background in video games and this type of technology, the HTC was fairly easy to grasp,” said Dr. Karl Welke, a pediatric congenital heart surgeon with Children’s Hospital of Illinois. “It puts you into a more intimate relationship with the heart and you can manipulate it in ways that you can’t do with a typical medical scan. As a surgeon, I can understand what I’m going to see in the operating room in much greater detail than I could before.”
Other surgeons found there to be educational value in using the Vive to view human anatomy. “I could have surgical residents view virtual models of cancer and ask them to practice removing tumors before going into surgery,” said Dr. Richard Anderson, thoracic and cardiac surgeon. “This would give residents more experience before operating on a real patient and prevent errors in the future.”
The Future of Medical Imaging
We see a future state where clinicians are truly immersed inside this environment, and interact with it in ways they were unable to before. We also see the potential for bringing people together inside virtual and augmented realities for group learning and comprehension.
I’ve been doing pediatric cardiac congenital MRI for nearly nine years, and I strongly believe that viewing medical images using immersive, visual technologies is not a fad. It’s the direction of the future. We are putting tools in clinicians’ hands that they didn’t realize they needed and helping them fulfill their potential in medical decision making with higher efficiency and quality so that we can improve health care.