It’s well-known that smoking cigarettes is not good for you. A lot of work has been successfully done to discourage young people from picking up the unhealthy
vice. However, the Centers for Disease Control estimates that every day, thousands of people under the age of 18 smoke their first cigarette. I, along with a team of other Jump staff members, are hoping to decrease that trend with the development of an interactive, educational video game.
This project is part of a larger collaborative effort with OSF Saint Francis Medical Center to curb smoking among young people in central Illinois. Jump Simulation is hosting the annual event, Kick Butts on March 14th, 6:00-9:00 p.m. that teaches students in fourth grade through high school about the health risks of smoking in an innovative way.
Most of the time children are exposed to this information through lectures or handouts at school. While lecturing gets the point across, sometimes hands-on experience and visuals can leave a greater lasting impact. I recall going to the Robert Crown Center for Health Education as a kid and actively participating in various demonstrations. To this day, I remember educators inflating a healthy lung and one that’s been damaged by smoking. I was completely grossed out. That lasting image and experience are what Kick Butts wants to leave with its audience.
Smoking from the Inside Out
Kick Butts will feature a number of activities to effectively demonstrate the effect smoking has in and outside the body. Participants will watch a documentary, see different parts of anatomy impacted by smoking and step into Virtual Reality goggles to get up close and personal with the way lung cancer evolves over time.
My part in all of this was to work with designers and biomedical visualization specialists here at Jump to create an interactive and educational video game. The idea is for participants to pilot a ship (a la “Fantastic Voyage”), using an Xbox 360 controller, through a smoker’s airways to destroy a tumor. Along the way, students will learn about the surrounding anatomy and observe what “smoking from the inside” looks like in a real patient.
We quickly designed the video game using the Unity game engine tool. However, we have the ability to turn this game into an even more immersive experience by further developing it for the Virtual Reality space.
The Future of Video Gaming
We are excited about the game we developed for the upcoming Kick Butts event and we believe it will leave a lasting impression on teens who continually face peer pressure to smoke.
Beyond that, creating this game is a great opportunity for Jump to get involved in more education video game development. Video games and similar virtual learning tools have been present in medical simulation for quite some time. What we want to do is provide a fun gaming environment that is both entertaining and educational for users.
In addition, video games have such a wide range of audiences that we could tailor games to both children and adults, scaling up the lesson material and difficulty accordingly. With the help of our designers and biomedical visualization specialists, Jump has a unique opportunity to synergize and create some truly exceptional content.
The purpose of medical imaging from the very beginning was to figure out ways to look inside the body and learn what’s going on structurally and physiologically. To that end, physicians used x-rays or performed exploratory surgeries for decades to identify disease or injury. Then came the ultrasound in the 1960s that gave clinicians real-time images of internal body structures using sound waves. Imaging techniques progressed even further in the 1970s with the advent of CT scans and MRI, which are both commonly used today.
It’s my belief that 3D modeling will be the next critical tool used by physicians to not only diagnose, but improve surgical planning, patient outcomes and the education of future clinicians. It has the power to essentially produce exact replications of soft tissue structures, improving understanding among doctors and patients alike. But first, it will take collaboration across the U.S. to make this a reality.
I recently spoke at the American Heart Association-Midwest Affiliate’s Heart Innovation Forum to advocate for imaging techniques that lead to anatomic replication. The Advanced Imaging and Modeling (AIM) team at Jump Simulation has come up with a semi-automated process to convert CT and MRI scans into 3D digital images that can be printed or integrated into virtual environments like augmented and virtual realities (AR and VR). What we’ve learned is that these nearly perfect 3D surrogates of anatomy can’t happen without working to create quality images from the start.
Garbage In, Garbage Out
The old adage “garbage in, garbage out” applies directly to 3D modeling. The standard across the nation for the last ten years has been to quickly produce images that might not have the best quality but lead to diagnosis in an efficient and productive manner. The ability to print or view these images in three-dimensions, though, requires a little more time and effort but leads to discoveries we’ve never seen before.
There is a quality standard that must be met each step along the continuum of 3D modeling translation. If the image is poor – fail. If the segmentation is poor – fail. If the print is poor – fail. If the VR translation is poor – fail. The focus of our cardiovascular imaging efforts at OSF HealthCare is to generate the highest quality images we can attain.
Most recently, we sent a quality focused 3D heart digital file to the incredible engineers at Caterpillar’s additive manufacturing lab. They have a printer that allows us to produce a heart in a soft enough material that can be cut with a scalpel, allowing surgeons to effectively practice on a patient’s heart before surgery. The result was incredible. Not only were we able to practice the surgery before the operation, but we were able to see anatomic detail like never before seen, prompting an entirely new set of possibilities where 3D printing could potentially improve patient care.
Making a Case for High-Quality Imaging Standards
There are many physicians around the U.S who understand the impact 3D modeling can have on surgical planning, patient outcomes and the education of future clinicians. In fact, a group of us are working with the National Institutes of Health and the American Heart Association to create accuracy and quality standards for the Jump Simulation-curated 3D Heart Library, an open-source digital repository of hearts with congenital defects on the NIH 3D Print Exchange. However, I recognize there are still some skeptics out there who don’t understand the value of this technology.
My experience with these models has been that they give surgeons a point of reference they haven’t had before, giving them the ability to make informed decisions before operating on patients. They make viewing anatomical images intuitive across all medical specialties. 3D models give patients and their families a better understanding of procedures they may have to undergo. They also allow educators to easily explain different types of congenital heart disease and what they look like to physicians looking to master the skill of diagnosis or surgery.
Physicians are busy and it’s difficult to put the time and effort into higher quality imaging. However, doing so leads to exact anatomic replications and, in my opinion, is the next big jump in medical imaging surrogacy. It’s going to take clinicians making medical decisions or planning surgery to be impacted by this for the advocacy to come through the clinical community.
What if medical imaging and visualizing human anatomy were as easy, immersive, and intuitive as playing a video game? The latest project taken on by the Advanced Imaging and Modeling (AIM) program at Jump is working to achieve just that. AIM has nearly perfected the refined, manual process of converting traditional medical images (MRI and CT scans) into digital formats, allowing doctors and surgeons to interact with physical three-dimensional anatomic models for medical decision making, pre-surgical planning, and patient education.
In fact, hospitals from coast to coast are sending us medical images of congenital heart defects and other abnormal conditions for 3D printing. When medical decision making is involved, we provide this service for free in hopes of giving physicians and surgeons improved understanding of each patient’s clinical scenario. The ultimate goal is to utilize intuitive 3D tools to improve the clinical outcomes and standardize interpretations of anatomical images across all clinical specialties.
Now, AIM is taking digital formats of medical scans and plugging them into evolving visual technologies, eliminating the need for a physical model and shrinking the time it takes to view a complete 3D image. We believe virtual and augmented reality will revolutionize how radiologists and clinicians look at anatomy.
Physical 3D Models Vs. Digital Models
Just as there was not an existing process for converting medical images into a digital format, there was also not software to translate imaging data to be compatible with virtual environments like augmented and virtual realities. Our engineers at Jump have created a prototype program to view 3D images of hearts and other parts of the human anatomy for the HTC Vive, a virtual reality headset.
What we’ve found is that physical 3D representations of anatomy (3D printed content) can only be viewed in so many different ways. With augmented and virtual reality, there’s more flexibility to immerse oneself into the entire image, and expand viewing capabilities. It’s also scalable because I can send a video file much quicker than I can ship a physical 3D model.
The AIM team recently ran test case scenarios with several different surgeons to collect feedback on their experiences and whether they found viewing anatomy with the Vive beneficial.
“As one who does not have a background in video games and this type of technology, the HTC was fairly easy to grasp,” said Dr. Karl Welke, a pediatric congenital heart surgeon with Children’s Hospital of Illinois. “It puts you into a more intimate relationship with the heart and you can manipulate it in ways that you can’t do with a typical medical scan. As a surgeon, I can understand what I’m going to see in the operating room in much greater detail than I could before.”
Other surgeons found there to be educational value in using the Vive to view human anatomy. “I could have surgical residents view virtual models of cancer and ask them to practice removing tumors before going into surgery,” said Dr. Richard Anderson, thoracic and cardiac surgeon. “This would give residents more experience before operating on a real patient and prevent errors in the future.”
The Future of Medical Imaging
We see a future state where clinicians are truly immersed inside this environment, and interact with it in ways they were unable to before. We also see the potential for bringing people together inside virtual and augmented realities for group learning and comprehension.
I’ve been doing pediatric cardiac congenital MRI for nearly nine years, and I strongly believe that viewing medical images using immersive, visual technologies is not a fad. It’s the direction of the future. We are putting tools in clinicians’ hands that they didn’t realize they needed and helping them fulfill their potential in medical decision making with higher efficiency and quality so that we can improve health care.