The first head-mounted display offering 3-dimensional (3D) stereoscopic viewing of noninteractive film and stereo sound was invented in 1960 by Morton Heilig. However, the term “virtual reality” (abbreviated VR) wasn’t coined or popularized until 1987 by Jaron Lanier, founder of VPL Research, the first company to sell VR goggles. The next 2 decades saw VR devices developed primarily for gaming, but most failed and lacked consumer adoption. In 2012, Palmer Lucky launched a Kickstarter to fund the development of the Oculus Rift. Many see this as an inflection point that started the modern VR revolution. The following decade has seen incredible advancements in hardware and software, firmly establishing virtual reality as a technology that will revolutionize the way we connect, learn, and socialize.

Virtual reality in medicine has followed a similar timeline, dating back to 1965 for VR training and simulations. However, the current tech stack has accelerated the use and utility of VR technology in medicine, including surgical simulation and training, visual function assessment, and the metaverse.

SURGICAL TRAINING AND SIMULATION

“See one, do one, teach one.” — this has been the old adage of how surgeons learn. Apprenticeships, preceptorships, residency, fellowship: all of these involve trainees learning under a mentor and acquiring the skills necessary to become a proficient surgeon. Observing, assisting, and serving as primary surgeon on a variety of diverse cases allows a novice surgeon to learn the skills and techniques to thrive in their career. However, even with modern residency and fellowship programs, mastering surgical maneuvers and techniques could benefit from additional practice and experience for surgeons. Additionally, once in practice, staying up to date on new procedures and practicing these techniques can be challenging. Virtual reality simulations and haptic feedback simulators hold promise to train novice and expert surgeons alike. There are many companies exploring and developing in this space.

The Eyesi surgical simulator (Haag-Streit) has been available for several years and offers a high-end vitreoretinal surgery simulator. This is not a VR headset, but a microscope, instruments, and foot pedals that the surgeon uses to practice surgical steps. Surgeons can practice basic skills, steps of a procedure, or an entire procedure. It has a large footprint, and isn’t conducive to home training. It is appropriate at an academic institution in a skills lab. The Eyesi platform has an integrated dashboard to assess learning objectives and course status, performance analysis, and much more.

FundamentalVR develops VR-based surgical simulations paired to a patented haptic feedback instrument hardware (HapticVR). Surgeons view 3D simulations in a VR headset and use HapticVR to more accurately experience physical sensations as the virtual instruments interact with tissue. The haptic devices are similar to holding instruments in both hands, and foot pedals control typical parameters. Data insights assess surgeon proficiency and provide feedback on ways to improve. The company has a vitreoretinal simulator to train surgeons in subretinal gene therapy delivery. The FundamentalVR platform requires a VR headset and HapticVR hardware, but it has a smaller physical footprint than the Eyesi platform.

Several other companies are developing state-of-the-art VR surgical simulators and they are only scratching the surface on this technology’s potential to revolutionize surgical training and assessment. Current systems do a good job of simulating common procedures, but the costs and hardware requirements limit their use at home. Smaller and more compact hardware haptic add-ons with improved VR simulations could further facilitate access to people throughout the world.

VISUAL FUNCTION ASSESSMENT

In addition to surgical simulation, VR has made its way into assessment of visual function. A few companies are developing novel products that allow for advanced testing of visual function in patients. These devices could be used as remote home monitoring technologies or in clinic to improve current workflows.

Olleyes is a company that has developed the VisuALL, a VR-based headset that can perform a variety of tests, including visual fields (24-2, 10-2, 30-2), ptosis assessment, and Esterman visual fields. These tests take a fraction of the time of standard visual field tests. In addition, the Olleyes product can test near and far visual acuity, color vision, and contrast sensitivity. The product is mainly for office use, but home monitoring solutions could follow with additional testing functions.

Retina Technologies uses virtual reality headsets with the goal of “changing the future of vision care.” Their technology is aimed at vision screening tests that can be used anywhere and by any health care provider. OcuVue uses VR-based interactive modules to do examinations as well as retinal imaging from a single headset. Artificial intelligence and machine learning provides for assistance to health care providers for referrals to necessary specialists.

Whether for remote patient monitoring or screening tests and referrals from anywhere in the world, VR vision testing will likely serve a critical function in the years to come. This will save time and money, and could help to diagnose disease earlier and with timely referrals.

THE METAVERSE

In-real-life (IRL) education has been the main way to transfer knowledge across decades and centuries, via preceptorships, medical school, residency, fellowship, and attendance at conferences. Meeting people and learning from mentors IRL allows for connection and immersion unlike anything else. The COVID pandemic flipped this on its head almost immediately as the world locked down. In-person conferences stopped, and video conferencing exploded. However, video conferencing fails to capture what in-person meetings bring. Staring at boxes on a screen, inability to make eye contact, and difficulty with communication when more than 2 to 3 people are present, all create a scenario that is far inferior to an in-person experience. Virtual experiences should be immersive, engaging, and simulate how we interact IRL.We aimed to do that with Metamed, which has key features that include 3D spatial audio, 3D spatial environments, and a feeling of presence.

Three-dimensional spatial audio can be experienced with a VR headset, but a user can also have the same experience with a laptop, desktop, tablet, or smartphone when combined with headphones. Three-dimensional spatial audio creates an audio awareness of a 3D space. As a user moves around a 3D digital environment, their interaction and communication with other people simulates an IRL 3D environment. Users talking to a group of 2 other people might see one person on their left and one on their right. When the colleague to the left talks, the user hears them in their left ear ,and vice versa for the one to the right. Unlike other virtual platforms, both of those people can be talking at the same time and the user can hear both voices. If a group of colleagues is gathered and talking farther away, the user can hear them, but their voice volume is lower. A user can walk around a room as they would in real life, with volumn increasing as they approach a group of people talking or becoming quieter as they move away.

Three-dimensional spatial environments aren’t new; they’ve existed for years with video games like Fortnite, Roblox, Minecraft, and many others. Instead of a webcam or video of you, you are represented as a 3D avatar that can be photorealistic or not. You can dress and customize to your liking, but it will be your representation in the metaverse. In 3D virtual environments, you can walk around the space freely, interact with objects, sit and stand, and move your arms with VR controllers to gesture and actually pick up things. Combined with 3D spatial audio, you have an immersive experience that simulates real life and captures the presence that is lost with other virtual platforms. Metamed has been creating metaverse environments that bring together academic centers, societies, industry partners, and doctors/surgeons. The company aims to create opportunities to share and transfer knowledge, showcase new products in immersive ways, enhance IRL conferences to reach more people around the world, and provide more engaging experiences to bring people together.

CONCLUSION

This is just the start of a paradigm shift in how physicians interact virtually. New VR technology will pave the way for a future that is more immersive and engaging and presents new opportunities for retina specialists to learn and collaborate in new and exciting ways. RP