Can Augmented Reality Help Surgeons in the OR?

A new study explores the pros and cons of using augmented-reality headsets in the operating room.

Is augmented reality ready to play a bigger role in surgery? A new study led by a physician from Keck Medicine of USC explores the pros and cons of one specific augmented-reality headset: Apple Vision Pro. 

The Apple Vision Pro wearer sees their real-world surroundings — but overlaid with virtual elements. During surgery, the device displays data and images within the field of vision, letting surgeons access useful information without leaving the operating table or looking away from the patient. They can also use hand gestures or voice commands to interact with the data hands-free. 

Augmenting reality in the operating room 

Apple Vision Pro can enhance the way surgeons visualize complex anatomy. It can, for instance, superimpose images from preoperative scans, or images of underlying anatomy, onto the patient. It can even superimpose measurement tools that help surgeons during procedures. 

The study’s lead author, David G. Armstrong, DPM, a podiatric surgeon at Keck Medicine, and his colleagues used Apple Vision Pro during limb-preservation surgery on a 62-year-old male patient. During limb-salvage operations, augmented-reality headsets enable physicians to better visualize and analyze, in real time, blood flow and tissue conditions, as well as identify potential complications. 

The patient required surgery to address diabetic hallux gangrene in his foot. The procedure involved a multidisciplinary team of podiatric, orthopaedic and vascular surgeons. First, the surgeons performed an angioplasty to open the narrowed blood vessels in the patient’s foot and ankle. Next, they applied antibiotic bone cement to eradicate osteomyelitis (swelling of bone tissue). 

“During this procedure, Apple Vision Pro helped guide our incision placement and debridement of the bone and soft tissue. It enabled us to determine that the patient had enough blood flow in the area to allow it to heal,” Armstrong explains. 

He adds that the device also allowed him to view the recipe for the antibiotic bone cement as he prepared it. 

Still some limitations 

Apple Vision Pro, as it stands today, still has some drawbacks for surgeons. 

Its visual display might not be high-resolution enough for some procedures. “I think it’s fine for certain types of surgeries, but not others,” Armstrong says. “For intricate procedures, the resolution through the screen is not there yet.” 

The study authors also note that the device requires virtual content to be “perfectly aligned with the natural view of the environment” — a degree of precision not always possible during surgery. “In addition,” they add, “the discrepancy between the 4D real world and the 2D virtual content on the display leads to perceptual issues such as difficulties with depth-of-field perception.” 

Other drawbacks include the inability of people wearing the headsets to share what they’re seeing with each other, and a limited battery life (approximately 2 hours). 

Another drawback? “You basically have ski goggles on your face the whole time,” Armstrong says. 

Is there promise? 

The study concludes that while Apple Vision Pro can improve some elements of surgical work, there are still limitations to adapting this consumer technology in a clinical setting. 

For now, its main benefits may be in communication and documentation. 

Apple Vision Pro enables surgeons in the OR to communicate with off-site physicians. It can record high-resolution spatial video and photographs to share with colleagues in real time. 

It can also be a powerful training tool. For instance, during a second limb-preservation operation described in their study, Armstrong and his team were able to use Apple Vision Pro’s 3D camera to let medical trainees in Italy, California and Texas view the operation from the surgeon’s point of view simultaneously. 

“The ability to educate with this will be extraordinary,” Armstrong says. 

Apple Vision Pro may even help physicians better leverage preoperative and postoperative scans when designing custom prosthetics and implants to match a patient’s anatomy, or when creating 3D-printed biodressings. 

Armstrong says it’s important that physicians continue to test the potential of augmented-reality headsets. This way, he says, “we can evaluate the best and worst aspects of them and try to push them forward.” 

He says this is the first published study documenting use of Apple Vision Pro in surgery — and that he and his team were also first to document the medical use of Google Glass, FaceTime and WhatsApp. 

Regarding Apple Vision Pro, Armstrong and his co-authors state in the study: “These early observations indicate that while Apple Vision Pro may soon improve aspects of surgical performance and education, further iteration, evaluation and experience are needed to fully understand its impact on patient outcomes and to refine its integration into clinical practice.” 

Keck Medicine of USC

David G. Armstrong, DPM, is a podiatric surgeon at Keck Medicine of USC and an internationally recognized leader in the field of diabetic foot, limb preservation, tissue repair and wound healing. 

Topics