Taking AR to surgery

Augmented reality could superimpose critical clinical data directly on the surgeon’s view of the patient. Source: The Augmentarium at the University of Maryland. Illustration credit: Brian G. Payne

Augmented reality has been applied in a variety of niches, including some areas in health care. As technology progresses, both in rendering data and making them visible on a variety of surfaces and in processing of the data, more effective uses of AR are proposed.

At the University of Maryland, as presented in an article on Harvard Business Review, researchers believe that AR can enhance the doctors effectiveness and point out  a number of reasons, including:

  • the possibility of displaying images from the point of view of the doctor, rather than from the point of view of the imaging device used in the exam,
  • the possibility of integrating several data resulting from different exams in a single image,
  • possibility of placing the information in a place that is easily seen by the doctor without requiring her to shift the gaze from the patient to the data.

They are also pointing out the potential to decrease cost by getting rid of several screens today used to display the different data (normally each monitoring device has its own screen and on that screen, managed directly by the device, there is no space to display other data).

At the University of Alberta researchers have shown how to use AR to let the doctor look “inside” the patient’s body, using data gathered from medical exams (see clip).

In this case a projector displays the data/images directly on the body of the patient, exactly where they are relevant. It might seem easy but it is extremely complex.

The display system needs to recognise the body of the patient in 3D, it needs to adapt the image to be displayed to the different shapes the body takes as it changes its position. This is achieved through a motion tracking system that analyses the body image captured by cameras and uses markers on the body to pinpoint the positions among the different parts of the body. A software developed by the team morphs the image to fit the body.

The system demonstrates the possibility offered by AR. For the time being it will be used for educational purposes but in the future the researchers hope to see its application on real patients in real medical procedures.

About Roberto Saracco

Roberto Saracco fell in love with technology and its implications long time ago. His background is in math and computer science. Until April 2017 he led the EIT Digital Italian Node and then was head of the Industrial Doctoral School of EIT Digital up to September 2018. Previously, up to December 2011 he was the Director of the Telecom Italia Future Centre in Venice, looking at the interplay of technology evolution, economics and society. At the turn of the century he led a World Bank-Infodev project to stimulate entrepreneurship in Latin America. He is a senior member of IEEE where he leads the New Initiative Committee and co-chairs the Digital Reality Initiative. He is a member of the IEEE in 2050 Ad Hoc Committee. He teaches a Master course on Technology Forecasting and Market impact at the University of Trento. He has published over 100 papers in journals and magazines and 14 books.