When preparing and planning an operation, doctors need to study a patient’s available medical imaging data, such as CT scans or MRIs, very closely. In the operating room, the challenge is then to apply the plan created previously from studying the imaging data to the operating situation and, thus, to the patient. What makes this difficult is the three-dimensional imaging data is typically shown using individual section planes. Doctors must use their extensive knowledge of anatomy to apply this information to the situation in the OR.
Augmented reality (AR) glasses can superimpose three-dimensional imaging and planning data on human anatomy. This can be used to project three-dimensional content directly onto the patient. This helps doctors master the challenges in the OR better and faster, making their interventions targeted and safer.
In collaboration with its technical partners, Fraunhofer IGD offers AR technology for the OR, primarily developing software tools for operating different camera systems to extract spatial information and use it to locate objects. One example of this is tracking surgical instruments during an operation. In addition to visual tracking, Fraunhofer IGD also uses hardware tracking solutions, such as industrial electromagnetic sensors, for its tracking applications. These general tracking solutions can be used for various applications, especially AR applications. Virtual content, for example, can be shown relative to tracked objects, augmenting the doctor’s field of vision with useful information. Possible display elements are primarily risk and target structures. These include blood vessels or the spinal cord, which can result in increased blood loss and in paralysis of the patient if injured. In display and localization, image processing and registration technologies in particular play a major role and are used in a number of projects at Fraunhofer IGD.
A sample AR application for the OR was developed by Fraunhofer IGD in the 3D Arile project.