The injected ICG must be detected and tracked inside the body using NIR cameras and image processing methods. The ICG excitation is achieved through absorption of infrared light that passes the upper dermal layers. For excitation, IR LEDs are utilized. The resulting fluorescent light of the ICG can be detected using the employed NIR cameras. Due to very similar light wavelength of the absorbed and the emitted light, the use of hard camera filters is essential.
The lymph node tracking and 3D model generation requires a stereo camera calibration of the utilized camera systems. To correctly display the resulting 3D lymph node model in the HMD, the relative transformation between camera and display system must be determined. This should be done using a third normal optical camera, which is attached to the HMD. For this, a marker based tracking is implemented that is as robust and fast as possible.
After transformation calculation the generated 3D model can be displayed in the HMD by applying the determined transformation between camera system and display. This will result in an accurate overlay of lymph node information with the patient anatomy, enhancing the clinician’s online view of the operation scenario.