Virtually Transparent Epidermal Imagery

Abstract:

We have been developing a novel virtually transparent epidermal imagery (VTEI) system for laparo-endoscopic single-site (LESS) surgery. The system uses a network of multiple, micro wireless cameras and multiview mosaicing technique to obtain a panoramic view of the surgery area. This view provides visual feedback to surgeons with large viewing angles and areas of interest so that the surgeons can improve the safety of surgical procedures by being better aware of where the surgical instruments are relative to tissue and organs. The prototype VTEI system also projects the generated panoramic view on the abdomen area to create a transparent display effect that mimics equivalent, but higher risk, open-cavity surgeries. In addition, in order to improve situational awareness for the surgeon and reduce morbidity and mortality, we are working on an approach to create an augmented-reality-based system that can generate accurate visual overlay of patient specific radiologic data in minimally invasive surgery (MIS). This approach challenges the existing paradigm in surgery, which relies on surgeon experience and judgment when performing blind dissection to identify hidden anatomic structures such as those found in the retroperitoneum or within an organ.  To maintain correct registration, laparoscope should be tracked and the reliable features should be detected and used for the tracking. For special conditions in an MIS imaging environment, such as specular reflections and homogeneous  areas, the feature points extracted by general feature point detectors are less distinctive and repeatable in MIS images. We observe that abundant blood vessels are available on tissue surfaces and can be extracted as a new set of features to combine with general feature points for better MIS image analysis. In this paper, two types of blood vessel features are proposed for endoscopic images: branching points and branching segments.  Extensive in vivo experiments were conducted to evaluate the performance of the proposed methods and compare them with the state-of-the-art methods. The numerical results verify that, in MIS images, the blood vessel features can produce a large number of points. More importantly, those points are more robust and repeatable than the other types of features. Meanwhile, the computational complexity of the proposed methods is linear with respect to the number of pixels in images. A fundamental problem in rendering pre-operative data on inter-operative video using augmented reality technique is characterizing how visualizations affect surgeon perception. This problem is important because procedure outcomes depend on surgeon ability to perceive virtual and real world structures and their interrelation. AR techniques such as x-ray vision are designed to compensate for depth cues lost while rendering hidden structures. Every AR enhancement rendered is necessarily a deviation, which may obtrude. We carried out a study on depth rendering techniques and analysis and then proposed a framework and protocols for evaluation of AR rendering techniques in-vivo and an experiment comparing silhouette ghosting to a proposed minimally obtrusive forensic ghosting AR rendering technique called grid shadowing. The results provide prima facie evidence that the established protocol enables quantitative comparison of depth judgment among AR techniques in surgery and new techniques are needed.