Robotic and endoscopic surgery is increasingly used in clinical practice and typically relies on stereoscopic vision to enable 3D visualization of the surgical field. We combined this capability with a FLIm acquisition system suitable for the identification of tumor tissue to generate a 3D map of the surgical field that comprises both FLIm and white light image information. This result is achieved using semi-global matching and deep stereo matching neural network. In addition to the generation of a 3D model of the surgical cavity, this approach leads to a more realistic rendering of FLIm maps by including tissue shading.
Accurate cancer margin assessment prior to surgical resection is a key factor influencing the long-term survival of oral and oropharyngeal cancer patients. This leads to the need for additional guidance tools for real-time delineation of cancer margins. In this work, fiber-based fluorescence lifetime Imaging (FLIm) was combined with machine learning to perform intraoperative tumor identification. The developed classifier achieved a measurement-level ROC-AUC of 0.89±0.03 on an N=62 patient dataset. A transparent overlay of classifier output was augmented onto the surgical field and updated through tissue motion correction, ensuring co-registration between tissue and spectroscopic data/classifier output was maintained during imaging..
KEYWORDS: Visualization, Augmented reality, Tumors, Luminescence, Tissues, Surgery, Information visualization, Data acquisition, Real time imaging, Navigation systems
Real-time visualization of imaging data constitutes a critical part of surgical workflow. Augmented reality (AR) is a promising tool to assist in conventional surgical navigation systems. We have been developing an AR framework for clinical imaging and guidance using an optical see-through head-mounted display (OST-HMD) and fluorescence lifetime imaging (FLIm) instrumentation. This framework supports in vivo scanning of FLIm data and the real-time visualization of diagnostic information overlaid on the interrogated tissue area. With the high discriminative power of FLIm, our FLIm-AR concept has the potential for indicating tumor margins and assisting with tumor excision surgery.
In this work, we evaluate the potential for Fluorescence Lifetime Imaging (FLIm) to complement a surgeon's visual, endoscopic, and pathologic assessment of the adequacy of intraoperative tumor resection in clinical cancer applications of the oral cavity and oropharynx. Using a custom-built FLIm instrument during both non-robotic and robotic assisted surgical procedures, we show that intrapatient contrast between healthy and tumor tissue can be achieved intraoperatively in vivo prior to cancer resection with statistical significance (p<0.001) in 9/9 patients using at least 1/6 FLIm parameters, and ex vivo for surgically excised specimens (p<0.001) for 8/9 patients. We employ a multi-parameter linear discriminant analysis approach to demonstrate superior pathology discrimination ability through leveraging a weighted combination of all FLIm metrics. We also highlight interpatient comparisons to evaluate how FLIm signatures vary across different patients and disparate tissue anatomies.
The literature articulates the importance of advancing novel solutions which enable clinicians to intraoperatively resolve pathological tissue from healthy tissue in situ in order to guide the accuracy and efficiency of surgical tumor resection. A method which non-invasively provides real-time delineation of cancer margins has great potential to improve clinical outcomes by accelerating surgical procedural times, ensuring complete tumor resection, and by enabling more conservative resection approaches which preserves healthy tissue. Autofluorescence lifetime imaging is a powerful technique which holds great promise in addressing this clinically unmet need. Using a custom built, fiber-optic based, multi-spectral time-resolved fluorescence spectroscopy (ms-TRFS) instrument (excitation 355 nm) applied to cancer within head & neck anatomy, our preliminary results from 13 human patients indicate that tumor vs. healthy tissue regions (confirmed via histology) can be distinguished on the basis of lifetime and intensity ratio for both in vivo (pre-resection) and ex vivo (post-resection) applications. Each of the three major ms-TRFS spectral bands demonstrate highly conserved lifetime and intensity ratio trends within specific tissue types (palate, palatine tonsil, lingual tonsil, & base of tongue) for cancerous regions when juxtaposed to neighboring healthy peripheral tissue. Current results demonstrate distinct lifetime and intensity ratio results when comparing across tissue types. Collectively, our initial data suggests that time-resolved autofluorescence could serve as a valuable tool for providing real-time intraoperative diagnosis and surgical guidance during robot-assisted cancer removal in otolaryngologic applications.
An important step in establishing the diagnostic potential for emerging optical imaging techniques is accurate registration between imaging data and the corresponding tissue histopathology typically used as gold standard in clinical diagnostics. We present a method to precisely register data acquired with a point-scanning spectroscopic imaging technique from fresh surgical tissue specimen blocks with corresponding histological sections. Using a visible aiming beam to augment point-scanning multispectral time-resolved fluorescence spectroscopy on video images, we evaluate two different markers for the registration with histology: fiducial markers using a 405-nm CW laser and the tissue block’s outer shape characteristics. We compare the registration performance with benchmark methods using either the fiducial markers or the outer shape characteristics alone to a hybrid method using both feature types. The hybrid method was found to perform best reaching an average error of 0.78±0.67 mm. This method provides a profound framework to validate diagnostical abilities of optical fiber-based techniques and furthermore enables the application of supervised machine learning techniques to automate tissue characterization.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.