Dallas K. Tada, Pangyu Teng, Kalyani Vyapari, Ashley Banola, George Foster, Esteban Diaz, Grace Hyun J. Kim, Jonathan G. Goldin, Fereidoun Abtin, Michael McNitt-Gray, Matthew S. Brown
PurposeEvaluation of lung fissure integrity is required to determine whether emphysema patients have complete fissures and are candidates for endobronchial valve (EBV) therapy. We propose a deep learning (DL) approach to segment fissures using a three-dimensional patch-based convolutional neural network (CNN) and quantitatively assess fissure integrity on CT to evaluate it in subjects with severe emphysema.ApproachFrom an anonymized image database of patients with severe emphysema, 129 CT scans were used. Lung lobe segmentations were performed to identify lobar regions, and the boundaries among these regions were used to construct approximate interlobar regions of interest (ROIs). The interlobar ROIs were annotated by expert image analysts to identify voxels where the fissure was present and create a reference ROI that excluded non-fissure voxels (where the fissure is incomplete). A CNN configured by nnU-Net was trained using 86 CT scans and their corresponding reference ROIs to segment the ROIs of left oblique fissure (LOF), right oblique fissure (ROF), and right horizontal fissure (RHF). For an independent test set of 43 cases, fissure integrity was quantified by mapping the segmented fissure ROI along the interlobar ROI. A fissure integrity score (FIS) was then calculated as the percentage of labeled fissure voxels divided by total voxels in the interlobar ROI. Predicted FIS (p-FIS) was quantified from the CNN output, and statistical analyses were performed comparing p-FIS and reference FIS (r-FIS).ResultsThe absolute percent error mean (±SD) between r-FIS and p-FIS for the test set was 4.0% (±4.1%), 6.0% (±9.3%), and 12.2% (±12.5%) for the LOF, ROF, and RHF, respectively.ConclusionsA DL approach was developed to segment lung fissures on CT images and accurately quantify FIS. It has potential to assist in the identification of emphysema patients who would benefit from EBV treatment.
KEYWORDS: Computed tomography, Lung, Emphysema, Image segmentation, Education and training, Chest, Voxels, Radio over Fiber, Deep learning, Data modeling
We propose to develop a prediction model that uses the quantitative lung fissure integrity score from chest CT scans that can identify emphysema patients that successfully respond to endobronchial valve (EBV) treatment. It is hypothesized that patients with high fissure integrity are more likely to respond to EBV treatment and achieve volume reduction of the emphysematous lobe. This study retrospectively collected 89 anonymized pre-treatment chest CT exams from patients with moderate to severe emphysema and who underwent EBV treatment. Previous work used a deep learning approach that segmented lung fissure and quantified a fissure integrity score (FIS) for the right horizontal fissure (RHF), right oblique fissure (ROF), and left oblique fissure (LOF). A FIS is defined as the percentage of total fissure voxels present along the interlobar region. Fissures were categorized as complete with a FIS of ≥90%; otherwise, it was considered incomplete. The response to EBV treatment was recorded as the amount of targeted lobe volume reduction (TLVR) compared to target lobe volume prior to treatment. EBV placement was considered successful with a TLVR of ≥350 ml. Statistical analyses were performed separately for each targeted lobe and a logistic regression model was trained using the extracted FIS. From the test set, 8 subjects achieved TLVR with a mean(±SD) FIS of 0.943(±0.052). 23 targeted lobes did not achieve the desired TLVR, with a mean(±SD) FIS of 0.751(±0.201). The EBV prediction model using the FIS achieved an AUC of 0.842. A model using the quantified FIS shows potential as a predictive biomarker for whether a targeted lobe will achieve successful volume reduction from EBV treatment.
Complete pulmonary fissures are assessed on computed tomography (CT) and are required for emphysema patients to be successfully treated with endobronchial valve (EBV) therapy. We propose a deep learning (DL) pipeline that uses a patch-based approach to quantitatively assess fissure completeness on CT and evaluate it in a clinical trial cohort. From the EBV for emphysema palliation trial (VENT), 130 CT scans were used in this study. The DL model utilizes nnU-Net as a backbone for the automatic pre- and post-processing of CT images and configuration of a 3D U-Net to segment patches of fissure and non-fissure. Five-fold cross validation is applied for training and inferences are obtained using a sliding window approach. Average symmetric surface distance (ASSD) and surface dice coefficient (SDC) at a threshold of 2mm evaluates segmentation performance. A fissure integrity score (FIS) is calculated as the percentage of complete fissure voxels along the surface of the assumed interlobar region using pulmonary lobar segmentations. A predicted-FIS (p-FIS) is quantified from the CNN output and is compared to the reference-FIS (r-FIS) as complete (FIS≥90%), partial (10%≤ FIS< 90%) or absent (FIS< 10%). A mean(±SD) SDC of 0.95(±0.037) is achieved for the left oblique fissure (LOF); 0.84(±0.144) for the right horizontal fissure (RHF), and 0.94(±0.098) for the right oblique fissure (ROF). Concordance rate of p-FIS and r-FIS is 86.4%, 88.6%, and 86.4% for the LOF, RHF, and ROF, respectively. A DL pipeline using a patch-based approach has potential to segment interlobar fissures from CT to quantitatively assess fissure completeness.
Kidneys are most easily segmented by convolutional neural networks (CNN) on contrast enhanced CT (CECT) images, but their segmentation accuracy may be reduced when only non-contrast CT (NCCT) images are available. The purpose of this work was to investigate the improvement in segmentation accuracy when implementing a generative adversarial network (GAN) to create virtual contrast enhanced (vCECT) images from non-contrast inputs. A 2D cycleGAN model, incorporating an additional idempotent loss function to restrict the GAN from making unnecessary modifications to data already in the translated domain, was trained to generate virtual contrast enhanced images on 286 paired non-contrast and contrast enhanced inputs. A 3D CNN trained on contrast enhanced images was applied to segment the kidneys in a test set of 20 paired non-contrast and contrast enhanced images. The non-contrast images were converted to virtual contrast enhanced images, then kidneys in both image conditions were segmented by the CNN. Segmentation results were compared to analyst annotations on non-contrast images visually and by Dice Coefficient (DC). Segmentation on virtual contrast enhanced images were more complete with fewer extraneous detections compared to non-contrast images in 16/20 cases. Mean(±SD) DC was 0.88(±0.80), 0.90(±0.03), and 0.95(±0.05) for non-contrast, virtual contrast enhanced, and real contrast enhanced, respectively. Virtual contrast enhancement visually improved segmentation quality, poor performing cases had their performance improved resulting in an overall reduction in DC variation, and the minimum DC increased from 0.65 to 0.85. This work provides preliminary results demonstrating the potential effectiveness of using a GAN for virtual contrast enhancement to improve CNN-based kidney segmentation on non-contrast images.
Two major challenges hinder the deployment of deep learning-based systems into clinical practice: the need for numerous high-quality well-labeled data and the lack of explainability. Attention models, originated from natural language processing, have been popular to address the label scarcity problem and encourage model explainability. In this work, we developed a domain knowledge-guided attention model for disease diagnosis with only coarse scan-level labels and the population-level domain knowledge. The use of guided attention models encourages the deep learning-based diagnosis model to focus on the area of interests in an end-to-end manner. The research interest is to diagnose subjects with idiopathic pulmonary fibrosis (IPF) among subjects with interstitial lung disease (ILD) using an axial chest high resolution computed tomography (HRCT) scan. Our dataset contains 279 IPF patients and 423 non-IPF ILD patients. The network’s performance was evaluated by the area under the receiver operating characteristic curve (AUC). We observe that without attention-based loss function, the IPF diagnosis model reaches satisfactory performance (AUC=0.972), but lack explainability; when increasing the relative importance of attention-based loss, the IPF diagnosis model increases performance (AUC=0.988), along with the model explainability. Our contributions are (1) developing an IPF diagnosis model that only uses scan-level weak supervision; (2) incorporating population-level domain knowledge into the training of IPF diagnosis model in an end-to-end manner; (3) enhancing the explainability of deep learning systems by introducing attention mechanisms.
When mining image data from PACs or clinical trials or processing large volumes of data without curation, the relevant scans must be identified among irrelevant or redundant data. Only images acquired with appropriate technical factors, patient positioning, and physiological conditions may be applicable to a particular image processing or machine learning task. Automatic labeling is important to make big data mining practical by replacing conventional manual review of every single-image series. Digital imaging and communications in medicine headers usually do not provide all the necessary labels and are sometimes incorrect. We propose an image-based high throughput labeling pipeline using deep learning, aimed at identifying scan direction, scan posture, lung coverage, contrast usage, and breath-hold types. They were posed as different classification problems and some of them involved further segmentation and identification of anatomic landmarks. Images of different view planes were used depending on the specific classification problem. All of our models achieved accuracy >99 % on test set across different tasks using a research database from multicenter clinical trials.
A computer-aided method for finding an optimal imaging plane for simultaneous measurement of the arterial blood
inflow through the 4 vessels leading blood to the brain by phase contrast magnetic resonance imaging is presented. The
method performance is compared with manual selection by two observers. The skeletons of the 4 vessels for which
centerlines are generated are first extracted. Then, a global direction of the relatively less curved internal carotid arteries
is calculated to determine the main flow direction. This is then used as a reference direction to identify segments of the
vertebral arteries that strongly deviates from the main flow direction. These segments are then used to identify
anatomical landmarks for improved consistency of the imaging plane selection. An optimal imaging plane is then
identified by finding a plane with the smallest error value, which is defined as the sum of the angles between the plane's
normal and the vessel centerline's direction at the location of the intersections. Error values obtained using the
automated and the manual methods were then compared using 9 magnetic resonance angiography (MRA) data sets. The
automated method considerably outperformed the manual selection. The mean error value with the automated method
was significantly lower than the manual method, 0.09±0.07 vs. 0.53±0.45, respectively (p<.0001, Student's t-test).
Reproducibility of repeated measurements was analyzed using Bland and Altman's test, the mean 95% limits of
agreements for the automated and manual method were 0.01~0.02 and 0.43~0.55 respectively.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.