Recently published guidelines recommend the validation of models for medical image processing against variations of image acquisition parameters. This work presents a novel concept for systematic tests that investigate a model’s performance to MRI acquisition shifts in brain scans of multiple sclerosis patients. Contrast changes of T2-weighted FLAIR images related to changes of two scan parameters (TE, TI) are simulated using artificial data. These images were created from 114 brain MRI, normal tissue segmentation masks and expert MS lesion masks (NAMIC, OpenMS, MSSEG2, ISBI15, LC08). Contrast changes were simulated by using the FLAIR signal equation. The experiments evaluate the F1 scores of models trained on images based on different uniform and non-uniform acquisition parameters when tested on typical acquisition parameter shifts. The acquisition parameter variations of the experiments are based on guidelines for quality assurance and the analysis of protocols and images from different institutions and open datasets. The dependence of the F1 score on both parameters could be approximated by linear functions with R2 scores of 0.94 to 0.98. Thus, linear functions can be used to model the dependence of the F1 score on the influencing factors, thereby allowing for the derivation of a "save" range of image acquisition parameters to meet a desired performance metric. The segmentation model was more sensitive to changes in TE compared to TI. The maximum lesion F1 loss when applying the models to out-of-distribution data ranged from 0.04 to 0.36 and was significant in all the experiments, even when the model was trained on data representing scans of different contrast (p<0.01). This underlines the need for testing segmentation models against acquisition shifts.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.