|
1.IntroductionConcentration of tissue constituents, such as hemoglobin, water, or lipid, provides vital functional information about tissue health, which can assist healthcare practitioners in making important decisions. For instance, quantitative information about tissue oxygenation and blood volume fraction can provide visual assistance during surgery and monitoring tissue condition during and after treatment.1 Even though devices that provide this vital information at a single point already exist, there is a constant push and need toward attaining noncontact, real-time, wide-field and quantitative mapping of tissue functional properties. To address this need, spatial frequency domain imaging (SFDI) is a promising candidate within the field of diffuse optical imaging that has already found commercial application in aiding medical treatment.2 In essence, SFDI exploits the properties of interaction of spatially modulated light (i.e., structured illumination) with the tissue by measuring the “blurring” of the projected fringes and using light propagation modeling to extract optical properties.3 As depicted in Fig. 1(a), a typical imaging system consists of a projection module capable of projecting patterned light at a single wavelength, and a camera for recording images of the field-of-view. Polarizers, arranged in a cross configuration, are commonly used to reduce the effect of specular reflectance and record only diffuse light out of the tissue. The modulation amplitudes of the sample of interest are obtained at each projected spatial frequency by illuminating, acquiring raw data, and demodulating images using multiple phases3 (using SFDI) or a single phase4 [using single snapshot of optical properties (SSOP)]. A calibration measurement using a reference phantom with known optical properties is also imaged to obtain modulation amplitudes at the same spatial frequencies . The diffuse reflectance of the tissue at spatial frequency is then given by , where is the simulated diffuse reflectance of the calibration phantom at spatial frequency . Finally, a light propagation model is used to extract optical properties from the measured diffuse reflectance at several spatial frequencies. This workflow is illustrated in Fig. 1(b). Because entire images are processed at once in the frequency domain, SFDI has shown the potential to provide very rapid maps of optical properties. The speed of the method can be typically optimized by (1) using a fast acquisition and demodulation method, such as SSOP, and (2) using a fast inversion method to recover the optical properties from the measured data.5 It should be noted that both are necessary for SFDI to perform rapidly. To accelerate the entire process, it has been shown that using only two spatial frequencies can provide accurate estimation of optical properties with errors under 7%.3,5,6 In a two-frequency SFDI scheme, the amplitudes at low frequency (typically ) and a high frequency (e.g., ) are recorded, and the corresponding reflectance maps are obtained using the same calibration step. The diffuse reflectance at these two spatial frequencies and the optical properties have a unique, nonlinear, one-to-one mapping relationship.3,5,6 This nonlinear mapping relationship is shown in Fig. 2 for a wide range of values of the optical properties. Given the diffuse reflectance images obtained at two spatial frequencies, the task is to solve an inverse problem of obtaining the optical properties, namely the absorption coefficient () and the reduced scattering coefficient (). The inversion is possible by fitting the diffuse reflectance to either diffusion approximation of photon transport or by using a Monte-Carlo (MC) based simulation of photon migration in tissues.3 Using either of these two light propagation models, two basic approaches have been used so far to perform this inversion, namely, least-square fit and look-up table (LUT) method.3,5,6 On the one hand, precise and robust fitting usually requires diffuse reflectance at multiple frequencies, but it is computationally slow to fit a large number of pixels using either the analytical function provided by the standard diffusion approximation or, especially, with the MC-based simulation. On the other hand, the LUT method uses a pregenerated table of diffuse reflectance for a wide range of optical properties, generally using interpolation to estimate the optical properties.5 The LUT method is relatively fast but the computational complexity increases with the density of the grid used to generate the LUT. For a low-resolution LUT, the accuracy of estimation is hampered. A compromise between computational time and accuracy has to be made to use the LUT method. Recent methods that improve the computational time many folds have been proposed.5 However, inversion at multiple spatial frequencies and scaling the LUT method to include more parameters other than reflectance has significant computational time and/or memory costs. In this study, we propose an alternative method for the inversion using a publicly available machine learning technique to estimate the optical properties maps from diffuse reflectance images applied to SFDI. A large set of simulated data using MC was used to train a random forest regression (RFR) algorithm to attain the inversion, allowing in return to obtain directly optical properties maps from diffuse reflectance images in the frequency domain with similar errors than state-of-the-art LUT methods. In the next sections, the method is detailed and its performances evaluated against standard models (diffusion and MC-LUT) on simulated and real experimental data. 2.Material and Methods2.1.Random Forest Regressor MethodMachine learning and regression techniques like artificial neural networks and RFR have received an increased interest for processing large amounts of data, and more recently, for replacing time-consuming model-base algorithms in diffuse optics.7–10 What makes such methods attractive is their capacity to perform particularly well in learning nonlinear mappings. In the spatial frequency domain, the mapping between and is strongly nonlinear, as demonstrated in Fig. 2. One possible implementation of machine learning in our case consists of generating a training set with a standard forward model provided by a MC simulation and creating the inverse mapping by a machine learning algorithm. In this study, RFR, which is an ensemble learning algorithm, has been chosen as it typically works well with the default hyperparameter settings.10 The training set was generated by using the MC simulation on MATLAB for a fixed frequency of and refractive index of 1.43. A wide range of optical properties, with and , was used and the optical properties space was uniformly and randomly sampled to obtain data points. For each pair of randomly generated optical properties, the reflectance were obtained using the MC simulation. Then, a random forest regressor implemented by scikit-learn package in python11 was trained on this dataset to obtain a reverse transformation from to . The number of estimator was fixed to 15 for relatively fast calculation and the default minimum samples at leaf were set to 2.10 The trained model was saved and used as test over another randomly generated independent test set. 2.2.Inversion MethodsFollowing demodulation and calibration, an inversion method must be used to recover the optical properties [Fig. 1(b)]. Four different methods of inversion were used in this study: the standard diffusion approximation (diffusion), a dense LUT generated from MC simulations of points in and (MC LUT), a low-resolution LUT generated from MC simulation of points in and (MC LUT low), and the RFR algorithm described previously. To compare computation times, all inversions were performed using MATLAB on a standard PC desktop, have 16 GB of RAM and an Intel I5-7500 processor (6 MB/4T/3,4 GHz). 2.3.Imaging SystemThe instrumental setup was custom built using a digital micromirror device (Vialux, Germany) for the projection of custom patterns, fiber-coupled to a 665-nm laser diode (LDX Optronics, Maryville, Tennessee). The projection system projects a sine wave pattern over a field-of-view at 45-cm working distance. Images were acquired using a scientific-grade monochrome CCD camera (PCO pixelfly, Kelheim, Germany). Polarizers (PPL05C, Moxtek, Orem, Utah), arranged in a crossed configuration, are used to minimize the contribution from specular reflections at the surface of the sample. A silicone-based optical phantom was used for calibration and built using titanium dioxide () as a scattering agent and India ink as an absorbing agent.12 Its large size () accommodates the system’s field-of-view with reduced scattering and absorption at 665 nm. 2.4.Validation Experiments2.4.1.Simulation experimentsAn independent test set of 106 optical properties was uniformly and randomly generated for a wide range of optical properties, with and and used to compute diffuse reflectance at two spatial frequencies (0 and ) using MC simulations. Diffuse reflectance values were then inverted using all models (diffusion, MC LUT, MC LUT low and RFR) and compared. 2.4.2.Phantom experimentsSilicone-based optical phantoms were built using titanium dioxide () as a scattering agent and India ink as an absorbing agent. Six small phantoms having various optical properties ranging from 0.0135 to in absorption, and from 0.982 and in reduced scattering were made to test the RFR method against the state-of-the-art inversion model, the MC LUT. Acquisitions were performed by our imaging system at two spatial frequencies (0 and ) and processed using the RFR and MC LUT methods. A region in the neighborhood of a central pixel at each phantom was chosen and mean and standard deviation of the optical properties were computed. 3.Results3.1.Simulation ExperimentsResults from simulation experiments are shown in Fig. 3. The plot shows expected optical properties versus optical properties predicted by the diffusion approximation, MC LUT, and the RFR method, which are shown in cyan, red, and blue, respectively. Overall, the MC LUT inversion shows 0.035% and 0.003% mean relative error in estimation of absorption and reduced scattering coefficients, respectively. However, such large size LUT can take up to 8.8 s to compute the optical properties for a set with data points. The RFR method computes the predictions in 0.45 s (nonparallelized) and produces a mean relative of error of 0.556% (in ) and 0.126% (in ). To obtain, a similar run time on MC LUT, we compare the prediction of the MC LUT Low obtained with data points. The run time was improved to 0.43 s, but the accuracy of prediction was deteriorated to 1.86% (in ) and 0.097% (in ). 3.2.Phantom ExperimentsFigure 4 shows the comparison of the estimated optical properties on six tissue-simulating phantoms by plotting the mean and standard deviation values obtained from each method against each other. The optical properties estimated from RFR method and the dense LUT method are shown to be similar with a maximum difference of 0.46% in absorption and 0.026% in reduced scattering. 3.3.In Vivo ExperimentsThe optical properties maps of an in vivo human hand obtained from the MC LUT, diffusion approximation, MC MUT low, and RFR methods are shown in Fig. 5. As expected, the MC LUT and the RFR methods perform similarly while the diffusion method visually exhibits deviations in both absorption and scattering. To further quantify these results, the percentage error map with respect to the MC LUT was computed. The diffusion approximation shows a mean percentage error of and in estimation of and , respectively, where the mean is taken over a region indicated by the black rectangle shown in Fig. 5. The mean error in a low resolution MC-LUT is 0.8% and in estimation of and , respectively, while the corresponding error for the RFR method is 0.16% and 0.01% in estimation of and , respectively. 4.DiscussionIn this study, we presented a machine learning approach for estimating optical property maps from diffuse reflectance images in the spatial frequency domain. A training set was generated from MC simulations and used to train a RFR algorithm. The RFR method was then tested and validated onto simulations, phantoms, and in vivo onto a human hand, against a dense () MC LUT. The accuracy of the RFR method was shown to be slightly degraded compared with the MC LUT in all cases, with differences on average less than 0.16% in absorption and 0.01% in reduced scattering in vivo. With very little difference in the accuracy of the recovered optical properties compared to LUTs, the main interest of this work is to propose an alternative approach to estimating optical properties for SFDI without the need of a photon propagation model. Other machine learning algorithms have been proposed in diffuse optics and in the spatial frequency domain.7–10 The originality of this work is to propose to apply machine learning to process megapixel images rapidly (450 ms) for SFDI while providing accuracies similar to state-of-the-art, dense LUT methods. While our implementation is not faster than previously published work using optimized LUTs,5 this study lays the foundation for using machine learning methods for SFDI with the purpose to replace model-based inversion methods to provide images in real-time during surgery. Our machine learning inversion method can certainly be optimized and improved greatly in processing time by using GPU processing, making such approaches potentially as fast as or even faster than LUT-based methods. Coupled with acquisition methods,4,13,14 both acquisition and processing can be performed in real-time, a requirement to operate in time-constrained environments such as the operating room. While limited to simple inversion from diffuse reflectance images at two spatial frequencies to obtain optical properties (similar to LUTs), machine learning is particularly interesting for its versatility as it can be easily scaled to more complex data environments for surgical guidance, such as multispatial frequencies analysis, where the instantaneous spatial frequency at each pixel may differ such as during endoscopic SFDI,15 multispectral imaging to quantitatively estimate the concentrations of oxyhemoglobin, deoxyhemoglobin, and oxygenation16 or fluorescence imaging to quantitatively estimate local the fluorescence concentration.17 The method can also be extended to depth-sensitive measurements,7 tomography,18 3-D profile correction,19 metabolic imaging,20 and chromophores mapping21 and their combination22 in real-time. These measurement modalities make use of a series of models that are usually nonlinear. Given the access to large amounts of simulated or experimental data, these processes can be learned by machine learning models and provide significant improvements in computation time while preserving accuracies of more computationally costly methods. In addition, GPU implementations of machine learning models can help in speeding up the processing of such complex imaging modalities. 5.ConclusionMachine learning methods provide a versatile means for replacing nonlinear models to estimate optical properties from diffuse reflectance images in SFDI without compromising accuracy of estimation. In this study, we presented an RFR algorithm that we trained to perform the nonlinear mapping between diffuse reflectance images and optical properties maps in the spatial frequency domain. The model we presented is capable of computing megapixel maps of optical properties with similar accuracies compared with a standard MC-based LUT. Along with recent developments in real-time acquisition methods in the spatial frequency domain, this work lays the foundation for an integrated imaging system performing acquisition and processing of wide-field images in real-time for surgical applications. AcknowledgmentsThis work was supported by the European Research Council (ERC, award# 715737, S.G.) and the University of Strasbourg IdEx (S.G.). ReferencesS. Gioux et al.,
“First-in-human pilot study of a spatial frequency domain oxygenation imaging system,”
J. Biomed. Opt., 16
(8), 086015
(2011). https://doi.org/10.1117/1.3614566 JBOPFO 1083-3668 Google Scholar
D. J. Cuccia et al.,
“Quantitation and mapping of tissue optical properties using modulated imaging,”
J. Biomed. Opt., 14
(2), 024012
(2009). https://doi.org/10.1117/1.3088140 JBOPFO 1083-3668 Google Scholar
J. Vervandier and S. Gioux,
“Single snapshot imaging of optical properties,”
Biomed. Opt. Express, 4
(12), 2938
–2944
(2013). https://doi.org/10.1364/BOE.4.002938 BOEICL 2156-7085 Google Scholar
J. Angelo et al.,
“Ultrafast optical property map generation using lookup tables,”
J. Biomed. Opt., 21
(11), 110501
(2016). https://doi.org/10.1117/1.JBO.21.11.110501 JBOPFO 1083-3668 Google Scholar
T. A. Erickson et al.,
“Lookup-table method for imaging optical properties with structured illumination beyond the diffusion theory regime,”
J. Biomed. Opt., 15
(3), 036013
(2010). https://doi.org/10.1117/1.3431728 JBOPFO 1083-3668 Google Scholar
D. Yudovsky and A. J. Durkin,
“Spatial frequency domain spectroscopy of two layer media,”
J. Biomed. Opt., 16
(10), 107005
(2011). https://doi.org/10.1117/1.3640814 JBOPFO 1083-3668 Google Scholar
M. Jager, F. Foschum and A. Kienle,
“Application of multiple artificial neural networks for the determination of the optical properties of turbid media,”
J. Biomed. Opt., 18
(5), 57005
(2013). https://doi.org/10.1117/1.JBO.18.5.057005 JBOPFO 1083-3668 Google Scholar
Y. W. Chen and S. H. Tseng,
“Efficient construction of robust artificial neural networks for accurate determination of superficial sample optical properties,”
Biomed. Opt. Express, 6
(3), 747
–760
(2015). https://doi.org/10.1364/BOE.6.000747 BOEICL 2156-7085 Google Scholar
S. J. Wirkert et al.,
“Robust near real-time estimation of physiological parameters from megapixel multispectral images with inverse Monte Carlo and random forest regression,”
Int. J. Comput. Assist. Radiol. Surg., 11
(6), 909
–917
(2016). https://doi.org/10.1007/s11548-016-1376-5 Google Scholar
“Scikit-learn machine learning in Python,”
(2018) http://scikit-learn.org/ ( October ). 2018). Google Scholar
F. Ayers et al.,
“Fabrication and characterization of silicone-based tissue phantoms with tunable optical properties in the visible and near infrared domain,”
Proc. SPIE, 6870 687007
(2008). https://doi.org/10.1117/12.764969 PSISDG 0277-786X Google Scholar
K. P. Nadeau, A. J. Durkin and B. J. Tromberg,
“Advanced demodulation technique for the extraction of tissue optical properties and structural orientation contrast in the spatial frequency domain,”
J. Biomed. Opt., 19
(5), 056013
(2014). https://doi.org/10.1117/1.JBO.19.5.056013 JBOPFO 1083-3668 Google Scholar
M. van de Giessen, J. P. Angelo and S. Gioux,
“Real-time, profile-corrected single snapshot imaging of optical properties,”
Biomed. Opt. Express, 6
(10), 4051
–4062
(2015). https://doi.org/10.1364/BOE.6.004051 BOEICL 2156-7085 Google Scholar
J. P. Angelo, M. van de Giessen and S. Gioux,
“Real-time endoscopic optical properties imaging,”
Biomed. Opt. Express, 8
(11), 5113
–5126
(2017). https://doi.org/10.1364/BOE.8.005113 BOEICL 2156-7085 Google Scholar
A. M. Laughney et al.,
“Spectral discrimination of breast pathologies in situ using spatial frequency domain imaging,”
Breast Cancer Res., 15
(4), R61
(2013). https://doi.org/10.1186/bcr3455 BCTRD6 Google Scholar
P. A. Valdes et al.,
“qF-SSOP: real-time optical property corrected fluorescence imaging,”
Biomed. Opt. Express, 8
(8), 3597
–3605
(2017). https://doi.org/10.1364/BOE.8.003597 BOEICL 2156-7085 Google Scholar
S. D. Konecky et al.,
“Quantitative optical tomography of sub-surface heterogeneities using spatially modulated structured light,”
Opt. Express, 17
(17), 14780
–14790
(2009). https://doi.org/10.1364/OE.17.014780 OPEXFF 1094-4087 Google Scholar
Y. Zhao et al.,
“Angle correction for small animal tumor imaging with spatial frequency domain imaging (SFDI),”
Biomed. Opt. Express, 7
(6), 2373
–2384
(2016). https://doi.org/10.1364/BOE.7.002373 BOEICL 2156-7085 Google Scholar
M. Ghijsen et al.,
“Quantitative real-time optical imaging of the tissue metabolic rate of oxygen consumption,”
J. Biomed. Opt., 23
(3), 036013
(2018). https://doi.org/10.1117/1.JBO.23.3.036013 JBOPFO 1083-3668 Google Scholar
A. Mazhar et al.,
“Wavelength optimization for rapid chromophore mapping using spatial frequency domain imaging,”
J. Biomed. Opt., 15
(6), 061716
(2010). https://doi.org/10.1117/1.3523373 JBOPFO 1083-3668 Google Scholar
A. E. Cerussi et al.,
“Frequent optical imaging during breast cancer neoadjuvant chemotherapy reveals dynamic tumor physiology in an individual patient,”
Acad. Radiol., 17
(8), 1031
–1039
(2010). https://doi.org/10.1016/j.acra.2010.05.002 Google Scholar
|