Structured illumination microscopy (SIM) has been widely applied in the superresolution imaging of subcellular dynamics in live cells. Higher spatial resolution is expected for the observation of finer structures. However, further increasing spatial resolution in SIM under the condition of strong background and noise levels remains challenging. Here, we report a method to achieve deep resolution enhancement of SIM by combining an untrained neural network with an alternating direction method of multipliers (ADMM) framework, i.e., ADMM-DRE-SIM. By exploiting the implicit image priors in the neural network and the Hessian prior in the ADMM framework associated with the optical transfer model of SIM, ADMM-DRE-SIM can further realize the spatial frequency extension without the requirement of training datasets. Moreover, an image degradation model containing the convolution with equivalent point spread function of SIM and additional background map is utilized to suppress the strong background while keeping the structure fidelity. Experimental results by imaging tubulins and actins show that ADMM-DRE-SIM can obtain the resolution enhancement by a factor of ∼1.6 compared to conventional SIM, evidencing the promising applications of ADMM-DRE-SIM in superresolution biomedical imaging.
KEYWORDS: Image restoration, Signal to noise ratio, Mitochondria, Light sources and illumination, Reconstruction algorithms, Microscopy, Deep learning, Model based design, Photobleaching, Photonics
Structured illumination microscopy (SIM) has been widely used in live-cell superresolution (SR) imaging. However, conventional physical model-based SIM SR reconstruction algorithms are prone to artifacts in handling raw images with low signal-to-noise ratios (SNRs). Deep-learning (DL)-based methods can address this challenge but may lead to degradation and hallucinations. By combining the physical inversion model with a total deep variation (TDV) regularization, we propose a hybrid restoration method (TDV-SIM) that outperforms conventional or DL methods in suppressing artifacts and hallucinations while maintaining resolutions. We demonstrate the performance superiority of TDV-SIM in restoring actin filaments, endoplasmic reticulum, and mitochondrial cristae from extremely low SNR raw images. Thus TDV-SIM represents the ideal method for prolonged live-cell SR imaging with minimal exposure and photodamage. Overall, TDV-SIM proves the power of integrating model-based reconstruction methods with DL ones, possibly leading to the rapid exploration of similar strategies in high-fidelity reconstructions of other microscopy methods.
The emergence of super-resolution (SR) fluorescence microscopy has rejuvenated the search for new cellular sub-structures. However, SR fluorescence microscopy achieves high contrast at the cost of the lack of a holistic view of their interacting partners and surrounding environment. Thus we develop SR fluorescence-assisted diffraction computational tomography (SR-FACT), which combines label-free three-dimensional optical diffraction tomography (ODT) with two-dimensional fluorescence Hessian structured illumination microscopy. The ODT module is capable of resolving mitochondria, lipid droplets, the nuclear membrane, chromosomes, the tubular endoplasmic reticulum and lysosomes. Using dual-mode correlated live cell imaging for prolonged period of time, we observe the dynamics of a novel subcellular structure named dark-vacuole bodies. These works demonstrate the unique capabilities of SR-FACT, which suggest its wide applicability in cell biology in general.
To increase the temporal resolution and maximal imaging time of super-resolution (SR) microscopy, we have developed a deconvolution algorithm for structured illumination microscopy based on Hessian matrixes (Hessian-SIM). It uses the continuity of biological structures in multiple dimensions as a priori knowledge to guide image reconstruction and attains artifact-minimized SR images with less than 10% of the photon dose used by conventional SIM. Hessian-SIM enables spatiotemporal resolution of 88 nm and 188 Hz, and hour-long time-lapse SR imaging of actin filaments in live cells. Finally, we observed the structural dynamics of mitochondrial cristae and structures that, to our knowledge, have not been observed previously, such as enlarged fusion pores during vesicle exocytosis.
This work is a result of an exploration in response to the following observation: though reconstruction methodology of SIM image since its origin is carried out in frequency domain, recent works in this field, specially when it comes to SIM reconstruction using lesser than 9 raw images, have ventured into reconstruction methodologies that operate directly in the spatial domain. This work formulates and demonstrates a frequency domain reconstruction of SIM image using four raw images – one wide-field image and three SIM images. One of the chief feature of the presented reconstruction algorithm is that it employs the standard ‘tools and tricks’ used by the conventional 9-frame SIM reconstruction algorithm. Results indicate that the presented reconstruction algorithm provides high resolution reconstructions successfully as long as the noise level in the raw images is lower than 10%. For higher level of noise, the reconstruction result shows little resolution enhancement.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.