Multimodal computational microscopy based on transport of intensity equation

Jiaji Li; Qian Chen; Jiasong Sun; Jialin Zhang; Chao Zuo

doi:10.1117/1.JBO.21.12.126003

5 December 2016 Multimodal computational microscopy based on transport of intensity equation

Jiaji Li, Qian Chen, Jiasong Sun, Jialin Zhang, Chao Zuo

Author Affiliations +

Journal of Biomedical Optics, Vol. 21, Issue 12, 126003 (December 2016). https://doi.org/10.1117/1.JBO.21.12.126003

Abstract

Transport of intensity equation (TIE) is a powerful tool for phase retrieval and quantitative phase imaging, which requires intensity measurements only at axially closely spaced planes without a separate reference beam. It does not require coherent illumination and works well on conventional bright-field microscopes. The quantitative phase reconstructed by TIE gives valuable information that has been encoded in the complex wave field by passage through a sample of interest. Such information may provide tremendous flexibility to emulate various microscopy modalities computationally without requiring specialized hardware components. We develop a requisite theory to describe such a hybrid computational multimodal imaging system, which yields quantitative phase, Zernike phase contrast, differential interference contrast, and light field moment imaging, simultaneously. It makes the various observations for biomedical samples easy. Then we give the experimental demonstration of these ideas by time-lapse imaging of live HeLa cell mitosis. Experimental results verify that a tunable lens-based TIE system, combined with the appropriate postprocessing algorithm, can achieve a variety of promising imaging modalities in parallel with the quantitative phase images for the dynamic study of cellular processes.

1. Introduction

In microscopy imaging, the propagating wave field contains both amplitude and phase information by passing through some biological samples. As a statistical phenomenon of wave field,¹ the amplitude can be recorded by the digital detector (CCD or CMOS camera), whereas the phase, which carries important information about the object’s structure and optical properties, is lost. Traditional bright-field imaging is suitable for observing commonly stained samples or strong absorption objects. However, the shape and density variations are invisible for unstained and transparent biological samples. There are several well-known phase visualization methods for label-free samples, such as the phase contrast²^,³ and differential interference contrast (DIC)⁴^,⁵ imaging, which can provide reliable information about transparent cells and weakly absorbing objects. But neither of those methods gives quantitative phase information, making data interpretation difficult.

Over past decades, a great deal of scientific attention has been paid to the phase retrieval techniques for quantitative phase acquisition. Interference techniques, such as digital holography microscopy,⁶^,⁷ are well-established methods for quantitative phase measurement. However, the traditional interferometric methods rely on coherent illumination, and the speckle noise prevents the formation of high-quality images. Recently, some alternative common path geometries allow for self-interference using white-light illumination by lateral shearing⁸ or spatial decomposition of the object field into its scattered and unscattered components.⁹^–¹¹ Unlike traditional interferometry, these methods with white-light eliminate the coherence noise, thus greatly improving the spatial resolution of measurement. On a different note, noninterferometric quantitative phase imaging can be realized by transport of intensity equation (TIE)¹² using only object field intensities at multiple axially displaced planes. Thus, the direct phase can be uniquely determined by solving the TIE with $I_{0}$ and $\partial I / \partial z$ , where $I_{0}$ and $\partial I / \partial z$ are the determined intensity and the longitudinal intensity derivative at the in-focus plane, respectively. TIE phase imaging has been increasingly investigated during recent years due to its unique advantages over interferometric techniques: it is noninterferometric, works with partially coherent illumination, computationally simple, and does not require phase unwrapping or a complicated optical system. TIE is a second-order elliptic partial differential equation, which can be efficiently solved by Fourier transform-based methods for periodic or some simplified homogeneous boundary conditions,¹³^,¹⁴ or the discrete cosine transform-based method for inhomogeneous Neumann boundary conditions.¹⁵ Because of its computational simplicity and flexible experimental configuration, TIE-based phase imaging approaches have been reported for the characterization of micro-optical elements¹⁶^,¹⁷ and refractive index measurement of optical fibers.¹⁸^,¹⁹ Dynamic TIE phase imaging techniques also have been developed with use of the chromatic aberrations,²⁰ electrically tunable lens,²¹^,²² and spatial light modulators,²³ which further offers the possibility of extending the TIE to the study of fast moving objects and cellular dynamics.

In addition to being used for noninterferometric quantitative phase acquisition, TIE provides the potential for multiple computational observation methods in biomedicine without any other special optical elements. Paganin et al.²⁴ presented the simulated phase contrast images of human femur bone and dust mite in x-ray microscopy imaging. Barone-Nugent et al.²⁵ gave a good experiment result of an emulated DIC image. It has been demonstrated that accurate and high-quality quantitative phase imaging can be achieved based on the TIE with partially coherent illuminations.²⁶^–²⁸ And Zuo et al.²⁹^,³⁰ extended TIE to the general case of partially coherent illumination with the Wigner distribution function (WDF) in phase space. The WDF describes an optical signal in space and frequency domain simultaneously, and the normalized spatial frequency moment of WDF is related to the phase gradient. The physically measurable light field approaches WDF at the geometric optics limit.³¹^,³² By using the angular moment of light field, a three-dimensional (3-D) rotating perspective view of a scene can be presented in light field moment imaging (LMI).³¹

Although TIE has been applied to computational phase contrast, DIC, and LMI in optical microscopy, an integrated efficient multimodal computational imaging system for dynamic cellular processes has not been proposed yet. The goal of this paper is to introduce an effective imaging system that can achieve all three imaging modes simultaneously apart from quantitative phase imaging. And TIE is proved to be a potential method, which makes the various observations for biomedical samples easy. Due to the quantitative results on control samples (e.g., polystyrene beads and microlens array) that have been proved in previous work,²¹^,²⁴^,²⁵ only the results of some biological samples are presented here. On the basis of quantitative phase, images of other modes can be calculated for visualized and complementary information. And the experimental demonstration of fixed unstained samples and time-lapse imaging of live HeLa cell division dynamically will be given.

2. Transport of Intensity Equation for Multimodal Imaging

2.1.

Brief Review of Transport of Intensity Equation

The complex wave field is transferred leaving the object to the imaging plane without magnification or degradation in a perfect paraxial imaging system.²⁵ While the wave field propagates in free-space, the relationship between the intensity and phase in the Fresnel region can be described by the TIE¹²

Eq. (1)

- k \frac{\partial I (r)}{\partial z} = \nabla \cdot [I (r) \nabla φ (r)],

where

k

is the wave number

2 π / λ

,

I (r)

is the intensity of in-focus image,

\nabla

denotes the gradient operator over the transverse direction

r

, and

φ (r)

represents the phase of object. The left hand of TIE is the spatial derivative of intensity at the in-focus plane along the z-axis, and the intensity measurements in multiple planes aim to reduce the impact of the higher order axial intensity derivatives.³³ To minimize the effect of noise, the Savitzky–Golay differentiation filter³⁴ is used here for first-order intensity derivatives along the

z

-axis. The part of a right hand is a second-order elliptic partial differential equation, and we treat it as a Poisson equation, ideally, which can be easily solved with fast-Fourier transform (FFT)²⁷^,³⁵^,³⁶ by introducing an auxiliary function. TIE is originally derived by Teague¹² from Helmholtz equation under paraxial approximation. However, Teague’s TIE as well as his derivation assumes a monochromatic, coherent beam, which might encounter trouble when dealing with fields exhibiting nonnegligible partial coherence. TIE for the partially coherent field with the WDF²⁹^,³⁰ derived from the mutual coherence function is

Eq. (2)

\frac{\partial I (r)}{\partial z} = - λ \nabla_{r} \cdot \int u W (r, u) d u,

where

r

and

u

are the two-dimensional (2-D) space and frequency vectors,

\nabla_{r}

is the gradient operator over

r

, and the representation of intensity

I (r)

equals a projection of the WDF onto

u

plane

\int W (r, u) d u

. Equation (2) reduces to Eq. (1) in the coherent limit, and the normalized spatial frequency moment of WDF is related to the phase gradient

Eq. (3)

\frac{\int u W (r, u) d u}{\int W (r, u) d u} = \frac{1}{2 π} \nabla_{r} φ (r) .

Equations (2) and (3) are the generalized TIE and phase for partially coherent field phase retrieval, respectively. And it has been proved that as long as the source distribution is symmetric with respect to the optical axis, the phase can be accurately retrieved, regardless of the source size (spatial coherence of the illumination, such as Köhler illumination). Thus, the quantitative phase can be recovered using TIE under the coherent illuminations or partially coherent illuminations.

2.2.

Phase Contrast Imaging

Phase contrast microscopy is a typical $4 f$ imaging system by placing a phase-shifting glass plate in the back focal plane to separate different wave components. There are two computational models for phase contrast imaging (PCI), and the detail description is presented as follows.

One way is to assume that the observed sample is a weak-phase object, and the first-order Taylor expansion of complex amplitude can be described as

Eq. (4)

U (r) = \sqrt{I (r)} \exp [i φ (r)] \approx \sqrt{I (r)} [1 + i φ (r)] .

The object is transferred into Fourier plane by lens, and the Fourier transform of

U (r)

is

F (u)

. Then

F (u)

is modulated by the phase-shifting plate whose transfer function is

Eq. (5)

H (u) = {\begin{cases} \exp (i φ_{0}), & u \leq u_{r} \\ 1, & u > u_{r} \end{cases},

where

φ_{0}

is the phase-shift angle of the

D

(diffracted) wave relative to the

S

(surround) wave. This angle is equal to

π / 2

or

- π / 2

generally, and the

u_{r}

is proportional to the radius of the phase-shift plate spot (0.08 NA in simulation). So, the filtered Fourier frequency spectrum is

F^{'} (u) = F (u) H (u)

. Implementing the inverse Fourier transform to

F^{'} (u)

, the intensity of phase contrast image is defined as

Eq. (6)

I_{zer} (r) = {∥ IFT [F^{'} (u)] ∥}^{2} \approx I (r) [1 + 2 φ (r)] .

The above equation shows that there is an approximate linear relationship between the intensity of phase contrast image and phase, therefore, the phase of the object is visible.

Another approach is to take full account of the partially coherent annular oblique illumination (inner ring 0.35 NA, outer ring 0.45 NA, measured experimentally). The phase-shift plate consists of a $- π / 2$ phase-shifting ring with 60% attenuation (size matches the source). In this computational process, an oblique plane wave from each source point shifts the samples spectrum in Fourier domain. Then the wave field is filtered by the pupil function before calculating the intensity image in space domain. The intensity of the phase contrast image is the total intensity contributions of every point on the annular source.³⁷ And the halo artifact issue of partially coherence has been proposed as well.³⁸^,³⁹

The PCI mode in our system is achieved by a virtual phase-shifting ring of phase plate. A phase-shifting ring with 60% attenuation is used for separation of an object’s undiffracted and diffracted component in Fourier domain. Since the quantitative phase is derived from TIE, we could have a pretty good visualization of the unstained object by PCI.

2.3.

Differential Interference Contrast Imaging

In a computational imaging system, a DIC image can be modeled as the superposition of one image over an identical copy that is displaced by a differential amount, $d (r)$ and phase shifted by $π / 2$ .²⁵^,⁴⁰ Extracting the 2-D first-order Taylor series, the results could be written as $A (r + d r) \approx A (r) + d r \cdot \nabla A (r)$ . Applying this approximation and adding phase shift, the distribution of the DIC image is given as

Eq. (7)

A_{DIC} (r) = \sqrt{I (r)} \exp [i φ (r)] + \exp [i φ (r)] [d r \cdot \nabla \sqrt{I (r)} + i \sqrt{I (r)} d r \cdot \nabla φ (r) + \sqrt{I (r)} \exp (i \frac{π}{2})] .

So that the approximative intensity of the DIC image is

Eq. (8)

I_{DIC} = A_{DIC} A_{DIC}^{*} = 2 \sqrt{I (r)} + 2 \sqrt{I (r)} d r \cdot \nabla φ (r) .

In the limiting case of a pure phase object, with intensity

I_{0}

, Eq. (8) reduces to

Eq. (9)

I_{DIC} (r) = 2 I_{0} [1 + d r \cdot \nabla φ (r)] .

The above equation shows that the image is described by the component of the phase gradient in a certain shear direction. So, once the quantitative phase is available, the computational DIC imaging mode can be realized finally.

2.4.

Light Field Moment Imaging

The LMI is introduced by Orth and Crozier,³¹ and this imaging technique uses the continuity equation to extract the first angular moments of light field. Utilizing these moments to construct perspective views of a scene, a 3-D view can be present from different perspectives. The definition of scalar potential $S (r)$ ³¹ to solve angular moments is given as

Eq. (10)

\nabla S (r) = I (r) M (r),

where

M (r)

denotes a vector containing the normalized first moments of the light field over its angular coordinates. In fact, Eq. (10) is the auxiliary function introduced by Teague¹² for coherent illumination

Eq. (11)

\nabla Γ (r) = I (r) \nabla φ (r) .

For a general case of partially coherent illumination, the $M (r)$ is defined as

Eq. (12)

\frac{\int u W (r, u) d u}{\int W (r, u) d u} = \frac{1}{2 π} M (r),

from Eq. (3). The detailed derivation of WDF from mutual coherent function is presented by Zuo et al.³⁰ Under a geometric optical limitation scene,³² it has been proved the equivalence of the WDF and the light field in phase space.²⁹^,³⁰ And applying

W (r, λ u)

to

L (r, θ)

approximately, the redefinition of angular moments is

Eq. (13)

M (r) = \frac{\int θ L (r, θ) d θ}{\int L (r, θ) d θ},

where

L (r, θ)

is the light field. Equation (13) is just another definition of Eq. (3) about the angular moments of the light field. Therefore, it can be seen from Eqs. (1)–(3), (12), and (13) that the “phase” measured by TIE is a scalar whose gradient yields the normalized first-order local moments of WDF. The phase gradient is equal to the angular moment of the light field in LMI at the approximation of geometric optics limit.

Thus, once the deterministic phase is retrieved through TIE, the angular moments can be easily calculated using Eq. (3). Assuming the angular distribution of light ray obeys Gaussian statistics, then the specification of the four-dimensional light field is fully constituted by

Eq. (14)

L (r, θ) = I (r) \exp {\frac{- [θ - M {(r)}^{2}]}{σ^{2}}},

where

σ^{2}

is equal to

{NA}^{2}

, and the Gaussian assumption enables the reconstruction of a scene as viewed at different perspectives by choosing 2-D slices of the light field in Eq. (14) at a given

(θ_{x}, θ_{y})

coordinate,⁴¹ then the light field moment image can be presented. While the angular distribution is Gaussian, a good perspective shifting effect could be achieved, but it is still groundless and lacks in physical evidence.³⁰^,³¹

3. Experimental Configuration and Sample Preparation

The experimental setup is based on an inverted IX71 (Olympus, Tokyo, Japan) microscope as shown in Fig. 1(b), wherein a halogen white-light source with a green interference filter (central wavelength $λ = 550 nm$ , 45-nm bandwidth) is used for illumination. The microscope is equipped with a 5-MP CCD camera (Olympus UC50, $2588 \times 1960 pixels$ , $3.4 - μ m$ pixel pitch, Germany) and an electrically tunable lens (EL-C-10-30-VIS-LD, Optotune AG, Switzerland) module. The electrically tunable lens module is synchronized with the camera at different focal distances along the $z$ -axis and controlled by software via a standard USB cable. The image stack is acquired via plan semiapochromat objective (Olympus, LUCPLFLN $40 \times$ , Half magnification, NA 0.6) in 8-bit grayscale range. Figure 1(a) shows the schematic of experimental setup and an electrically tunable lens in a standard $4 f$ -relay system.

Fig. 1

(a) The schematic of experimental setup and an electrically tunable lens in a standard $4 f$ -relay system. (b) Our multimodal computational imaging system is based on existing microscope Olympus IX71 without any other special optical elements.

The electrically tunable lens changes focal length every 100 ms, and the camera takes five intensity images at different focal distances along the $z$ -axis with subsecond. MATLAB^® software is used for the computation of TIE FFT-solver and the rendering of quantitative phase and multiple modes images. Implementing 2-D FFT on a graphics processing unit (GPU), the time required for all calculation processes (for every five intensity images) of the whole field of view (FOV) ( $2588 \times 1960$ ) is $\sim 1200 ms$ on a laptop computer (Intel Core i7-6700 CPU, 3.4 GHz, 8 GB RAM, NVidia GeForce GTX970M GPU). Therefore, the achievable speed for the multimodal imaging system is per frame within 2 s. In a time-lapse series, an image stack along the $z$ -axis is recorded every 2 min over a period of 266 min considering slow cell changes.

For the experiments, the fixed samples are the unstained pollen grains. The medium between the glass microscope slide and coverslip is formaldehyde solution. And HeLa cells are cultivated in a glass bottom Petri dish (35 mm, MatTek) with L-glutamine Dulbecco’s modified Eagles medium (Gibco, American) supplemented with 10% Nu-serum (Corning, American), 10% fetal calf serum (Gibco, Australia), and 1% vitamin mix ( $100 \times$ ) (Lonza, Cologne, Germany). The cells are cultured in a stage-mounted climate chamber (Tokai Hit INUF-IX3W, Japan) for stabilization of temperature at 37°C and ${CO}_{2}$ gas at 5%. The medium is changed every other day and cells are passed with trypsin upon reaching 80% confluency. In preparation for cell division imaging, cells are washed once with phosphate-buffered saline and detached with either accutase or trypsin.

4. Results

4.1.

System Resolution Limits and Comparison of Multimodal

In our system, a Savitzky–Golay differentiation filter and five intensity images at two different defocus distances are used for the calculation of quantitative phase. Figure 2(a) shows the phase transfer function of TIE for different distances, and the distances $Δ z_{1}$ and $Δ z_{2}$ depend on the transfer threshold value $α$ .

Fig. 2

(a) Phase transfer function of TIE for different distances is combined for optimal transfer function. $f_{1}$ is the normalized frequency corresponding to threshold value $α$ , which equals 0.55, and $f_{cutoff}$ is the system cutoff frequency that determined system frequency limit $NA / λ$ . (b) The grains in cell are distinguishable and the minimum distance between two spots is $1 μ m$ (6 pixel pitch) approximately. Scale bar represents 25 and $1 μ m$ , respectively.

The optimal spatial frequency components in Fourier domain for each distance are combined as shown in Fig. 2(a). $f_{1}$ is the normalized frequency corresponding to threshold value $α$ and $f_{cutoff}$ is the system cutoff frequency, which determines system resolution limits. Figure 2(b1) is the quantitative phase of interphase HeLa cell. Figures 2(b2) and 2(b3) are the enlarged images of grains in cytoplasm. The theoretical system resolution limit $λ / NA$ is equal to $0.92 μ m$ in our imaging system. These spots are distinguishable under Abbe diffraction limit, and the measured distance between two grains is $1 μ m$ (6 pixel pitch) approximately in Figs. 2(b2) and 2(b3). Thus, the resolution limits of this system basically reach theoretical value.

Before presenting multimodal results of time-lapse live HeLa cell mitosis, we compare the actual phase contrast and DIC images with the computational multimodal images. Figure 3 shows the comparison results of an unstained lung cancer cell (LC-06). The computational phase contrast image takes the off-axis illumination into account fully. From this figure, the results are similar to each other, but the actual phase contrast image gives more contrast effects with the least halo artifact in some region. And the DIC image offers better 3-D bas-relief effect of phase change region than the computational one.

Fig. 3

The comparison results of unstained lung cancer cell (LC-06). (a),(c), and (e) The bright-field image at in-focus plane, and the images of phase contrast and DIC in reality, respectively. (b) The measured quantitative phase based on TIE. (d) and (f) The multimodal results of phase contrast and DIC calculated from phase image, respectively. Scale bar represents $25 μ m$ .

4.2.

Experiments

First, the unstained pollen grains are used for multiple modes observation. Pollen grains are an important source to obtain information over time about flora, therefore it is important to implement several tools to make palinotaxonomy characterization in biology.⁴² The refractive index of pollen grain is about 1.43.⁴² Figure 4 shows the multimodal imaging results of pollen grains. The pollen grain cannot be treated as a thin phase object and has a slight absorption at the margin of the object. The effects of LMI are not significant, so we will not present the LMI experimental results of pollen grains.

Fig. 4

The images of multimodal imaging system for unstained pollen grains. (a) The bright-field image at the in-focus plane and (b) is the quantitative phase image. (c) and (d) The experimental results of phase contrast and DIC image, respectively. Scale bar represents $100 μ m$ .

Then, our system is used for the live HeLa cells imaging. The experimental multiple modes images of mitotic cell and interphase cells are shown in Fig. 5. Figure 5(a) shows the quantitative phase of HeLa cells in the whole imaging plane based on TIE. During the cell cycle phases where the cells adhere on the substrate subcellular regions with a higher density than that of the surrounding areas such as the nucleus, the nuclear membrane and the nucleoli become visible. Thus, three cells begin to divide in our FOV at the end of the G2 phase, which is the final growth and activity stage before mitosis during the cell life cycle.

Fig. 5

(a) When the elapsed time equals 0 min, the live HeLa cells quantitative phase of whole imaging plane via $40 \times$ (half magnification), NA 0.6 objective with $2588 \times 1960 pixels$ . (b) and (c) The experimental results for bright-field intensity, phase contrast, DIC, and quantitative phase images of mitotic cell and interphase cells. The cell is surrounded by halos in (b2) and (c2), which is one of the typical features of phase contrast. The simulated phase contrast image enhances the margin of object and make cell counting easy. (b3) and (c3) are the experimental DIC images giving a 3-D bas-relief effect protruded the region of phase change, such as cytoplasm and cell nucleus. The scale bar denotes 20 and $50 μ m$ , respectively.

We choose the biggest one among three cells as our observed object in the smaller red dotted-line rectangle. Cells in the bigger red dotted-line rectangle neither divide nor change their shape too much during the experimental period. Therefore, we judge that these cells belong to G1 phase or S phase, which lasts 19 h at least, and the cell replicates DNA and centrioles during these phases. When the elapsed time equals 0 min, the multimodal images of mitotic cell and interphase cells are clarified in Figs. 5(b) and 5(c), respectively. The phase contrast and DIC images can provide more visualized and complementary information about the cell. The halo around the cell in Figs. 5(b2) and 5(c2) is one of the typical features of phase contrast. The simulated phase contrast image enhances the margin of objects and makes cell counting easy. Figures 5(b3) and 5(c3) are the experimental DIC images giving a 3-D bas-relief effect and protruding the region of phase change, such as cytoplasm and cell nucleus. Video 1 presents a fast-motion movie of the mitotic cell quantitative phase, phase contrast, and DIC images during the whole experimental period.

The images of the HeLa cell at different time points in multimodal imaging system during mitotic phase are illustrated in Fig. 6. Once the cell becomes rounded, the phase of the cell will be significantly increased. The chromosomes line up at the center of the cell in metaphase or the end of prophase, and the spindle fibers are attached from daughter cells to chromosomes at the centromere. These phenomena denote the final preparation for division. When the elapse time equals 74 min, the cell becomes an ellipse, and the cytoplasm is uniformly distributed in the whole cell in anaphase. The spindle fibers pull chromosomes apart and a chromosome (called chromotid) moves to each daughter cell. After one and a half hours, the cell nucleus is divided into two parts closely and the cell wall pinches in to form the two new daughter cells in telophase. Each daughter cell contains only half a chromosome of the chromosome pair and the size of each is smaller than the normal one. At the end of the experimental period about 236 min, the cytokinesis shows the cell division is finished and two daughter HeLa cells are reproduced. Beyond that, some important features of this process can be observed from phase contrast and DIC images. For example, the halo of Fig. 6(d2) represents that two independent daughter HeLa cells are completely reproduced and Fig. 6(b3) gives an intuitive effect that how the cytoplasm distributed uniformly in the whole cell.

Fig. 6

The images of a HeLa cell at different time points in multimodal imaging system during mitotic phase. (a) The cell rounding in metaphase or end of prophase. The chromosomes line up at the center of the cell, which is the final preparation for division. (b) In anaphase, cell becomes an ellipse, and the cytoplasm is uniformly distributed in the whole cell. (c) The cell nucleus is divided into two parts closely and the cell wall pinches in to form the two new daughter cells in the telophase. (d) The cytokinesis shows the cell division is finished and two daughter HeLa cells are reproduced. In addition, some important features of this process can be observed from phase contrast and DIC images. For example, the halos of (d2) represent that two independent daughter HeLa cells are completely reproduced and (b3) gives an intuitional effect that how the cytoplasm distributed uniformly in the whole cell. A fast-motion movie about this mitosis process during the whole experimental period (see Video 1, MP4, 1,357 KB) [URL: http://dx.doi.org/10.1117/1.JBO.21.12.126003.1].

The morphological changes of HeLa cells at different stages during the mitotic phase are illustrated in Fig. 7. The above-mentioned phenomena can be obtained from those profile curves as well. The chromosomes line up at the center of the cell in the metaphase, so there is a peak at the quantitative phase profile curve in Figs. 7(a). Figure 7(b) shows the cytoplasm is uniformly distributed in the whole cell, thus the profile curve is smooth relatively. While the elapsed time equals 164 min, the cell nucleus is split into two parts closely in telophase and two parts are separated tiny in the profile curve. At the end of experimental period about 236 min, two daughter HeLa cells are reproduced and there are two parts in the curve individually. Therefore, we can get more information about cell mitosis from this profile curve directly.

Fig. 7

The morphological changes of a HeLa cell at different time points during mitotic phase. (a) A peak in cell quantitative phase profile curve represents that the chromosomes line up at the center of the cell in the metaphase (b) The quite large phase areas in cell images indicate the separation of the chromosomes and the cytoplasm is uniformly distributed in the whole cell, thus profile curve is smoothing relatively. (c) The cell nucleus is divided into two parts closely in telophase, so curve is separated tightly. (d) Two daughter HeLa cells are reproduced and there are two peaks individually in curve.

Finally, the LMI is introduced for a HeLa cell from different perspective directions at different time points. In addition to a 3-D rotating perspective view of the scene, some details can be seen more clearly. From 45 deg direction, the protuberant cytoplasm is observed vividly in Fig. 8. And the chromosomes line up at the cell center in metaphase; more details, such as the center protruding part of the cell, are visible clearly. The perspective-shifting animation of a HeLa cell in those phases is shown in Video 2.

Fig. 8

A single shot from Video 2, (MP4, 530 KB) [URL: http://dx.doi.org/10.1117/1.JBO.21.12.126003.2]. More details, such as protuberant cytoplasm, are observed clearly from different perspective direction.

5. Discussion and Conclusion

We have demonstrated a multimodal computational imaging system based on TIE with a bright-field microscope. The images of quantitative phase, Zernike phase contrast, DIC, and light field slice are calculated with an intensity stack captured by an electrically tunable lens module. This system allows for a variety of rapid promising imaging modalities in parallel combined with the appropriate postprocessing algorithm, and simple and straightforward implementation on a conventional microscope at low cost without altering its original high imaging quality. And the multiple modes experimental results of the fixed unstained pollen grains and live HeLa cell mitosis are given in our system as well.

But there are still some issues worthy of our attention. The phase transfer function of TIE equals zero at zero frequency; this may cause the loss of the retrieved phase maps at low frequencies and the final imaging quality relies on the adjustment of algorithm regularization parameter. And the phase transfer function of partially coherent illumination for TIE is an important topic for further quantitative analysis, interpretation, and resolution improvement of the quantitative phase imaging.

In addition to various imaging methods integrated, the potential of TIE for computational imaging is provided in biomedicine without any other special optical elements. The results of live cell imaging demonstrate that the proposed multimodal computational imaging system is a competitive and promising tool for investigations of dynamic cellular processes, such as cell migration, effects of drug effectiveness, and toxicity assessment of biofunctionalized nanomaterials on living cellular systems.

Acknowledgments

The authors have no relevant financial interests in this paper and no other potential conflicts of interest to disclose. This work was supported by the National Natural Science Fund of China (Nos. 11574152 and 61505081), the “Six Talent Peaks” project (No. 2015-DZXX-009, Jiangsu Province, China), the “333 Engineering” Research project (No. BRA2015294, Jiangsu Province, China), the Fundamental Research Funds for the Central Universities (No. 30915011318), and the Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense (No. 3092014012200417). C. Zuo acknowledges the support of the “Zijin Star” program of Nanjing University of Science and Technology.

References

1.

J. W. Goodman, Statistical Optics, John Wiley & Sons, New York (2015). Google Scholar

2.

F. Zernike, “Phase contrast, a new method for the microscopic observation of transparent objects,” Physica, 9 (7), 686 –698 (1942). http://dx.doi.org/10.1016/S0031-8914(42)80035-X Google Scholar

3.

F. Zernike, “How I discovered phase contrast,” Science, 121 (3141), 345 –349 (1955). http://dx.doi.org/10.1126/science.121.3141.345 SCIEAS 0036-8075 Google Scholar

4.

G. Nomarski and A. Weill, “Application la métallographie des méthodes interfrentielles deux ondes polarises,” Rev. Metall., 2 121 –128 (1955). Google Scholar

5.

P. McMahon et al., “Quantitative phase-amplitude microscopy II: differential interference contrast imaging for biological TEM,” J. Microsc., 206 (3), 204 –208 (2002). http://dx.doi.org/10.1046/j.1365-2818.2002.01026.x JMICAR 0022-2720 Google Scholar

6.

M. K. Kim, Digital Holographic Microscopy, Springer, New York (2011). Google Scholar

7.

P. Marquet et al., “Digital holographic microscopy: a noninvasive contrast imaging technique allowing quantitative visualization of living cells with subwavelength axial accuracy,” Opt. Lett., 30 (5), 468 –470 (2005). http://dx.doi.org/10.1364/OL.30.000468 OPLEDP 0146-9592 Google Scholar

8.

P. Bon et al., “Quadriwave lateral shearing interferometry for quantitative phase microscopy of living cells,” Opt. Express, 17 (15), 13080 –13094 (2009). http://dx.doi.org/10.1364/OE.17.013080 OPEXFF 1094-4087 Google Scholar

9.

B. Bhaduri et al., “Diffraction phase microscopy with white light,” Opt. Lett., 37 (6), 1094 –1096 (2012). http://dx.doi.org/10.1364/OL.37.001094 OPLEDP 0146-9592 Google Scholar

10.

B. Bhaduri, K. Tangella and G. Popescu, “Fourier phase microscopy with white light,” Biomed. Opt. Express, 4 (8), 1434 –1441 (2013). http://dx.doi.org/10.1364/BOE.4.001434 BOEICL 2156-7085 Google Scholar

11.

Z. Wang et al., “Spatial light interference microscopy (SLIM),” Opt. Express, 19 (2), 1016 –1026 (2011). http://dx.doi.org/10.1364/OE.19.001016 OPEXFF 1094-4087 Google Scholar

12.

M. R. Teague, “Deterministic phase retrieval: a Greens function solution,” J. Opt. Soc. Am., 73 (11), 1434 –1441 (1983). http://dx.doi.org/10.1364/JOSA.73.001434 JSDKD3 Google Scholar

13.

K. Nugent et al., “Quantitative phase imaging using hard x rays,” Phys. Rev. Lett., 77 (14), 2961 –2964 (1996). http://dx.doi.org/10.1103/PhysRevLett.77.2961 PRLTAO 0031-9007 Google Scholar

14.

J. Frank, S. Altmeyer and G. Wernicke, “Non-interferometric, non-iterative phase retrieval by Greens functions,” J. Opt. Soc. Am. A, 27 (10), 2244 –2251 (2010). http://dx.doi.org/10.1364/JOSAA.27.002244 Google Scholar

15.

C. Zuo, Q. Chen and A. Asundi, “Boundary-artifact-free phase retrieval with the transport of intensity equation: fast solution with use of discrete cosine transform,” Opt. Express, 22 (8), 9220 –9244 (2014). http://dx.doi.org/10.1364/OE.22.009220 OPEXFF 1094-4087 Google Scholar

16.

C. Dorrer and J. Zuegel, “Optical testing using the transport-of-intensity equation,” Opt. Express, 15 (12), 7165 –7175 (2007). http://dx.doi.org/10.1364/OE.15.007165 OPEXFF 1094-4087 Google Scholar

17.

C. Zuo et al., “Boundary-artifact-free phase retrieval with the transport of intensity equation II: applications to microlens characterization,” Opt. Express, 22 (15), 18310 –18324 (2014). http://dx.doi.org/10.1364/OE.22.018310 OPEXFF 1094-4087 Google Scholar

18.

J. Frank et al., “Refractive index determination of transparent samples by noniterative phase retrieval,” Appl. Opt., 50 (4), 427 –433 (2011). http://dx.doi.org/10.1364/AO.50.000427 APOPAI 0003-6935 Google Scholar

19.

M. H. Jenkins, J. M. Long and T. K. Gaylord, “Multifilter phase imaging with partially coherent light,” Appl. Opt., 53 (16), D29 –D39 (2014). http://dx.doi.org/10.1364/AO.53.000D29 APOPAI 0003-6935 Google Scholar

20.

L. Waller et al., “Phase from chromatic aberrations,” Opt. Express, 18 (22), 22817 –22825 (2010). http://dx.doi.org/10.1364/OE.18.022817 OPEXFF 1094-4087 Google Scholar

21.

C. Zuo et al., “High-speed transport-of-intensity phase microscopy with an electrically tunable lens,” Opt. Express, 21 (20), 24060 –24075 (2013). http://dx.doi.org/10.1364/OE.21.024060 OPEXFF 1094-4087 Google Scholar

22.

R. Schubert et al., “Enhanced quantitative phase imaging in self-interference digital holographic microscopy using an electrically focus tunable lens,” Biomed. Opt. Express, 5 (12), 4213 –4222 (2014). http://dx.doi.org/10.1364/BOE.5.004213 BOEICL 2156-7085 Google Scholar

23.

C. Zuo et al., “Noninterferometric single-shot quantitative phase microscopy,” Opt. Lett., 38 (18), 3538 –3541 (2013). http://dx.doi.org/10.1364/OL.38.003538 OPLEDP 0146-9592 Google Scholar

24.

D. Paganin et al., “X-ray Omni microscopy,” J. Microsc., 214 (3), 315 –327 (2004). http://dx.doi.org/10.1111/j.0022-2720.2004.01315.x JMICAR 0022-2720 Google Scholar

25.

E. Barone-Nugent, A. Barty and K. Nugent, “Quantitative phase-amplitude microscopy I: optical microscopy,” J. Microsc., 206 (3), 194 –203 (2002). http://dx.doi.org/10.1046/j.1365-2818.2002.01027.x JMICAR 0022-2720 Google Scholar

26.

T. Gureyev, A. Roberts and K. Nugent, “Partially coherent fields, the transport-of-intensity equation, and phase uniqueness,” J. Opt. Soc. Am. A, 12 (9), 1942 –1946 (1995). http://dx.doi.org/10.1364/JOSAA.12.001942 Google Scholar

27.

D. Paganin and K. A. Nugent, “Noninterferometric phase imaging with partially coherent light,” Phys. Rev. Lett., 80 (12), 2586 –2589 (1998). http://dx.doi.org/10.1103/PhysRevLett.80.2586 PRLTAO 0031-9007 Google Scholar

28.

A. Barty et al., “Quantitative optical phase microscopy,” Opt. Lett., 23 (11), 817 –819 (1998). http://dx.doi.org/10.1364/OL.23.000817 OPLEDP 0146-9592 Google Scholar

29.

C. Zuo, Q. Chen and A. Asundi, “Light field moment imaging: comment,” Opt. Lett., 39 (3), 654 –654 (2014). http://dx.doi.org/10.1364/OL.39.000654 OPLEDP 0146-9592 Google Scholar

30.

C. Zuo et al., “Transport of intensity phase retrieval and computational imaging for partially coherent fields: the phase space perspective,” Opt. Lasers Eng., 71 20 –32 (2015). http://dx.doi.org/10.1016/j.optlaseng.2015.03.006 OLENDN 0143-8166 Google Scholar

31.

A. Orth and K. B. Crozier, “Light field moment imaging,” Opt. Lett., 38 (15), 2666 –2668 (2013). http://dx.doi.org/10.1364/OL.38.002666 OPLEDP 0146-9592 Google Scholar

32.

Z. Zhang and M. Levoy, “Wigner distributions and how they relate to the light field,” in IEEE Int. Conf. on Computational Photography (ICCP’09), 1 –10 (2009). http://dx.doi.org/10.1109/ICCPHOT.2009.5559007 Google Scholar

33.

L. Waller, L. Tian and G. Barbastathis, “Transport of intensity phase-amplitude imaging with higher order intensity derivatives,” Opt. Express, 18 (12), 12552 –12561 (2010). http://dx.doi.org/10.1364/OE.18.012552 OPEXFF 1094-4087 Google Scholar

34.

C. Zuo et al., “Transport-of-intensity phase imaging using Savitzky–Golay differentiation filter-theory and applications,” Opt. Express, 21 (5), 5346 –5362 (2013). http://dx.doi.org/10.1364/OE.21.005346 OPEXFF 1094-4087 Google Scholar

35.

L. Allen and M. Oxley, “Phase retrieval from series of images obtained by defocus variation,” Opt. Commun., 199 (1), 65 –75 (2001). http://dx.doi.org/10.1016/S0030-4018(01)01556-5 OPCOB8 0030-4018 Google Scholar

36.

T. E. Gureyev and K. A. Nugent, “Rapid quantitative phase imaging using the transport of intensity equation,” Opt. Commun., 133 (1), 339 –346 (1997). http://dx.doi.org/10.1016/S0030-4018(96)00454-3 OPCOB8 0030-4018 Google Scholar

37.

L. Mandel and E. Wolf, Optical Coherence and Quantum Optics, Cambridge University Press, Cambridge (1995). Google Scholar

38.

C. Edwards et al., “Effects of spatial coherence in diffraction phase microscopy,” Opt. Express, 22 (5), 5133 –5146 (2014). http://dx.doi.org/10.1364/OE.22.005133 OPEXFF 1094-4087 Google Scholar

39.

T. H. Nguyen et al., “Quantitative phase imaging with partially coherent illumination,” Opt. Lett., 39 (19), 5511 –5514 (2014). http://dx.doi.org/10.1364/OL.39.005511 OPLEDP 0146-9592 Google Scholar

40.

E. Bostan et al., “Phase retrieval by using transport-of-intensity equation and differential interference contrast microscopy,” in IEEE Int. Conf. on Image Processing (ICIP’14), 3939 –3943 (2014). http://dx.doi.org/10.1109/ICIP.2014.7025800 Google Scholar

41.

R. Ng, “Fourier slice photography,” ACM Trans. Graph., 24 (3), 735 –744 (2005). http://dx.doi.org/10.1145/1073204 ATGRDF 0730-0301 Google Scholar

42.

F. A. M. Ramirez, “Simultaneous measure of refractive integral index and nexine thickness of a pollen grain by digital holographic microscopy,” in Digital Holography and Three-Dimensional Imaging, (2014). Google Scholar

Biographies for the authors are not available.

Citation Download Citation

Jiaji Li, Qian Chen, Jiasong Sun, Jialin Zhang, and Chao Zuo "Multimodal computational microscopy based on transport of intensity equation," Journal of Biomedical Optics 21(12), 126003 (5 December 2016). https://doi.org/10.1117/1.JBO.21.12.126003

Published: 5 December 2016

Access the abstract

JOURNAL ARTICLE
9 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

CITATIONS

Cited by 20 scholarly publications.

Explore citations on Lens.org

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Imaging systems

Phase contrast

Digital image correlation

Microscopy

Wigner distribution functions

Computing systems

Multimodal imaging

1.

Introduction

2.

Transport of Intensity Equation for Multimodal Imaging

2.1.

Brief Review of Transport of Intensity Equation

Eq. (1)

Eq. (2)

Eq. (3)

2.2.

Phase Contrast Imaging

Eq. (4)

Eq. (5)

Eq. (6)

2.3.

Differential Interference Contrast Imaging

Eq. (7)

Eq. (8)

Eq. (9)

2.4.

Light Field Moment Imaging

Eq. (10)

Eq. (11)

Eq. (12)

Eq. (13)

Eq. (14)

3.

Experimental Configuration and Sample Preparation

Fig. 1

4.

Results

4.1.

System Resolution Limits and Comparison of Multimodal

Fig. 2

Fig. 3

4.2.

Experiments

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

5.

Discussion and Conclusion

Acknowledgments

References

Show All Keywords

Keywords/Phrases

Search In:

Publication Years