Spectral imaging is at the cornerstone of many fields, including astronomy, environmental monitoring, food processing, agriculture, and biomedical imaging. While most current spectral techniques lack imaging speed, we describe a computational approach that allows for fast high spectral resolution imaging.
Our set-up maps the image of the scene into the fiber of a compact spectrometer through a digital micromirror device (DMD), where a series of multiplexing patterns are uploaded. After a reconstruction step, the hypercube of the scene can be recovered. DMDs represent the fastest technology (e.g., >20 kHz) for spatial light modulators. The raw data is acquired for a (x,y,λ) hypercube of size 64x64x2048 in about 12 s, while the Hadamard inversion takes <1 s. The acquisition speed can be further reduced by limiting the number of Hadamard patterns; however, the resulting imaging reconstruction problem is turned into an underdetermined inverse problem, which requires regularization techniques to be used to obtain acceptable solutions, in particular, in the presence of noise.
Deep learning is a very efficient framework to solve inverse problems in imaging. Following a recent trend, several convolution neural network architectures have provided a link between deep and optimization-based image reconstruction methods. Contrary to the initially proposed “black box” networks, these deep-learning methods rely on a forward operator and lead to more interpretable networks. Here, we review deep architectures for single-pixel image reconstruction and show that the network can be trained easily, in a end-to-end manner, using databases such as STL-10 or ImageNet.
We present reconstruction results from simulated and experimental single-pixel acquisitions. We show that EM-Net generalizes very well to noise levels that are unseen during the training, despite having fewer learned parameters than alternative methods. The proposed EM-Net generally reconstructs images with fewer artifacts and with higher signal-to-noise ratios, particularly in high-noise situations.
Single-pixel imaging is a paradigm that enables the capture of an image from a single point detector using a spatial light modulator. This approach is particularly interesting for optical set-ups where pixelated arrays of detectors are either too expensive or too cumbersome (e.g., multispectral, infrared imaging). It acquires the inner product between the image of the scene and a set of user-defined patterns that are sequentially uploaded onto the spatial light modulator. Compressed data acquisition reduces the acquisition time, although it leads to an ill-posed reconstruction problem, which is very challenging for real-time applications. Recently, neural networks have emerged as competitive alternatives to traditional reconstruction methods. Neural networks are parametric models that are trained by exploiting large datasets. Their noniterative nature allows for fast reconstructions, which opens the door to real-time image reconstruction from compressed acquisition. In this study, we evaluate the different networks for static and dynamic imaging. In particular, we introduce a recurrent neural network that is designed to exploit the spatiotemporal redundancy in videos via a memory state. We validate our algorithms on simulated data from the UCF-101 dataset, with a resolution of 128x128 pixels and a compression ratio of 98%. We also show experimentally that we can resolve small spectral differences in the spectrum of human skin measured in vivo.
Pattern generalization was proposed recently as an avenue to increase the acquisition speed of single-pixel imaging setups. This approach consists of designing some positive patterns that reproduce the target patterns with negative values through linear combinations. This avoids the typical burden of acquiring the positive and negative parts of each of the target patterns, which doubles the acquisition time. In this study, we consider the generalization of the Daubechies wavelet patterns and compare images reconstructed using our approach and using the regular splitting approach. Overall, the reduction in the number of illumination patterns should facilitate the implementation of compressive hyperspectral lifetime imaging for fluorescence-guided surgery.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.