Conditional GAN-based deep network for seamless large-FOV imaging by camera array

Weihang Zhang; Lianglong Li; Jinli Suo; Qionghai Dai

doi:10.1117/12.2642316

4 January 2023 Conditional GAN-based deep network for seamless large-FOV imaging by camera array

Weihang Zhang, Lianglong Li, Jinli Suo, Qionghai Dai

Proceedings Volume 12317, Optoelectronic Imaging and Multimedia Technology IX; 1231708 (2023) https://doi.org/10.1117/12.2642316
Event: SPIE/COS Photonics Asia, 2022, Online Only

Abstract

Due to limited spatial bandwidth, one has to compromise between large field of view and high spatial resolution in both photography and microscopy. This dilemma largely hampers revealing fine details and global structures of the target scene simultaneously. Recently, a mainstream method is formed by utilizing multiple sensors for synchronous acquisition across different sub-FOVs with high resolution and stitching the patches according to the spatial position of the cameras. Various inpainting algorithms have been proposed to eliminate the intensity discontinuities, but conventional optimization methods are prone to misalignment, seaming artifacts or long processing time, and thus unable to achieve dynamic gap elimination. By taking advantage of generative adversarial networks (GANs) on image generation and padding, we propose a conditional GAN-based deep neural network for seamless gap inpainting. Specifically, a short series of displaced images are acquired to characterize the system configuration, under which we generate patch pairs with and without gap for deep network training. After supervised learning, we can achieve seamless inpainting in gap regions. To validate the proposed approach, we apply our approach on real data captured by large-scale imaging systems and demonstrate that the missing information at gaps can be retrieved successfully. We believe the proposed method holds potential for all-round observation in various fields including urban surveillance and systems biology.

Conference Presentation

Citation Download Citation

Weihang Zhang, Lianglong Li, Jinli Suo, and Qionghai Dai "Conditional GAN-based deep network for seamless large-FOV imaging by camera array", Proc. SPIE 12317, Optoelectronic Imaging and Multimedia Technology IX, 1231708 (4 January 2023); https://doi.org/10.1117/12.2642316

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available