Image restoration employing cross-ViT combined generative adversarial networks

Hanxiao Liu; Lin Li

doi:10.1117/12.3038514

23 August 2024 Image restoration employing cross-ViT combined generative adversarial networks

Hanxiao Liu, Lin Li

Author Affiliations +

Proceedings Volume 13250, Fourth International Conference on Image Processing and Intelligent Control (IPIC 2024); 132500P (2024) https://doi.org/10.1117/12.3038514
Event: Fourth International Conference on Image Processing and Intelligent Control (IPIC 2024), 2024, Kuala Lumpur, Malaysia

Abstract

We propose Cross-ViT GAN, a novel image restoration model that addresses the limitations of CNNs by integrating Cross-ViT into GANs. The Cross-ViT GAN includes both generators and discriminators. In the generator, we replace the original normalization layer with a self-modulation layer, allowing the intermediate feature map to adapt to the input noise vector during backpropagation, thus enhancing generated image quality. The discriminator incorporates Cross-ViT, dividing the input image into large and small branches and using cross-attention to exchange features between different scales, enhancing image features for better recognition. We use three datasets, CelebA, Palces2 and CIFAR-10, to evaluate the performance of the model. The experimental results show that the evaluation metrics PSNR:26.48, SSIM:0.923, IS:4.11, and FID:1.65 of the Cross-ViT GAN model in the CelebA dataset outperform those of other comparative models, which proves the effectiveness of the Cross-ViT GAN model.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Hanxiao Liu and Lin Li "Image restoration employing cross-ViT combined generative adversarial networks", Proc. SPIE 13250, Fourth International Conference on Image Processing and Intelligent Control (IPIC 2024), 132500P (23 August 2024); https://doi.org/10.1117/12.3038514

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

;

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
6 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Gallium nitride

Image restoration

Transformers

Data modeling

RGB color model

Education and training

Image enhancement

Show All Keywords

Keywords/Phrases

Search In:

Publication Years