3 May 2023 Adaptive adjustment with semantic embedding for zero-shot object detection
Wen Lv, Hongbo Shi, Shuai Tan, Bing Song, Yang Tao
Author Affiliations +
Abstract

Traditional zero-shot object-detection algorithms detect images of untrained classes in the model with the help of semantic embedding. However, these approaches may perform poorly due to the limitations of fixed semantic embedding. Given that fixed semantic attributes lead to a lack of generalization capabilities in the model, a semantic enhancement mechanism is proposed to update the semantic embedding, which is used to serve the needs of the visual space. Specifically, considering that the original semantic space is not enough to construct a visual-semantic mapping relationship, an augmented semantic embedding (ASE) approach is designed to supplement semantic attribute information. Then, a semantic channel attention mechanism is used to adjust the ASE. The adjustment strategy retains adequate attribute information, which is highly relevant to visual features. Finally, to alleviate the domain shift problem, a clustering association strategy is introduced to establish an inferred relationship, which ensures that the predictor is generalized to the unseen domain during training. The superiority of the proposed method is demonstrated by the MS-COCO and PASCAL VOC datasets.

© 2023 SPIE and IS&T
Wen Lv, Hongbo Shi, Shuai Tan, Bing Song, and Yang Tao "Adaptive adjustment with semantic embedding for zero-shot object detection," Journal of Electronic Imaging 32(3), 033001 (3 May 2023). https://doi.org/10.1117/1.JEI.32.3.033001
Received: 2 November 2022; Accepted: 13 April 2023; Published: 3 May 2023
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Semantics

Visualization

Object detection

Education and training

Scanning electron microscopy

Data modeling

Associative arrays

Back to Top