Paper
1 May 2022 Expression-assisted facial action unit detection through an attention mechanism and smooth class-weighted Loss
Zhongling Liu, Ziqiang Shi, Rujie Liu, Liu Liu, Xiaoyu Mi, Kentaro Murase
Author Affiliations +
Proceedings Volume 12171, Thirteenth International Conference on Signal Processing Systems (ICSPS 2021); 121710R (2022) https://doi.org/10.1117/12.2631478
Event: Thirteenth International Conference on Signal Processing Systems (ICSPS 2021), 2021, Shanghai, China
Abstract
The performance of facial action unit (AU) detection is often limited owing to the lack of annotated AU data and data imbalance. The data scarcity problem can be partially mitigated by abundant expression data, as AU detection and facial expression recognition (FER) are closely related. Accordingly, in this study, FER and AU detection are trained jointly in a multi-task learning framework, in which FER serves as an auxiliary task for AU detection by providing supplementary information. Meanwhile, we propose to use an attention gate unit between the two tasks to flexibly select valuable information from each other. To address the model bias issue caused by AU data imbalance, a smooth class-weighted loss is adopted to alleviate the dominance of negative AU classes. The best average F1-score obtained using our approach on the BP4D dataset is 63.5%, which is very close to the state-of-the-art performance and exceeds the single task baseline by 3.2%.
© (2022) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Zhongling Liu, Ziqiang Shi, Rujie Liu, Liu Liu, Xiaoyu Mi, and Kentaro Murase "Expression-assisted facial action unit detection through an attention mechanism and smooth class-weighted Loss", Proc. SPIE 12171, Thirteenth International Conference on Signal Processing Systems (ICSPS 2021), 121710R (1 May 2022); https://doi.org/10.1117/12.2631478
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Gold

Data modeling

Convolution

Databases

Binary data

Facial recognition systems

Feature selection

RELATED CONTENT


Back to Top