Facial Expression Recognition Using Frequency Neural Network

Abstract

Facial expression recognition has become a newly-emerging topic in recent decades, which has important value in the field of human-computer interaction. In this paper, we present a deep learning based approach, named frequency neural network (FreNet), for facial expression recognition. Different from convolutional neural network in spatial domain, FreNet inherits the advantages of processing image in frequency domain, such as efficient computation and spatial redundancy elimination. First, we propose the learnable multiplication kernel and construct multiple multiplication layers to learn features in frequency domain. Second, a summarization layer is proposed following multiplication layers to further yield high-level features. Third, based on the property of discrete cosine transform (DCT), we utilize multiplication layers and summarization layer to construct the Basic-FreNet, which can yield high-level features on the widely used DCT feature. Finally, to further achieve better performance on Basic-FreNet, we propose the Block-FreNet in which the weight-shared multiplication kernel is designed for feature learning and the block sub-sampling is designed for dimension reduction.

Existing System

Convolutional neural networks (CNNs)

Proposed System

We propose a novel frequency based deep learning approach for FER. To our best knowledge, it is the first attempt to fill in the blank of frequency based deep learning model for FER. We propose the LMK and construct multiplication layers for feature learning in frequency domain. The multiplication layers can learn features on the widely used handcrafted feature: upper-left DCT coefficients and obtain high-level features. We propose the summarization layer following the multiplication layer to further yield high-level features, which improves discriminability of the learned feature and provides better performance. We propose the Block-FreNet, in which the weight-shared LMK is designed for feature learning and the BSS is designed for dimension reduction in frequency domain, to further achieve better performance.

Architecture

BLOCK DIAGRAM