LEARNet: Dynamic Imaging Network for Micro Expression Recognition

Abstract

Unlike prevalent facial expressions, micro expressions have subtle, involuntary muscle movements which are short-lived in nature. These minute muscle movements reflect true emotions of a person. Due to the short duration and low intensity, these micro-expressions are very difficult to perceive and interpret correctly. In this paper, we propose the dynamic representation of micro-expressions to preserve facial movement information of a video in a single frame. We also propose a Lateral Accretive Hybrid Network (LEARNet) to capture micro-level features of an expression in the facial region. The LEARNet refines the salient expression features in accretive manner by incorporating accretion layers (AL) in the network. The response of the AL holds the hybrid feature maps generated by prior laterally connected convolution layers. Moreover, LEARNet architecture incorporates the cross decoupled relationship between convolution layers which helps in preserving the tiny but influential facial muscle change information. The visual responses of the proposed LEARNet depict the effectiveness of the system by preserving both high- and micro-level edge features of facial expression. The effectiveness of the proposed LEARNet is evaluated on four benchmark datasets: CASME-I, CASME-II, CAS(ME)^2 and SMIC.

Existing System

Robust PCA (RPCA) and Local Spatiotemporal Directional Features (LSTD)

Proposed System

In this paper, dynamic imaging and CNN-based approach (LEARNet) is cohesively used to classify the micro expressions from expression image sequences. In this paper we propose a LEARNet model, that focuses in learning salient features of micro expressions. The LEARNet is designed to process multiple feature instances through laterally connected layers. It also incorporates the accretion layer to enhance the feature learning capability of the neurons. These accretion layers culminate the hybrid feature responses by combining the previous lateral layer outcomes. Accretion layers maintain the feature map quality by preserving the non-trivial features and improve the discriminability performance of the network. Furthermore, to represent the spatio-temporal features we have elicited the dynamic images from the video sequences, which incorporate features of the both appearance and motion dynamics. These images are processed by the LEARNet. Lateral Accretive Hybrid network is designed to spot the micro level involuntary changes in an expression sequence and classify them according to basic emotion classes.

Architecture

BLOCK DIAGRAM