Landmark Free Face Attribute Prediction

Abstract

Face attribute prediction in the wild is important for many facial analysis applications yet it is very challenging due to ubiquitous face variations. In this paper, we address face attribute prediction in the wild by proposing a novel method, Andmark Free Face Attribute prediction (AFFAIR). Unlike traditional face attribute prediction methods that require facial landmark detection and face alignment, AFFAIR uses an end to- end learning pipeline to jointly learn a hierarchy of spatial transformations that optimize facial attribute prediction with no reliance on landmark annotations or pre-trained landmark detectors. AFFAIR achieves this through simultaneously learning a global transformation which effectively alleviates negative effect of global face variation for the following attribute prediction tailored for each face, locating the most relevant facial part for attribute prediction and aggregating the global and local features for robust attribute prediction. Within AFFAIR, a new competitive learning strategy is developed that effectively enhances global transformation learning for better attribute prediction. We show that with zero information about landmarks, AFFAIR achieves state-of-the-art performance on three face attribute prediction benchmarks, which simultaneously learns the face-level transformation and attribute-level localization within a unified framework.

Existing System

Face attribute prediction is usually tackled via a Detection- Alignment-Recognition (DAR) pipeline. Within DAR, an off-the-shelf face detector is used to detect faces in images in the detection stage. Then in the alignment stage a face landmark detector is applied to faces, followed by the operation of establishing correspondence between detected landmarks and canonical locations where domain experts’ input is required. Finally faces are aligned by transformations estimated from the correspondence. In the recognition stage, features are extracted from the aligned faces and fed into a classifier to predict the face attributes. However, the alignment stage in the DAR pipeline suffers several issues. It heavily depends on the quality of landmark detection results. Despite good performance on near frontal faces, the current state-of-the-art face landmark detectors cannot give satisfactory results on unconstrained faces with large pose angles, occlusion or blurriness. The error in landmark localization would definitely harm the performance of attribute prediction. Besides, even with accurate facial landmarks, one still needs to handcraft specific face alignment protocols (canonical locations, transformation methods, etc.), which demands dense domain expert knowledge. Some warping artifacts of mapping landmark locations to canonical positions are also inevitable in aligning the faces. Thus facial attribute prediction error accumulates due to a combination of erroneous off-the-shelf landmark detection and handcrafted protocols. More importantly, the DAR alignment process is decoupled from the objective of predicting facial attributes. That is, the alignment process is not explicitly optimized for the objective of predicting facial attributes. This brings the need for an end-to-end learning process that finds facial transformations without distorting the faces and at the same time is optimized for the objective of predicting facial attributes.

Proposed System

The proposed AFFAIR provides an end-to-end learning framework for finding the appropriate transformation that optimizes the final objective of facial attribute prediction without requiring face landmark information or pre-trained landmark detectors. AFFAIR has a novel transformation-localization architecture that adaptively transforms any face deviated from a normal one and locates the most discriminative facial part for attribute prediction. AFFAIR introduces a novel competitive learning strategy to effectively augment the learning of good global transformation tailored for each face without requiring extra supervision information.

Architecture

ARCHITECTURE DIAGRAM