Multi-scale multi-feature context modeling for scene recognition in the semantic manifold

Abstract

This paper aims to develop a novel cost-effective framework for face identification, which progressively maintains a batch of classifiers with the increasing face images of different individuals. By naturally combining two recently rising techniques: active learning (AL) and self-paced learning (SPL), our framework is capable of automatically annotating new instances and incorporating them into training under weak expert recertification. We first initialize the classifier using a few annotated samples for each individual, and extract image features using the convolutional neural nets. Then, a number of candidates are selected from the unannotated samples for classifier updating, in which we apply the current classifiers ranking the samples by the prediction confidence. In particular, our approach utilizes the high-confidence and low-confidence samples in the self-paced and the active user-query way, respectively. The neural nets are later fine-tuned based on the updated classifiers. Such heuristic implementation is formulated as solving a concise active SPL optimization problem, which also advances the SPL development by supplementing a rational dynamic curriculum constraint. The new model finely accords with the “instructor-student-collaborative” learning mode in human education. The advantages of this proposed framework are two-folds: i) The required number of annotated samples is significantly decreased while the comparable performance is guaranteed. A dramatic reduction of user effort is also achieved over other state-of-the-art active learning techniques. ii) The mixture of SPL and AL effectively improves not only the classifier accuracy compared to existing AL/SPL methods but also the robustness against noisy data.

Existing System

Sufficient Spanning Set Approximation, Convex Optimization, Support Vector Machine (SVM).

Proposed System

In this paper, we have introduced, first, an effective framework to solve incremental face identification, which build classifiers by progressively annotating and selecting unlabeled samples in an active self-paced way, and second, a theoretical interpretation of the proposed framework pipeline from the perspective of optimization Third, we evaluate our approach on challenging scenarios and show very promising results. The main contributions of this work are several folds. i) To the best of our knowledge, our work is the first one to make a face recognition framework capable of automatically annotating high-confidence samples and involve them into training without need of extra human labor in a purely self-paced manner under weak recertification of active learning. Especially in that along the learning process, we can achieve more and more pseudo labeled samples to facilitate learning totally for free. Our framework is thus suitable in practical large-scale scenarios. The proposed framework can be easily extended to other similar visual recognition tasks. ii) We provide a concise optimization problem and theoretically interpret that the proposed ASPL is a rational implementation for solving this problem. iii) This work also advances the SPL development, by setting a dynamic curriculum variation. The new SPL setting better complies with the “instructor-student-collaborative” learning mode in human education than previous models. iv) Extensive experiments on challenging CACD and CASIA-WebFace datasets show that our approach is capable of achieving competitive or even better performance under only small fraction of sample annotations than that under overall labeled data. A dramatic reduction (> 30%) of user interaction is achieved over other state-of-the-art active learning methods.

Architecture

BLOCK DIAGRAM