Knowledge Guided Disambiguation for Large-Scale Scene Classification with Multi-Resolution CNNs

Abstract

Convolutional Neural Networks (CNNs) have made remarkable progress on scene recognition, partially due to these recent large-scale scene datasets, such as the Places and Places2. Scene categories are often defined by multi-level information, including local objects, global layout, and background environment, thus leading to large intra-class variations. In addition, with the increasing number of scene categories, label ambiguity has become another crucial issue in large-scale classification. This paper focuses on large-scale scene recognition and makes two major contributions to tackle these issues. First, we propose a multi-resolution CNN architecture that captures visual content and structure at multiple levels. The multi-resolution CNNs are composed of coarse resolution CNNs and fine resolution CNNs, which are complementary to each other. Second, we design two knowledge guided disambiguation techniques to deal with the problem of label ambiguity. (i) We exploit the knowledge from the confusion matrix computed on validation data to merge ambiguous classes into a super category. (ii) We utilize the knowledge of extra networks to produce a soft label for each image. Then the super categories or soft labels are employed to guide CNN training on the Places2.

Existing System

Convolutional Hierarchical Recurrent Neural Networks, Directed Acyclic Graph Convolutional Neural Network (DAG-CNN).

Proposed System

In this paper we have studied the problem of scene recognition on large-scale datasets such as the Places, Places2, and LSUN. Large-scale scene recognition suffers from two major problems: visual inconsistence (large intra-class variation) and label ambiguity (small inter-class variation). We developed powerful multi-resolution knowledge guided disambiguation framework that effectively tackle these two crucial issues. We introduced multi-resolution CNNs which are able to capture visual information from different scales. Furthermore, we proposed two knowledge guided disambiguation approaches to exploit extra knowledge, which guide CNNs training toward a better optimization, with improved generalization ability.

Architecture

BLOCK DIAGRAM