BODY STRUCTURE AWARE DEEP CROWD COUNTING

Abstract

Crowd counting is a challenging task, mainly due to the severe occlusions among dense crowds. This work aims to take a broader view to address crowd counting from the perspective of semantic modelling. In essence, crowd counting is a task of pedestrian semantic analysis involving three key factors: pedestrians, heads, and their context structure. The information of different body parts is an important cue to help us judge whether there exists a person at a certain position. Existing methods usually perform crowd counting from the perspective of directly modelling the visual properties of either the whole body or the heads only, without explicitly capturing the composite body-part semantic structure information that is crucial for crowd counting. In our approach, we first formulate the key factors of crowd counting as semantic scene models. Then, we convert the crowd counting problem into a multi-task learning problem such that the semantic scene models are turned into different sub-tasks. Finally, the deep convolutional neural networks (CNNs) are used to learn the sub-tasks in a unified scheme. Our approach encodes the semantic nature of crowd counting and provides a novel solution in terms of pedestrian semantic analysis. In experiments, our approach outperforms the state-of-the-art methods on four benchmark crowd counting datasets. The semantic structure information is demonstrated to be an effective cue in scene of crowd counting.

Existing System

Density map based on the annotated points with a 2D Gaussian kernel

Proposed System

For the purpose of accurately estimating the count of pedestrians, we reformulate crowd counting as a multi-task learning problem. There are three sub-tasks: inferring two types of semantic scene models and estimating the crowd count. We build deep convolutional neural networks (CNNs) to jointly learn these sub-tasks. The CNNs first model the mappings from scene image to semantic scene models including the body part map and structured density map, followed by calculating the crowd count based on them. The CNNs are able to extract powerful visual representations from images. The feature extraction and multi-task crowd counting problem are addressed in a unified scheme. We summarize our main contributions as follows: 1) We provide a novel solution for crowd counting in terms of pedestrian semantic analysis. We formulate three key factors of crowd counting and model them as two types of semantic scene models. The models recover rich semantic structure information from images and are effective in learning our crowd counting framework. 2) We reformulate the crowd counting problem as multitask learning problem such that the semantic scene models are converted into its sub-tasks. We present a unified framework to jointly learn these sub-tasks based on the CNNs. Experiments show that our method achieves better results compared to the state-of-the-art methods.

Architecture

BLOCK DIAGRAM