Track Everything: Limiting Prior Knowledge in Online Multi-Object Recognition

Abstract

This paper addresses the problem of online tracking and classification of multiple objects in an image sequence. Our proposed solution is to first track all objects in the scene without relying on object-specific prior knowledge, which in other systems can take the form of hand-crafted features or user-based track initialization. We then classify the tracked objects with a fast learning image classifier that is based on a shallow convolutional neural network architecture and demonstrate that object recognition improves when this is combined with object state information from the tracking algorithm. We argue that by transferring the use of prior knowledge from the detection and tracking stages to the classification stage we can design a robust, general purpose object recognition system with the ability to detect and track a variety of object types.

Existing System

Neural Network, Deep Convolutional Neural Network.

Proposed System

We have presented a system for online object recognition that can autonomously locate and recognize multiple types of objects using biologically inspired what and where processing streams. Our proposed solution is to first track all objects in the scene without relying on object-specific prior knowledge. Our overall approach may be characterized as a shift of the use of object-specific prior knowledge out of the where stream and into the what stream. This enables the where stream, which is implemented as a general purpose multi object tracking algorithm, to locate every salient object in the scene, including sources of occluding or distracting clutter. Online recognition of localized objects is then handled by reintegration of the what and where processing streams. This takes the form of a SLFN ensemble that combines object tracking state information with class label estimate information from the S-CNN to provide robust object recognition outputs, the performance of which is comparable to the state-of-the-art.

Architecture

BLOCK DIAGRAM