REAL-TIME OBJECT TRACKING ON A DRONE WITH MULTI-INERTIAL SENSING DATA

Abstract

Real-time object tracking on a drone under a dynamic environment has been a challenging issue for many years, with existing approaches using off-line calculation or powerful computation units on board. This paper presents a new lightweight real-time onboard object tracking approach with multi-inertial sensing data, wherein a highly energy-efficient drone is built based on the Snapdragon flight board of Qualcomm. The flight board uses a digital signal processor core of the Snapdragon 801 processor to realize PX4 autopilot, an opensource autopilot system oriented toward inexpensive autonomous aircraft. It also uses an ARM core to realize Linux, robot operating systems, open-source computer vision library, and related algorithms. A lightweight moving object detection algorithm is proposed that extracts feature points in the video frame using the oriented FAST and rotated binary robust independent elementary features algorithm and adapts a local difference binary algorithm to construct the image binary descriptors. The K-nearest neighbor method is then used to match the image descriptors. Finally, an object tracking method is proposed that fuses inertial measurement unit data, global positioning system data, and the moving object detection results to calculate the relative position between coordinate systems of the object and the drone. All the algorithms are run on the Qualcomm platform in real time. Experimental results demonstrate the superior performance of our method over the state-of-the-art visual tracking method.

Existing System

Binary robust independent elementary features (BRIEF), local difference binary (LDB).

Proposed System

The main contributions of this paper can be summarized as follows: (1) A lightweight object detection algorithm on a drone is proposed that uses an Oriented FAST and Rotated Binary Robust Independent Elementary Features (ORB) algorithm under an outdoor environment for feature extraction, LDB for feature binary descriptors, and k-nearest neighbor (KNN) to match the image descriptors. (2) An object tracking method is proposed that fuses object detection results, Euclidean space equations, IMU, and Global Positioning System (GPS) data to calculate the relative position between coordinate systems of the drone and the object. (3) A highly energy-efficient drone is built based on the Snapdragon flight board of the Qualcomm. Our proposed object detection and tracking method can achieve real-time performance in the drone.

Architecture

BLOCK DIAGRAM