Moving objects detection using a thermal Camera and IMU on a vehicle

TLDR: In this article, a novel algorithm for moving object detection in thermal images taken by a moving thermal camera is presented, which uses fusion of an inertial measurement unit (IMU) and a thermal camera.

Abstract: In this paper we present a novel algorithm for moving object detection in thermal images taken by a moving thermal camera. It allows a detection of moving objects in thermal images of low quality without imposing restrictions on the temperature and/or shape of the object. The main assumption required for good performance of the algorithm is that the transversal movement of the vehicle will not produce significant change in the optical flow of the static objects in the scene between two consecutive image frames. Our algorithm does not use any temperature thresholds and works well in urban environments detecting moving humans and other moving objects as well. To achieve this we use fusion of an inertial measurement unit (IMU) and a thermal camera. First we use IMU data to compensate for rotational movements of the thermal camera between two consecutive thermal images. Then we differentiate those images and filter the resulting image based on dense optical flow calculated using Farneback technique. After that moving objects are detected and further filtering is applied using random sampling consensus algorithm based on optical flow model.