—Recent object detection models for infrared (IR) imagery are based upon deep neural networks (DNNs) and require large amounts of labeled training imagery. However, publicly-available datasets that can be used for such training are limited in their size and diversity. To address this problem, we explore cross-modal style transfer (CMST) to leverage large and diverse color imagery datasets so that they can be used to train DNN-based IR image based object detectors. We evaluate six contemporary stylization methods on four publicly-available IR datasets – the ﬁrst comparison of its kind - and ﬁnd that CMST is highly effective for DNN-based detectors. Surprisingly, we ﬁnd that existing data-driven methods are outperformed by a simple grayscale stylization (an average of the color channels). Our analysis reveals that existing data-driven methods are either too simplistic or introduce signiﬁcant artifacts into the imagery. To overcome these limitations, we propose meta-learning style transfer (MLST), which learns a stylization by composing and tuning well-behaved analytic functions. We ﬁnd that MLST leads to more complex stylizations without introducing signiﬁcant im- age artifacts and achieves the best overall detector performance on our benchmark datasets.

Meta-Learning for Color-to-Infrared Cross-Modal Style Transfer

MoDLF

This paper presents the results of applying several transfer learning front-ends (e.g. Resnet50, Inception, MobileNet) commonly utilized in academia based upon the ImageNet database to perform feature extraction for the DSIAC ATR data set followed by classification layers. This paper describes the performance of a machine learning system (MLS) composed of a feature generating front-end followed by a classification backend trained on electro-optical (EO) and mid-wave infrared (MWIR) imagery from the DSIAC dataset. The baseline MLS architecture achieves over 99 percent accuracy on both the EO and MWIR DSIAC datasets.

Explorations in transfer learning and machine learning architectures utilizing the DSIAC ATR algorithm development data set

A two-phase deep learning prototype for detection and classification of armoured fighting vehicles was developed. The two phases consist of an instance segmentation model and an image classification model. The instance segmentation model detects and outlines armoured fighting vehicles. These images are then piped into an image classification model which classifies the vehicles by model. The instance segmentation model reached a mean average precision of 0.399 at intersection over union of range 0.5 to 0.95. The image classification model reached 84.5% top 1 accuracy. Overall, adding synthetic images to the training data for instance segmentation yields a model with about 10% higher overall average precision. 

Instance Segmentation and Classification of Armoured Fighting Vehicles

A two-phase deep learning prototype for detection and classification of armoured fighting vehicles was developed. The two phases consist of an instance segmentation model and an image classification model. The instance segmentation model detects and outlines armoured fighting vehicles. These images are then piped into an image classification model which classifies the vehicles by model. The instance segmentation model reached a mean average precision of 0.399 at intersection over union of range 0.5 to 0.95. The image classification model reached 84.5% top 1 accuracy. Overall, adding synthetic images to the training data for instance segmentation yields a model with about 10% higher overall average precision.

Automatic Target Detection (ATD) and Recognition (ATR) are critical for video analysis and image understanding for many military and commercial applications deployed on satellites and UAV platforms. Infrared (IR) sensors can be used to detect targets during day and night time but there are few effective ATR algorithms that can exploit these sensors. Several years ago, Defense Systems Information Analysis Center (DSIAC) released an ATR Algorithm Development Image Database containing a large collection of mid-wave infrared (MWIR) imagery with multiple military and civilian vehicles as labelled targets. With the DSIAC database, we have developed AI models, which combine layers of the open-source You Only Look Once (YOLOv2) detection model with customized Convolutional Neural Network (CNN) feature extraction layers. The CNN layers are trained to extract the key features of the vehicle targets from the IR images. YOLOv2 provides target detection, classification, and localization. We have trained and tested the CNN YOLOv2 models with four different military and civilian vehicles (T72, ZSU -23-4, SUV, pickup truck) at distances ranging from 2,000m to 5,000m. Our ATR results with IR datasets show high mean average precision (mAP) between 97.25%-99.5% for day and night time images at distances of 2,000m to 5,000m. That is, we can both reliably detect and recognize different targets with only a few missed detections, and without a falsely recognized target (e.g., mistakenly classifying a civilian vehicle as a military vehicle) from as far as 5,000m. This work represents significant progress in being able to perform ATR at all times (day and night).

Mo Gharbieh

Papers

Advanced Automatic Target Recognition (ATR) with Infrared (IR) Sensors