Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Global terrorist threats and illegal migration have intensified concerns for the security of citizens, and every effort is made to exploit all available technological advances to prevent adverse events and protect people and their property. Due to the ability to use at night and in weather conditions where RGB cameras do not perform well, thermal cameras have become an important component of sophisticated video surveillance systems. In this paper, we investigate the task of automatic person detection in thermal images using convolutional neural network models originally intended for detection in RGB images. We compare the performance of the standard state-of-the-art object detectors such as Faster R-CNN, SSD, Cascade R-CNN, and YOLOv3, that were retrained on a dataset of thermal images extracted from videos that simulate illegal movements around the border and in protected areas. Videos are recorded at night in clear weather, rain, and in the fog, at different ranges, and with different movement types. YOLOv3 was significantly faster than other detectors while achieving performance comparable with the best, so it was used in further experiments. We experimented with different training dataset settings in order to determine the minimum number of images needed to achieve good detection results on test datasets. We achieved excellent detection results with respect to average accuracy for all test scenarios although a modest set of thermal images was used for training. We test our trained model on different well known and widely used thermal imaging datasets as well. In addition, we present the results of the recognition of humans and animals in thermal images, which is particularly important in the case of sneaking around objects and illegal border crossings. Also, we present our original thermal dataset used for experimentation that contains surveillance videos recorded at different weather and shooting conditions.

/pdf/thermal-object-detection-in-difficult-weather-conditions-2xebrqlnbj.pdf

Thermal Object Detection in Difficult Weather Conditions Using YOLO

This paper reports on the use of a neural network in infrared thermography to classify defects, such as air, oil, and water, which can degrade material performance. A finite element method and experiment were adopted to simulate air, water, and oil ingress. Raw data, and thermographic signal reconstruction coefficients were used to train, and test the two multilayer, feed-forward NN models. Quantitative comparisons showed that the model using coefficients as features performed better than the one using raw data. It was more precise and had better test repeatability. This indicates the model is more generalizable.

Automated defect classification in infrared thermography based on a neural network

Acoustic signal-based fault detection of hydraulic piston pump using a particle swarm optimization enhancement CNN

When extracting land-use information from remote sensing imagery using image segmentation, obtaining fine edges for extracted objects is a key problem that is yet to be solved. In this study, we developed a new weight feature value convolutional neural network (WFCNN) to perform fine remote sensing image segmentation and extract improved land-use information from remote sensing imagery. The WFCNN includes one encoder and one classifier. The encoder obtains a set of spectral features and five levels of semantic features. It uses the linear fusion method to hierarchically fuse the semantic features, employs an adjustment layer to optimize every level of fused features to ensure the stability of the pixel features, and combines the fused semantic and spectral features to form a feature graph. The classifier then uses a Softmax model to perform pixel-by-pixel classification. The WFCNN was trained using a stochastic gradient descent algorithm; the former and two variants were subject to experimental testing based on Gaofen 6 images and aerial images that compared them with the commonly used SegNet, U-NET, and RefineNet models. The accuracy, precision, recall, and F1-Score of the WFCNN were higher than those of the other models, indicating certain advantages in pixel-by-pixel segmentation. The results clearly show that the WFCNN can improve the accuracy and automation level of large-scale land-use mapping and the extraction of other information using remote sensing imagery.

Improved Remote Sensing Image Classification Based on Multi-Scale Feature Fusion

Near infrared nighttime road pedestrians recognition based on convolutional neural network

Multi-task faster R-CNN for nighttime pedestrian detection and distance estimation

In this paper, a new Hough transform is proposed to detect lines with geometric meanings. In the standard Hough transform, lines are parameterized by the length and orientation of the normal vector from the origin to the line. The parameter space of the standard Hough transform is presented in a polar coordinate system. Geometric measurements such as distance and angle are not suitable in polar coordinate space due to the inconsistency of the axes. To deal with this problem, a Hough transform which is carried out in a bounded Cartesian coordinate parameter space is developed. The lines are represented by their perpendicular feet with respect to a fixed point. Since the parameter space is in Cartesian coordinate space, it is intuitive to the researchers. When detecting the peaks in the parameter space, distance and angle constraints can be applied. The effectiveness of the proposed method has been illustrated by two applications, that is, lane line detection and convex polygon extraction. The results show that the proposed Cartesian coordinate Hough transform is suitable for detecting meaningful lines. 

https://onlinelibrary.wiley.com/doi/pdfdirect/10.1049/ipr2.12489

A new hough transform operated in a bounded cartesian coordinate parameter space

In nighttime driving scenes, due to insufficient and uneven lighting, and the scarcity of high-quality datasets, the miss rate of nighttime pedestrian detection (PD) is much higher than that of daytime. Vision-based distance detection (DD) has the advantages of low cost and good interpretability, but the existing methods have low precision, poor robustness, and the DD is mostly performed independently of PD. A narrowband near-infrared (NIR) camera and an NIR lamp to relieve the impact of noisy visible light (VIS) and improve imaging quality have been selected; a LiDAR to obtain distance information has been embedded in the system; a nighttime driving scene pedestrian and its distance joint detection dataset NIRPed has been built. The NIRPed includes 146k pedestrian annotations, which is 3 times that of NightOwls, the largest VIS nighttime pedestrian dataset. Based on Faster-RCNN, a joint PD and DD method using monocular imaging has been proposed. The propo- sed method realizes the joint detection of pedestrian and its distance on the NIRPed dataset with the PD log-average miss rate and DD mean absolute error rate of 6.5% and 5.5%, respectively. For comparison, the joint detection method was implemented on other large-scale VIS pedestrian datasets as well. Moreover, compared with the existing vision-based DD methods, the proposed method is less affected by pedestrian distance and height, exhibits higher accuracy and robustness, and satisfied the ISO requirements for intelligent transportation systems (absolute error rate < 15%).

Junping Hu

Papers

Near infrared nighttime road pedestrians recognition based on convolutional neural network

Multi-task faster R-CNN for nighttime pedestrian detection and distance estimation

A new hough transform operated in a bounded cartesian coordinate parameter space

NIRPed: A Novel Benchmark for Nighttime Pedestrian and Its Distance Joint Detection