Application and improvement of Canny edge-detection algorithm for exterior wall hollowing detection using infrared thermal images

In recent years, longwave infrared (LWIR) cameras have become potential in visual simultaneous localization and mapping (SLAM) research since the delivered thermal images can provide information beyond the visible spectrum and are robust to environment illumination. However, due to modality differences, SLAM methods designed for visible cameras cannot be directly applied to thermal data. In this paper, we propose a thermal-inertial SLAM method for all-day autonomous systems. To overcome the challenge of the thermal data association, the proposed method represents several improvements, including singular-value-decomposition-based (SVD-based) image processing and ThermalRAFT tracking methods. Based on the characteristics of the thermal images, the SVD-based image processing method can exploit the fixed noise pattern of thermal images and enhance the image quality to improve the performance of subsequent steps, including thermal feature extraction and loop detection. To achieve real-time and robust feature tracking, we develop ThermalRAFT, an efficient optical flow network with iterative optimization. Moreover, the system introduces a bag-of-words-based loop detection method to maintain global consistency in long-term operation. The experimental results demonstrate that the proposed method can provide competitive performance in indoor and outdoor environments and is robust under challenging illumination conditions.

Thermal-Inertial SLAM for the Environments With Challenging Illumination

It is difficult to find correct correspondences for infrared and visible image registration because of different imaging principles Traditional registration methods based on the point feature require designing the complicated feature descriptor and eliminate mismatched points, which results in unsatisfactory precision and much calculation time To tackle these problems, this paper presents an artful method based on constrained point features to align infrared and visible images The proposed method principally contains three steps First, constrained point features are extracted by employing an object detection algorithm, which avoids constructing the complex feature descriptor and introduces the senior semantic information to improve the registration accuracy Then, the left value rule (LV-rule) is designed to match constrained points strictly without the deletion of mismatched and redundant points Finally, the affine transformation matrix is calculated according to matched point pairs Moreover, this paper presents an evaluation method to automatically estimate registration accuracy The proposed method is tested on a public dataset Among all tested infrared-visible image pairs, registration results demonstrate that the proposed framework outperforms five state-of-the-art registration algorithms in terms of accuracy, speed, and robustness

An Infrared-Visible Image Registration Method Based on the Constrained Point Feature

Deep convolutional neural networks (CNNs) have achieved huge success in the image denoising fields. However, there still have three drawbacks to be overcome: firstly, it is a big cost to train a deeper CNN model for better denoising performance, secondly, most models often lack the flexibility to remove spatially variant additive Gaussian noise (AWGN), finally, it is difficult to achieve a good trade-off between denoising performance and details preservation. In this paper, we propose a novel network integrated with nonsubsampled shearlet transform (NSST) and a broad convolutional neural network, namely NSTBNet. The proposed NSTBNet has the following desirable properties: (1) a single model has the ability to deal with different noise levels and spatially variant AWGN; (2) the model combines the two networks make it wider to improve denoising performance without increasing too much computational cost; (3) the NSST and inverse NSST are employed to acquire more texture and to avoid gridding effect. Extensive simulation experiments on AWGN and realistic noise show that NSTBNet has state-of-the-art denoising performance, making it a high application value. 

NSTBNet: Toward a nonsubsampled shearlet transform for broad convolutional neural network image denoising

The quality of images captured in rainy days is severely degraded, which affects the accuracy of subsequent computer vision tasks. Recently, many deep learning-based methods have demonstrated superior performance for single image deraining. However, there are still many issues left. Since real-world rain images and their corresponding ground truths are difficult to collect, models trained on limited data may lead to overfitting. Meanwhile, although many methods can remove part of the rain streaks, most of them cannot reconstruct precise edges and textures. For the first issue, we use the transfer learning approach. Loading pre-trained parameters trained on the ImageNet enables the network to have robust feature representation, which improves the generalization of the network. For the second issue, we restore clear details by making full use of the frequency domain information of the image. Specifically, we design a novel frequency domain residual block (FRDB) and use an efficient fusion strategy in FRDB to fuse spatial and frequency domain features. Then, we propose a frequency domain reconstruction loss function (FDR loss) to restore details by reducing the differences in high-frequency space. Finally, a simple detail enhancement attention module (DEAM) is used to further enhance the image details. Extensive experimental results demonstrate that our DPNet has superior performance on both synthetic and real data. Furthermore, we verify the effectiveness of our method on downstream computer vision tasks. The source codes will be open at https://github.com/noxsine/DPNet . 

Jingzhi Zhang

Papers

Infrared image denoising based on the variance-stabilizing transform and the dual-domain filter

Registration of multimodal images with edge features and scale invariant PIIFD