Liyuan Li

Bio: Liyuan Li is an academic researcher from Chinese Academy of Sciences. The author has contributed to research in topics: Deep learning & Geostationary orbit. The author has an hindex of 2, co-authored 6 publications receiving 12 citations.

Lightweight U-Net for cloud detection of visible and thermal infrared remote sensing images

Abstract: Accurate and rapid cloud detection is exceedingly significant for improving the downlink efficiency of on-orbit data, especially for the microsatellites with limited power and computational ability. However, the inference speed and large model limit the potential of on-orbit implementation of deep-learning-based cloud detection method. In view of the above problems, this paper proposes a lightweight network based on depthwise separable convolutions to reduce the size of model and computational cost of pixel-wise cloud detection methods. The network achieves lightweight end-to-end cloud detection through extracting feature maps from the images to generate the mask with the obtained maps. For the visible and thermal infrared bands of the Landsat 8 cloud cover assessment validation dataset, the experimental results show that the pixel accuracy of the proposed method for cloud detection is higher than 90%, the inference speed is about 5 times faster than that of U-Net, and the model parameters and floating-point operations are reduced to 12.4% and 12.8% of U-Net, respectively.

A review on deep learning techniques for cloud detection methodologies and challenges

Abstract: Cloud detection (CD) with deep learning (DL) algorithms has been greatly developed in the applications involving the predictions of extreme weather and climate. In this review, the different conventional CD methods based on threshold, time differentiation, machine learning, and the intelligent algorithms including convolution neural networks (CNN), simple linear iterative clustering (SLIC), and semantic segmentation algorithms (SSAs) are introduced in detail, and, especially, the majority of CD publications employing the advanced and prevalent DL algorithms during the last decade are summarized and analyzed. First, in terms of the detection for different types of clouds, we meticulously compare the labels, scenarios and volumes of three popular CD datasets and put forward further the constructive recommendations about the cloud images selection, multi-bands images preprocessing, and truth labels combination for creating similar datasets. Subsequently, the structures, detection accuracies, and operating speeds of several different CD network models comprising the fully convolutional neural networks (FCNs), U-Net, SegNet, pyramid scene parsing network (PSPNet), as well as the associated derivatives are conducted elaborately to explore the comprehensively optimal performance for CD. In addition, aiming at expanding the applications in the resource-limited space-borne environment, we conclude the mainstream compression strategies of a number of different lightweight networks. Finally, the various limitations constraining the performance of the existing state-of-the-art DL CD methods and the corresponding development tendency are presented, which, expectantly, could be referential for the following researches.

Attention Mechanism Cloud Detection With Modified FCN for Infrared Remote Sensing Images

Abstract: Semantic segmentation (SS) has been widely applied for cloud detection (CD) in remote sensing images (RSIs) with high spatial and spectral resolution because of its effective pixel-level feature extraction structure. However, the typical model of lightweight SS, namely the fully convolutional network (FCN) with only seven layers, has difficulty in extracting high-level features, and the heavy pyramid scene parsing network (PSPNet) with complicated calculations is not practical in real-time CD, let alone on-orbit CD. So, in view of the problems above, we propose a compact attention mechanism cloud detection network (AM-CDN) based on the modified FCN to refine and fuse the multi-scale features for on-orbit CD. Specifically, taking the FCN as the baseline, our model increases the numbers of hidden layers and adds the residual connections between the input and output to eliminate the network degradation and extract the advanced context feature maps effectively. To expand the receptive field without losing the spatial information, the ordinary convolutions in FCN are replaced by the dilated convolution in AM-CDN. And inspired by the selective kernels of human vision, we introduce the convolutional attention mechanism (AM) into the encoder to adaptively adjust the receptive field to highlight the key texture features. According to experimental results using Landsat-8 infrared RSIs, the accuracy of the proposed CD method is 95.31%, which is 10.17% higher than that of FCN. And the calculation complexity of AM-CDN is only 7.63% of that of PSPNet.

Dim and Small Target Detection in Multi-Frame Sequence Using Bi-Conv-LSTM and 3D-Conv Structure

Abstract: Infrared dim and small target detection is widely used in military and civil fields. Traditional methods in that application rely on the local contrast between the target and background for single-frame detection. On the other hand, those algorithms depend on the motion model with fixed parameters for multi-frame association. For the great similarity of gray value and the dynamic changes of motion model parameters in the condition of low SNR and strong clutter, those methods possess weak robustness, low detection probability, and high false alarm rate. In this paper, an infrared video sequences encoding and decoding model based on Bidirectional Convolutional Long Short-Term Memory structure (Bi-Conv-LSTM) and 3D Convolutional structure (3D-Conv) is proposed, addressing the problem of high similarity and dynamic changes of parameters. For solving the problem of dynamic change in parameters, Bi-Conv-LSTM structure is used to learn the motion model of targets. And for the problem of low local contrast, 3D-Conv structure is adopted to extend receptive field in the time dimension. In order to improve the precision of detection, the Decoding part is divided into two different full connections with distinctive active function. Simulation results show that the trajectory detection accuracy of the proposed model is more than 90% under the condition of low SNR and maneuvering motion, which is better than traditional method of 80% in DB-TBD 20% in others. Real data experiment to illustrate that that our proposed method can detect small infrared targets of a low false alarm rate and high detection probability.

A 3D Mask Presentation Attack Detection Method Based on Polarization Medium Wave Infrared Imaging

Abstract: Facial recognition systems are often spoofed by presentation attack instruments (PAI), especially by the use of three-dimensional (3D) face masks. However, nonuniform illumination conditions and significant differences in facial appearance will lead to the performance degradation of existing presentation attack detection (PAD) methods. Based on conventional thermal infrared imaging, a PAD method based on the medium wave infrared (MWIR) polarization characteristics of the surface material is proposed in this paper for countering a flexible 3D silicone mask presentation attack. A polarization MWIR imaging system for face spoofing detection is designed and built, taking advantage of the fact that polarization-based MWIR imaging is not restricted by external light sources (including visible light and near-infrared light sources) in spite of facial appearance. A sample database of real face images and 3D face mask images is constructed, and the gradient amplitude feature extraction method, based on MWIR polarization facial images, is designed to better distinguish the skin of a real face from the material used to make a 3D mask. Experimental results show that, compared with conventional thermal infrared imaging, polarization-based MWIR imaging is more suitable for the PAD method of 3D silicone masks and shows a certain robustness in the change of facial temperature.

Automatic cloud classification of whole sky images

Abstract: Abstract. The recently increasing development of whole sky imagers enables temporal and spatial high-resolution sky observations. One application already performed in most cases is the estimation of fractional sky cover. A distinction between different cloud types, however, is still in progress. Here, an automatic cloud classification algorithm is presented, based on a set of mainly statistical features describing the color as well as the texture of an image. The k-nearest-neighbour classifier is used due to its high performance in solving complex issues, simplicity of implementation and low computational complexity. Seven different sky conditions are distinguished: high thin clouds (cirrus and cirrostratus), high patched cumuliform clouds (cirrocumulus and altocumulus), stratocumulus clouds, low cumuliform clouds, thick clouds (cumulonimbus and nimbostratus), stratiform clouds and clear sky. Based on the Leave-One-Out Cross-Validation the algorithm achieves an accuracy of about 97%. In addition, a test run of random images is presented, still outperforming previous algorithms by yielding a success rate of about 75%, or up to 88% if only "serious" errors with respect to radiation impact are considered. Reasons for the decrement in accuracy are discussed, and ideas to further improve the classification results, especially in problematic cases, are investigated.

A Complete YOLO-Based Ship Detection Method for Thermal Infrared Remote Sensing Images under Complex Backgrounds

Abstract: The automatic ship detection method for thermal infrared remote sensing images (TIRSIs) is of great significance due to its broad applicability in maritime security, port management, and target searching, especially at night. Most ship detection algorithms utilize manual features to detect visible image blocks which are accurately cut, and they are limited by illumination, clouds, and atmospheric strong waves in practical applications. In this paper, a complete YOLO-based ship detection method (CYSDM) for TIRSIs under complex backgrounds is proposed. In addition, thermal infrared ship datasets were made using the SDGSAT-1 thermal imaging system. First, in order to avoid the loss of texture characteristics during large-scale deep convolution, the TIRSIs with the resolution of 30 m were up-sampled to 10 m via bicubic interpolation method. Then, complete ships with similar characteristics were selected and marked in the middle of the river, the bay, and the sea. To enrich the datasets, the gray value stretching module was also added. Finally, the improved YOLOv5 s model was used to detect the ship candidate area quickly. To reduce intra-class variation, the 4.23–7.53 aspect ratios of ships were manually selected during labeling, and 8–10.5 μm ship datasets were constructed. Test results show that the precision of the CYSDM is 98.68%, which is 9.07% higher than that of the YOLOv5s algorithm. CYSDM provides an effective reference for large-scale, all-day ship detection.

Investigating One-, Two-, and Triple-Wave Solutions via Multiple Exp-Function Method Arising in Engineering Sciences

Abstract: The multiple Exp-function method is employed for searching the multiple soliton solutions for the new extended ( )- dimensional Jimbo-Miwa-like (JM) equation, the extended ( )- dimensional Calogero-Bogoyavlenskii-Schiff (eCBS) equation, the generalization of the ( )- dimensional Bogoyavlensky-Konopelchenko (BK) equation, and a variable-coefficient extension of the DJKM (vDJKM) equation, which contain one-soliton-, two-soliton-, and triple-soliton-kind solutions. The physical phenomena of these gained multiple soliton solutions are analyzed and indicated in figures by selecting suitable values.

Benchmarking Deep Learning Models for Cloud Detection in Landsat-8 and Sentinel-2 Images

Abstract: The systematic monitoring of the Earth using optical satellites is limited by the presence of clouds. Accurately detecting these clouds is necessary to exploit satellite image archives in remote sensing applications. Despite many developments, cloud detection remains an unsolved problem with room for improvement, especially over bright surfaces and thin clouds. Recently, advances in cloud masking using deep learning have shown significant boosts in cloud detection accuracy. However, these works are validated in heterogeneous manners, and the comparison with operational threshold-based schemes is not consistent among many of them. In this work, we systematically compare deep learning models trained on Landsat-8 images on different Landsat-8 and Sentinel-2 publicly available datasets. Overall, we show that deep learning models exhibit a high detection accuracy when trained and tested on independent images from the same Landsat-8 dataset (intra-dataset validation), outperforming operational algorithms. However, the performance of deep learning models is similar to operational threshold-based ones when they are tested on different datasets of Landsat-8 images (inter-dataset validation) or datasets from a different sensor with similar radiometric characteristics such as Sentinel-2 (cross-sensor validation). The results suggest that (i) the development of cloud detection methods for new satellites can be based on deep learning models trained on data from similar sensors and (ii) there is a strong dependence of deep learning models on the dataset used for training and testing, which highlights the necessity of standardized datasets and procedures for benchmarking cloud detection models in the future.

A review on deep learning techniques for cloud detection methodologies and challenges

Abstract: Cloud detection (CD) with deep learning (DL) algorithms has been greatly developed in the applications involving the predictions of extreme weather and climate. In this review, the different conventional CD methods based on threshold, time differentiation, machine learning, and the intelligent algorithms including convolution neural networks (CNN), simple linear iterative clustering (SLIC), and semantic segmentation algorithms (SSAs) are introduced in detail, and, especially, the majority of CD publications employing the advanced and prevalent DL algorithms during the last decade are summarized and analyzed. First, in terms of the detection for different types of clouds, we meticulously compare the labels, scenarios and volumes of three popular CD datasets and put forward further the constructive recommendations about the cloud images selection, multi-bands images preprocessing, and truth labels combination for creating similar datasets. Subsequently, the structures, detection accuracies, and operating speeds of several different CD network models comprising the fully convolutional neural networks (FCNs), U-Net, SegNet, pyramid scene parsing network (PSPNet), as well as the associated derivatives are conducted elaborately to explore the comprehensively optimal performance for CD. In addition, aiming at expanding the applications in the resource-limited space-borne environment, we conclude the mainstream compression strategies of a number of different lightweight networks. Finally, the various limitations constraining the performance of the existing state-of-the-art DL CD methods and the corresponding development tendency are presented, which, expectantly, could be referential for the following researches.