Deep learning has developed as an effective machine learning method that takes in numerous layers of features or representation of the data and provides state-of-the-art results. The application of deep learning has shown impressive performance in various application areas, particularly in image classification, segmentation and object detection. Recent advances of deep learning techniques bring encouraging performance to fine-grained image classification which aims to distinguish subordinate-level categories. This task is extremely challenging due to high intra-class and low inter-class variance. In this paper, we provide a detailed review of various deep architectures and model highlighting characteristics of particular model. Firstly, we described the functioning of CNN architectures and its components followed by detailed description of various CNN models starting with classical LeNet model to AlexNet, ZFNet, GoogleNet, VGGNet, ResNet, ResNeXt, SENet, DenseNet, Xception, PNAS/ENAS. We mainly focus on the application of deep learning architectures to three major applications, namely (i) wild animal detection, (ii) small arm detection and (iii) human being detection. A detailed review summary including the systems, database, application and accuracy claimed is also provided for each model to serve as guidelines for future work in the above application areas.

Convolutional neural network: a review of models, methodologies and applications to object detection

Facial attribute analysis has received considerable attention when deep learning techniques made remarkable breakthroughs in this field over the past few years. Deep learning based facial attribute analysis consists of two basic sub-issues: facial attribute estimation (FAE), which recognizes whether facial attributes are present in given images, and facial attribute manipulation (FAM), which synthesizes or removes desired facial attributes. In this paper, we provide a comprehensive survey of deep facial attribute analysis from the perspectives of both estimation and manipulation. First, we summarize a general pipeline that deep facial attribute analysis follows, which comprises two stages: data preprocessing and model construction. Additionally, we introduce the underlying theories of this two-stage pipeline for both FAE and FAM. Second, the datasets and performance metrics commonly used in facial attribute analysis are presented. Third, we create a taxonomy of state-of-the-art methods and review deep FAE and FAM algorithms in detail. Furthermore, several additional facial attribute related issues are introduced, as well as relevant real-world applications. Finally, we discuss possible challenges and promising future research directions.

A Survey of Deep Facial Attribute Analysis

The material attributes of foreign object debris (FOD) are the most crucial factors to understand the level of damage sustained by an aircraft. However, the prevalent FOD detection systems lack an effective method for automatic material recognition. This paper proposes a novel FOD material recognition approach based on both transfer learning and a mainstream deep convolutional neural network (D-CNN) model. To this end, we create an FOD image dataset consisting of images from the runways of Shanghai Hongqiao International Airport and the campus of our research institute. We optimize the architecture of the D-CNN by considering the characteristics of the material distribution of the FOD. The results show that the proposed approach can improve the accuracy of material recognition by 39.6% over the state-of-the-art method. The work here will help enhance the intelligence capability of future FOD detection systems and encourage other practical applications of material recognition technology.

/pdf/foreign-object-debris-material-recognition-based-on-4m1hcej0wf.pdf

Foreign object debris material recognition based on convolutional neural networks

In turbomachinery, their blade leading edges are critical to performance and therefore fuel efficiency, emission, noise, running and maintenance costs. Leading edge damage and therefore roughness is either caused by subtractive processes such as foreign object damage (bird strikes and debris ingestion) and erosion (hail, rain droplets, sand particles, dust, volcanic ash and cavitation) and additive processes such as filming (from dirt, icing, fouling, insect build-up). Therefore, this review focuses on the changes in topography induced by during service to blade leading edges and the effect of roughness and form on performance and efforts to predict and model these changes. The applications considered are focused on wind, gas and tidal turbines and turbofan engines. Repair and protection strategies for leading edges of blades are also reviewed. The review shows additive processes are typically worse than subtractive processes, as the roughness or even form change is significant with icing and biofouling. Antagonism is reported between additive and subtractive roughness processes. There are gaps in the current understanding of the additive and subtractive processes that influence roughness and their interaction. Recent work paves the way forward where modelling and machine learning is used to predict coated wind turbine blade leading edge delamination and the effects this has on aerodynamic performance and what changes in blade angle would best capture the available wind energy with such damaged blades. To do this generically there is a need for better understanding of the environment that the blades see and the variation along their length, the material or coated material response to additive and/or subtractive mechanisms and thus the roughness/form evolution over time. This is turn would allow better understanding of the effects these changes have on aerodynamic/ hydrodynamic efficiency and the population of stress raisers and distribution of residual stresses that result. These in turn influence fatigue strength and remaining useful life of the blade leading edge as well as inform maintenance/repair needs

/pdf/leading-edge-topography-of-blades-a-critical-review-jlzgjx65u1.pdf

Leading edge topography of blades - a critical review

This paper reports about potentials of hyperspectral imaging for object detection, especially on an application of foreign object detection (FOD) in meat products. A sequential deep-learning framework is proposed by using region-proposal networks (RPNs) and 3D convolutional networks (CNNs). Two independent datasets of images, contaminated with many types of foreign materials, were used for training and testing the proposed model. Results show that the proposed RPN model outperforms a selected search method in terms of accuracy, efficiency, and run-time. An FOD model based on RPN and 3D-CNN, or selected search with a 3D-CNN solve FOD with an average precision of 81.0% or 50.6%, respectively. This study demonstrates opportunities when using hyperspectral imaging systems for real-time object detection by using both spectral and spatial features combined.

A Sequential CNN Approach for Foreign Object Detection in Hyperspectral Images

Metric learning is a significant factor for media retrieval. In this paper, we propose an attribute label enhanced metric learning model to assist face image retrieval. Different from general cross-media retrieval, in the proposed model, the information of attribute labels are embedded in a hypergraph metric learning framework for face image retrieval tasks. The attribute labels serve to build a hypergraph, in which each image is abstracted as a vertex and is contained in several hyperedges. The learned hypergraph combines the attribute label to reform the topology of image similarity relationship. With the mined correlation among multiple facial attributes, the reformed metrics incorporates the semantic information in the general image similarity measure. We apply the metric learning strategy to both similarity face retrieval and interactive face retrieval. The proposed metric learning model effectively narrows down the semantic gap between human and machine face perception. The learned distance metric not only increases the precision of similarity retrieval but also speeds up the convergence distinctively in interactive face retrieval.

/pdf/attribute-enhanced-metric-learning-for-face-retrieval-fgor14pcqv.pdf

Attribute-enhanced metric learning for face retrieval

In this paper, we propose a novel framework of Foreign Object Debris (FOD) classification system. The system contains a FOD detection subsystem, electro-optical subsystem and the control center. The system not only provides continuous surveillance of scanned surfaces and achieves the goal of FOD detection, but also performs FOD classification. Both low level features and subspace features are compared to extract the FOD. Multiclass classifiers are trained in all the candidate feature spaces with the Support Vector Machine (SVM) to classify FOD. Experimental results show that it is promising to classify FOD with low-level features.

A Novel FOD Classification System Based on Visual Features

In this paper, we propose a novel framework of Foreign Object Debris (FOD) classification combining scale-invariant feature transform (SIFT) feature and color feature. This system contains FOD detection subsystem, image quality assessment, control center and FOD recognition subsystem. The system not only achieves the goal of FOD detection, but also fulfills the task of FOD classification. We propose a mixed feature method that combines SIFT feature and color feature to extract FOD feature and use Support vector machine (SVM) or nearest neighbor (NN) to classify FOD image. Experiment results show that the proposed framework is effective and accurate.

Fusion of low-level feature for FOD classification

Recently, keypoint descriptors such as Scale Invariant Feature Transform (SIFT) have been proved promising in similarity retrieval of images, which adopts matching score as similarity. However, the matching score is easy to be decreased once there are little variances between image details, and hence lead to low retrieval performance. In this paper, we propose a novel retrieval approach that improves the matching score with reduced time of matching by Kernel-based Fuzzy C-Means clustering (KFCM), which proves to be a better trade-off between matching and retrieval precision. Experiments conducted on three representative image databases show that our retrieval approach is surprisingly effective, outperforming the SIFT based method, not only in object-based image retrieval but also for searching scenes with similar semantic.

Yuchun Fang

Papers

Foreign object debris material recognition based on convolutional neural networks

Attribute-enhanced metric learning for face retrieval

A Novel FOD Classification System Based on Visual Features

Fusion of low-level feature for FOD classification

A KFCM and SIFT Based Matching Approach to Similarity Retrieval of Images