Feature selection, as a data preprocessing strategy, has been proven to be effective and efficient in preparing data (especially high-dimensional data) for various data-mining and machine-learning problems. The objectives of feature selection include building simpler and more comprehensible models, improving data-mining performance, and preparing clean, understandable data. The recent proliferation of big data has presented some substantial challenges and opportunities to feature selection. In this survey, we provide a comprehensive and structured overview of recent advances in feature selection research. Motivated by current challenges and opportunities in the era of big data, we revisit feature selection research from a data perspective and review representative feature selection algorithms for conventional data, structured data, heterogeneous data and streaming data. Methodologically, to emphasize the differences and similarities of most existing feature selection algorithms for conventional data, we categorize them into four main groups: similarity-based, information-theoretical-based, sparse-learning-based, and statistical-based methods. To facilitate and promote the research in this community, we also present an open source feature selection repository that consists of most of the popular feature selection algorithms (http://featureselection.asu.edu/). Also, we use it as an example to show how to evaluate feature selection algorithms. At the end of the survey, we present a discussion about some open problems and challenges that require more attention in future research.

https://dl.acm.org/doi/pdf/10.1145/3136625

Feature Selection: A Data Perspective

With the transition of facial expression recognition (FER) from laboratory-controlled to challenging in-the-wild conditions and the recent success of deep learning techniques in various fields, deep neural networks have increasingly been leveraged to learn discriminative representations for automatic FER. Recent deep FER systems generally focus on two important issues: overfitting caused by a lack of sufficient training data and expression-unrelated variations, such as illumination, head pose and identity bias. In this paper, we provide a comprehensive survey on deep FER, including datasets and algorithms that provide insights into these intrinsic problems. First, we describe the standard pipeline of a deep FER system with the related background knowledge and suggestions of applicable implementations for each stage. We then introduce the available datasets that are widely used in the literature and provide accepted data selection and evaluation principles for these datasets. For the state of the art in deep FER, we review existing novel deep neural networks and related training strategies that are designed for FER based on both static images and dynamic image sequences, and discuss their advantages and limitations. Competitive performances on widely used benchmarks are also summarized in this section. We then extend our survey to additional related issues and application scenarios. Finally, we review the remaining challenges and corresponding opportunities in this field as well as future directions for the design of robust deep FER systems.

Deep Facial Expression Recognition: A Survey

3D skeleton-based human action classification

Communication without words

Facial expressions are an important way through which humans interact socially. Building a system capable of automatically recognizing facial expressions from images and video has been an intense field of study in recent years. Interpreting such expressions remains challenging and much research is needed about the way they relate to human affect. This paper presents a general overview of automatic RGB, 3D, thermal and multimodal facial expression analysis. We define a new taxonomy for the field, encompassing all steps from face detection to facial expression recognition, and describe and classify the state of the art methods accordingly. We also present the important datasets and the bench-marking of most influential methods. We conclude with a general discussion about trends, important questions and future lines of research.

Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications

In this paper, we propose a novel approach for human action recognition based on motion capture (MOCAP) information using a Fuzzy convolutional neural network. The MOCAP tracking information of human joints is used to compute the temporal variation of displacement between joints during the execution of an action. Fuzzy membership functions designed to emphasize the discriminative pose associated with each action are considered for feature extraction. The temporal variation of membership values associated with these fuzzy membership functions is considered as the feature representation for action recognition. A convolutional neural network (CNN) capable of recognizing local patterns in input data is trained to recognize human actions from the local patterns in the feature representation. Experimental evaluation on Berkeley MHAD dataset demonstrates the effectiveness of the proposed approach.

Human action recognition based on motion capture information using fuzzy convolution neural networks

Existing image captioning approaches fail to generate fine-grained captions due to the lack of rich encoding representation of an image. In this paper, we present an attentive contextual network (ACN) to learn the spatially transformed image features and dense multi-scale contextual information of an image to generate semantically meaningful captions. At first, we construct deformable network on intermediate layers of convolutional neural network (CNN) to cultivate spatial invariant features. And the multi-scale contextual features are produced by employing contextual network on top of last layers of CNN. Then, we exploit attention mechanism on contextual network to extract dense contextual features. Further, the extracted spatial and contextual features are combined to encode the holistic representation of an image. Finally, a multi-stage caption decoder with visual attention module is incorporated to generate fine-grained captions. The performance of the proposed approach is demonstrated on COCO dataset, the largest dataset for image captioning.

Attentive Contextual Network for Image Captioning

The real-time detection of traffic rule violators in a city-wide surveillance network is a highly desirable but challenging task because it needs to perform computationally complex analytics on the live video streams from large number of cameras, simultaneously. In this paper, we propose an efficient framework using edge computing to deploy a system for automatic detection of bike-riders without helmet. First, we propose a novel robust and compact method for the detection of the motorcyclists without helmet using convolutional neural networks (CNNs). Then, we scale it for the real-time performance on an edge-device by dropping redundant filters and quantizing the model weights. To reduce the network latency, we place the detector module on edge-devices in the cameras. The edge-nodes send their detected alerts to a central alert database where the end users access these alerts through a web interface. To evaluate the proposed method, we collected two datasets of real traffic videos, namely, IITH_Helmet_1 which contains sparse traffic and IITH_Helmet_2 which contains dense traffic. The experimental results show that our method achieves a high detection accuracy of$\approx$95% while maintaining the real-time processing speed of $\approx$22fps on Nvidia-TXI.

Real-Time Detection of Motorcyclist without Helmet using Cascade of CNNs on Edge-device

Aircraft type recognition remains challenging, due to their tiny sizes and geometric distortions in large-scale panchromatic satellite images. This paper proposes a framework for aircraft type recognition by focusing on shape preservation, spatial transformations, and geospatial attributes derivation. First, we construct an aircraft segmentation model to obtain masks representing the shape of aircrafts by employing a learnable shape-preserved and deformable network in the mask RCNN architecture. Then, the orientation of the segmented aircrafts is determined by estimating the symmetrical axes using their gradient information. Besides template matching, we derive the length and width of aircrafts using the geotagged information of images to further categorize the types of aircrafts. Also, we present an effective inferencing mechanism to overcome the issue of partial detection or missing aircrafts in large-scale images. The efficacy of the proposed framework is demonstrated on large-scale panchromatic images with ground sampling distances of 0.65m (C2S). 

A framework to derive geospatial attributes for aircraft type recognition in large-scale remote sensing images

Autonomous vision-based taxi, takeoff, and landing systems are gaining popularity in fixed-wing commercial, military, and unmanned aerial vehicles presenting new technical and regulatory challenges. With ongoing advances in deep learning, machine learning, and big data management, the dangers of takeoff and landing operations have greatly decreased, and with less dependence on simultaneous and complex controllers. However, in a real-world scenario, airport runway detection for takeoff and landing operations is still deemed challenging, due to the vast amount of contextual information, complex backgrounds, and dangerous geographical scenarios. In this work, we propose an end-to-end airport runway detection network for automatic runway detection and localization centered on a two-stage region-based convolutional neural network that could significantly improve navigation efficiency and dependence on complex control systems. The first stage is an enhanced region proposal network with domain-specific constraints added, which automatically learns and extracts multiscale (high, mid, and low-level) features using a deep convolutional neural network and filters low-quality positive samples for robust runway detection. The second stage classifies the candidate regions and determines the precise localization of predictions using an enhanced anchor refinement and regression algorithm with a nonmaximum suppression function for bounding box optimization. Transfer learning and data augmentation techniques are used to propagate relevant domain-specific semantic and context information during the training phase to handle the problem of imbalances in the dataset classes. The results show the effectiveness of our algorithmic contributions as well as the benefits of the specific enhancements, in terms of, improved efficiency and accuracy of runway detection in remote sensing images.

C. Krishna Mohan

Papers

Human action recognition based on motion capture information using fuzzy convolution neural networks

Attentive Contextual Network for Image Captioning

Real-Time Detection of Motorcyclist without Helmet using Cascade of CNNs on Edge-device

A framework to derive geospatial attributes for aircraft type recognition in large-scale remote sensing images

A Robust Airport Runway Detection Network Based on R-CNN Using Remote Sensing Images