Statistical Analysis with Missing Data

Object detection, one of the most fundamental and challenging problems in computer vision, seeks to locate object instances from a large number of predefined categories in natural images. Deep learning techniques have emerged as a powerful strategy for learning feature representations directly from data and have led to remarkable breakthroughs in the field of generic object detection. Given this period of rapid evolution, the goal of this paper is to provide a comprehensive survey of the recent achievements in this field brought about by deep learning techniques. More than 300 research contributions are included in this survey, covering many aspects of generic object detection: detection frameworks, object feature representation, object proposal generation, context modeling, training strategies, and evaluation metrics. We finish the survey by identifying promising directions for future research.

/pdf/deep-learning-for-generic-object-detection-a-survey-1j2k8px4h1.pdf

Deep Learning for Generic Object Detection: A Survey

Deep learning and its applications to machine health monitoring

Representing features at multiple scales is of great importance for numerous vision tasks. Recent advances in backbone convolutional neural networks (CNNs) continually demonstrate stronger multi-scale representation ability, leading to consistent performance gains on a wide range of applications. However, most existing methods represent the multi-scale features in a layer-wise manner. In this paper, we propose a novel building block for CNNs, namely Res2Net, by constructing hierarchical residual-like connections within one single residual block. The Res2Net represents multi-scale features at a granular level and increases the range of receptive fields for each network layer. The proposed Res2Net block can be plugged into the state-of-the-art backbone CNN models, e.g., ResNet, ResNeXt, and DLA. We evaluate the Res2Net block on all these models and demonstrate consistent performance gains over baseline models on widely-used datasets, e.g., CIFAR-100 and ImageNet. Further ablation studies and experimental results on representative computer vision tasks, i.e., object detection, class activation mapping, and salient object detection, further verify the superiority of the Res2Net over the state-of-the-art baseline methods. The source code and trained models are available on https://mmcheng.net/res2net/ .

Res2Net: A New Multi-Scale Backbone Architecture

Deep Learning has enabled remarkable progress over the last years on a variety of tasks, such as image recognition, speech recognition, and machine translation. One crucial aspect for this progress are novel neural architectures. Currently employed architectures have mostly been developed manually by human experts, which is a time-consuming and error-prone process. Because of this, there is growing interest in automated neural architecture search methods. We provide an overview of existing work in this field of research and categorize them according to three dimensions: search space, search strategy, and performance estimation strategy.

Neural Architecture Search: A Survey

Deep neural networks (DNNs) have recently achieved great success in many visual recognition tasks. However, existing deep neural network models are computationally expensive and memory intensive, hindering their deployment in devices with low memory resources or in applications with strict latency requirements. Therefore, a natural thought is to perform model compression and acceleration in deep networks without significantly decreasing the model performance. During the past five years, tremendous progress has been made in this area. In this paper, we review the recent techniques for compacting and accelerating DNN models. In general, these techniques are divided into four categories: parameter pruning and quantization, low-rank factorization, transferred/compact convolutional filters, and knowledge distillation. Methods of parameter pruning and quantization are described first, after that the other techniques are introduced. For each category, we also provide insightful analysis about the performance, related applications, advantages, and drawbacks. Then we go through some very recent successful methods, for example, dynamic capacity networks and stochastic depths networks. After that, we survey the evaluation matrices, the main datasets used for evaluating the model performance, and recent benchmark efforts. Finally, we conclude this paper, discuss remaining the challenges and possible directions for future work.

A Survey of Model Compression and Acceleration for Deep Neural Networks

In recent years, machine learning techniques have been widely used to solve many problems for fault diagnosis. However, in many real-world fault diagnosis applications, the distribution of the source domain data (on which the model is trained) is different from the distribution of the target domain data (where the learned model is actually deployed), which leads to performance degradation. In this paper, we introduce domain adaptation, which can find the solution to this problem by adapting the classifier or the regression model trained in a source domain for use in a different but related target domain. In particular, we proposed a novel deep neural network model with domain adaptation for fault diagnosis. Two main contributions are concluded by comparing to the previous works: first, the proposed model can utilize domain adaptation meanwhile strengthening the representative information of the original data, so that a high classification accuracy in the target domain can be achieved, and second, we proposed several strategies to explore the optimal hyperparameters of the model. Experimental results, on several real-world datasets, demonstrate the effectiveness and the reliability of both the proposed model and the exploring strategies for the parameters.

Deep Model Based Domain Adaptation for Fault Diagnosis

In recent years, deep neural networks (DNNs) have received increased attention, have been applied to different applications, and achieved dramatic accuracy improvements in many tasks. These works rely on deep networks with millions or even billions of parameters, and the availability of graphics processing units (GPUs) with very high computation capability plays a key role in their success. For example, Krizhevsky et al. [1] achieved breakthrough results in the 2012 ImageNet Challenge using a network containing 60 million parameters with five convolutional layers and three fully connected layers. Usually, it takes two to three days to train the whole model on the ImagetNet data set with an NVIDIA K40 machine. In another example, the top face-verification results from the Labeled Faces in the Wild (LFW) data set were obtained with networks containing hundreds of millions of parameters, using a mix of convolutional, locally connected, and fully connected layers [2], [3]. It is also very time-consuming to train such a model to obtain a reasonable performance. In architectures that only rely on fully connected layers, the number of parameters can grow to billions [4].

Model Compression and Acceleration for Deep Neural Networks: The Principles, Progress, and Challenges

This paper is concerned of the loop closure detection problem for visual simultaneous localization and mapping systems. We propose a novel approach based on the stacked denoising auto-encoder (SDA), a multi-layer neural network that autonomously learns an compressed representation from the raw input data in an unsupervised way. Different with the traditional bag-of-words based methods, the deep network has the ability to learn the complex inner structures in image data, while no longer needs to manually design the visual features. Our approach employs the characteristics of the SDA to solve the loop detection problem. The workflow of training the network, utilizing the features and computing the similarity score is presented. The performance of SDA is evaluated by a comparison study with Fab-map 2.0 using data from open datasets and physical robots. The results show that SDA is feasible for detecting loops at a satisfactory precision and can therefore provide an alternative way for visual SLAM systems.

Unsupervised learning to detect loops using deep neural networks for visual SLAM system

In this paper, we design transmit beamforming for a full duplex base station (FD-BS) considering both self-interference mitigation and physical-layer security. The proposed design is formulated as minimizing the power consumption of FD-BS under different signal-to-interference-and-noise-ratio (SINR) constraints. Semidefinite relaxation (SDR) is used to convert the initial nonconvex optimization to be a convex semidefinite programming (SDP) problem. Then the optimality of SDR is strictly proved by showing the existence of the rank-one optimal solutions. To reduce the computational complexity, we develop zero forcing beamforming-based suboptimal algorithms, where the solutions can be obtained using golden search and closed-form solutions can be derived in each step. Simulation results are then provided to verify the efficiency of the proposed algorithms.

Tao Zhang

Papers

A Survey of Model Compression and Acceleration for Deep Neural Networks

Deep Model Based Domain Adaptation for Fault Diagnosis

Model Compression and Acceleration for Deep Neural Networks: The Principles, Progress, and Challenges

Unsupervised learning to detect loops using deep neural networks for visual SLAM system

Physical-Layer Security for Full Duplex Communications With Self-Interference Mitigation