“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Nowadays, deep learning is a current and a stimulating field of machine learning. Deep learning is the most effective, supervised, time and cost efficient machine learning approach. Deep learning is not a restricted learning approach, but it abides various procedures and topographies which can be applied to an immense speculum of complicated problems. The technique learns the illustrative and differential features in a very stratified way. Deep learning methods have made a significant breakthrough with appreciable performance in a wide variety of applications with useful security tools. It is considered to be the best choice for discovering complex architecture in high-dimensional data by employing back propagation algorithm. As deep learning has made significant advancements and tremendous performance in numerous applications, the widely used domains of deep learning are business, science and government which further includes adaptive testing, biological image classification, computer vision, cancer detection, natural language processing, object detection, face recognition, handwriting recognition, speech recognition, stock market analysis, smart city and many more. This paper focuses on the concepts of deep learning, its basic and advanced architectures, techniques, motivational aspects, characteristics and the limitations. The paper also presents the major differences between the deep learning, classical machine learning and conventional learning approaches and the major challenges ahead. The main intention of this paper is to explore and present chronologically, a comprehensive survey of the major applications of deep learning covering variety of areas, study of the techniques and architectures used and further the contribution of that respective application in the real world. Finally, the paper ends with the conclusion and future aspects.

A Survey of Deep Learning and Its Applications: A New Paradigm to Machine Learning

Today, a new generation of artificial intelligence has brought several new research domains such as computer vision (CV). Thus, target tracking, the base of CV, has been a hotspot research domain. Correlation filter (CF)-based algorithm has been the basis of real-time tracking algorithms because of the high tracking efficiency. However, CF-based algorithms usually failed to track objects in complex environments. Therefore, this article proposes a fuzzy detection strategy to prejudge the tracking result. If the prejudge process determines that the tracking result is not good enough in the current frame, the stored target template is used for following tracking to avoid the template pollution. During testing on the OTB100 dataset, the experimental results show that the proposed auxiliary detection strategy improves the tracking robustness under complex environment by ensuring the tracking speed.

Fuzzy Detection Aided Real-Time and Robust Visual Tracking Under Complex Environments

With the increasing population of Industry 4.0, industrial big data (IBD) has become a hotly discussed topic in digital and intelligent industry field. The security problem existing in the signal processing on large scale of data stream is still a challenge issue in industrial internet of things, especially when dealing with the high-dimensional anomaly detection for intelligent industrial application. In this article, to mitigate the inconsistency between dimensionality reduction and feature retention in imbalanced IBD, we propose a variational long short-term memory (VLSTM) learning model for intelligent anomaly detection based on reconstructed feature representation. An encoder–decoder neural network associated with a variational reparameterization scheme is designed to learn the low-dimensional feature representation from high-dimensional raw data. Three loss functions are defined and quantified to constrain the reconstructed hidden variable into a more explicit and meaningful form. A lightweight estimation network is then fed with the refined feature representation to identify anomalies in IBD. Experiments using a public IBD dataset named UNSW-NB15 demonstrate that the proposed VLSTM model can efficiently cope with imbalance and high-dimensional issues, and significantly improve the accuracy and reduce the false rate in anomaly detection for IBD according to F1, area under curve (AUC), and false alarm rate (FAR).

/pdf/variational-lstm-enhanced-anomaly-detection-for-industrial-57gptpbhzg.pdf

Variational LSTM Enhanced Anomaly Detection for Industrial Big Data

Energy efficiency (EE) and spectrum efficiency (SE) have received significant attentions on optimizing the network performance in cognitive radio networks. In this paper, an EE+SE tradeoff based target is considered for the primary users (PUs) and the secondary users (SUs). First of all, considering the orthogonal frequency division multiple access-based resource allocation (RA) for the underlying SUs, we formulate an objective function through minimizing a weighted sum of the secondary interference power, where the network performance of both PUs and SUs are guaranteed by the constraints on quality of service, power consumption and data rate. However, it is a NP-hard problem. In order to solve it, we propose a damped three dimensional (D3D) message-passing algorithm (MPA) based on deep learning. Specifically, a feed-forward neural network is devised and an analogous back propagation algorithm is developed to learn the optimal parameters of the D3D-MPA. To improve the computational efficiency of the allocation and the learning, a suboptimal RA scheme is deduced based on a damped two dimensional MPA. Finally, simulation results are provided to confirm the effectiveness of our proposed scheme.

Deep Learning-Inspired Message Passing Algorithm for Efficient Resource Allocation in Cognitive Radio Networks

Automated modulation classification plays a very important part in cognitive radio networks. Deep learning is also a powerful tool that we could not overlook its potential in addressing signal modulation recognition problem. In our last work, we propose a new data conversion algorithm in order to gain a better classification accuracy of communication signal modulation, but we still believe that the convolution neural network (CNN) can work better. However, its application to signal modulation recognition is often hampered by insufficient data and overfitting. Here, we propose a smart approach to programmatic data augmentation method by using the auxiliary classifier generative adversarial networks (ACGANs). The famous CNN model, AlexNet, has been utilized to be the classifier and ACGAN to be the generator, which will enlarge our data set. In order to alleviate the common issues in the traditional generative adversarial nets training, such as discriminator overfitting, generator disconverge, and mode collapse, we apply several training tricks in our training. With the result on original data set as our baseline, we will evaluate our result on enlarged data set to validate the ACGAN’s performance. The result shows that we can gain 0.1~6% increase in the classification accuracy in the ACGAN-based data set.

Digital Signal Modulation Classification With Data Augmentation Using Generative Adversarial Nets in Cognitive Radio Networks

The rapid development of communication systems poses unprecedented challenges, e.g., handling exploding wireless signals in a real-time and fine-grained manner. Recent advances in data-driven machine learning algorithms, especially deep learning (DL), show great potential to address the challenges. However, waveforms in the physical layer may not be suitable for the prevalent classical DL models, such as convolution neural network (CNN) and recurrent neural network (RNN), which mainly accept formats of images, time series, and text data in the application layer. Therefore, it is of considerable interest to bridge the gap between signal waveforms to DL amenable data formats. In this article, we develop a framework to transform complex-valued signal waveforms into images with statistical significance, termed contour stellar image (CSI), which can convey deep level statistical information from the raw wireless signal waveforms while being represented in an image data format. In this article, we explore several potential application scenarios and present effective CSI-based solutions to address the signal recognition challenges. Our investigation validates that CSI is a promising method to bridge the gap between signal recognition and DL.

Contour Stella Image and Deep Learning for Signal Recognition in the Physical Layer

Automatic modulation classification (AMC) plays an important role in both civilian and military applications. Today, increasingly more researchers apply a deep learning framework in AMC. However, few papers take into account that a typical deep model is difficult to deploy on resource constrained devices. In this paper, we propose a new filter-level pruning technique based on activation maximization (AM) that omits the less important convolutional filter. Compared to other network pruning techniques, the convolutional neural network pruned via the AM method achieves equal or higher classification accuracy in the RadioML2016.10a dataset.

An Improved Neural Network Pruning Technology for Automatic Modulation Classification in Edge Devices

Deep learning (DL) models are vulnerable to adversarial attacks, by adding a subtle perturbation which is imperceptible to the human eye, a convolutional neural network (CNN) can lead to erroneous results, which greatly reduces the reliability and security of the DL tasks. Considering the wide application of modulation recognition in the communication field and the rapid development of DL, by adding a well-designed adversarial perturbation to the input signal, this article explores the performance of attack methods on modulation recognition, measures the effectiveness of adversarial attacks on signals, and provides the empirical evaluation of the reliabilities of CNNs. The results indicate that the accuracy of the target model reduce significantly by adversarial attacks, when the perturbation factor is 0.001, the accuracy of the model could drop by about 50 ${\%}$ on average. Among them, iterative methods show greater attack performances than that of one-step method. In addition, the consistency of the waveform before and after the perturbation is examined, to consider whether the added adversarial examples are small enough (i.e., hard to distinguish by human eyes). This article also aims at inspiring researchers to further promote the CNNs reliabilities against adversarial attacks.

Adversarial Attacks in Modulation Recognition With Convolutional Neural Networks

Deep learning (DL) has been recognized as an effective solution for automatic modulation classification (AMC). However, most recent DL based AMC works are based on real-valued operations and representations. In this correspondence, we aim to demonstrate the high potential of complex-valued networks for AMC. We present the design of several key building blocks for complex-valued networks, such as complex convolution, complex batch-normalization, complex weight initialization, and complex dense strategies. We then provide a comparison study of three different neural network models and their complex-valued counterparts using the RadioML 2016.10 A dataset. Our results validate the superior performance in AMC achieved by the complex-valued networks.

Ya Tu

Papers

Digital Signal Modulation Classification With Data Augmentation Using Generative Adversarial Nets in Cognitive Radio Networks

Contour Stella Image and Deep Learning for Signal Recognition in the Physical Layer

An Improved Neural Network Pruning Technology for Automatic Modulation Classification in Edge Devices

Adversarial Attacks in Modulation Recognition With Convolutional Neural Networks

Complex-Valued Networks for Automatic Modulation Classification