Deep Image Restoration Model: A Defense Method Against Adversarial Attacks
TL;DR: This paper presents a deep image restoration model that restores adversarial examples so that the target model is classified correctly again and proves that its results are better than other rival methods.
Abstract: These days, deep learning and computer vision are much-growing fields in this modern world of information technology. Deep learning algorithms and computer vision have achieved great success in different applications like image classification, speech recognition, self-driving vehicles, disease diagnostics, and many more. Despite success in various applications, it is found that these learning algorithms face severe threats due to adversarial attacks. Adversarial examples are inputs like images in the computer vision field, which are intentionally slightly changed or perturbed. These changes are humanly imperceptible. But are misclassified by a model with high probability and severely affects the performance or prediction. In this scenario, we present a deep image restoration model that restores adversarial examples so that the target model is classified correctly again. We proved that our defense method against adversarial attacks based on a deep image restoration model is simple and state-of-the-art by providing strong experimental results evidence. We have used MNIST and CIFAR10 datasets for experiments and analysis of our defense method. In the end, we have compared our method to other state-ofthe-art defense methods and proved that our results are better than other rival methods.
Citations
More filters
TL;DR:
Abstract: Traditionally, nonlinear data processing has been approached via the use of polynomial filters, which are straightforward expansions of many linear methods, or through the use of neural network techniques. In contrast to linear approaches, which often provide algorithms that are simple to apply, nonlinear learning machines such as neural networks demand more computing and are more likely to have nonlinear optimization difficulties, which are more difficult to solve. Kernel methods, a recently developed technology, are strong machine learning approaches that have a less complicated architecture and give a straightforward way to transforming nonlinear optimization issues into convex optimization problems. Typical analytical tasks in kernel-based learning include classification, regression, and clustering, all of which are compromised. For image processing applications, a semisupervised deep learning approach, which is driven by a little amount of labeled data and a large amount of unlabeled data, has shown excellent performance in recent years. For their part, today’s semisupervised learning methods operate on the assumption that both labeled and unlabeled information are distributed in a similar manner, and their performance is mostly impacted by the fact that the two data sets are in a similar state of distribution as well. When there is out-of-class data in unlabeled data, the system’s performance will be adversely affected. When used in real-world applications, the capacity to verify that unlabeled data does not include data that belongs to a different category is difficult to obtain, and this is especially true in the field of synthetic aperture radar image identification (SAR). Using threshold filtering, this work addresses the problem of unlabeled input, including out-of-class data, having a detrimental influence on the performance of the model when it is utilized to train the model in a semisupervised learning environment. When the model is being trained, unlabeled data that does not belong to a category is filtered out by the model using two different sets of data that the model selects in order to optimize its performance. A series of experiments was carried out on the MSTAR data set, and the superiority of our method was shown when it was compared against a large number of current semisupervised classification algorithms of the highest level of sophistication. This was especially true when the unlabeled data had a significant proportion of data that did not fall into any of the categories. The performance of each kernel function is tested independently using two metrics, namely, the false alarm (FA) and the target miss (TM), respectively. These factors are used to calculate the proportion of incorrect judgments made using the techniques.
39 citations
01 Jan 2022
01 Jan 2022
TL;DR: In this article , the authors proposed a semi-supervised learning method of threshold filtering to solve the problem that the unlabeled data contains out-of-class data which affects the performance of the model.
Abstract: The semi-supervised deep learning technology driven by a small part of labeled data and a large amount of unlabeled data has achieved excellent performance in the field of image processing. However, the existing semi-supervised learning techniques are all carried out under the assumption that the labeled data and the unlabeled data are in the same distribution, and its performance is mainly due to the two being in the same distribution state. When there is out-of-class data in unlabeled data, its performance will be affected. In practical applications, it is difficult to ensure that unlabeled data does not contain out-of-category data, especially in the field of Synthetic Aperture Radar (SAR) image recognition. In order to solve the problem that the unlabeled data contains out-of-class data which affects the performance of the model, this paper proposes a semi-supervised learning method of threshold filtering. In the training process, through the two selections of data by the model, unlabeled data outside the category is filtered out to optimize the performance of the model. Experiments were conducted on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset, and compared with existing several state-of-the-art semi-supervised classification approaches, the superiority of our method was confirmed, especially when the unlabeled data contained a large amount of out-of-category data.
TL;DR: In this article , two IDS models with a convolutional neural network (CNN) were built to enhance the IDS accuracy and ensure it is more reliable in performing classification.
Abstract: An intrusion detection system (IDS) is an effective tool for securing networks and a dependable technique for improving a user’s internet security. It informs the administration whenever strange conduct occurs. An IDS fundamentally depends on the classification of network packets as benign or attack. Moreover, IDSs can achieve better results when built with machine learning (ML)/deep learning (DL) techniques, such as convolutional neural networks (CNNs). However, there is a limitation when building a reliable IDS using ML/DL techniques, which is their vulnerability to adversarial attacks. Such attacks are crafted by attackers to compromise the ML/DL models, which affects their accuracy. Thus, this paper describes the construction of a sustainable IDS based on the CNN technique, and it presents a method for defense against adversarial attacks that enhances the IDS’s accuracy and ensures it is more reliable in performing classification. To achieve this goal, first, two IDS models with a convolutional neural network (CNN) were built to enhance the IDS accuracy. Second, seven adversarial attack scenarios were designed against the aforementioned CNN-based IDS models to test their reliability and efficiency. The experimental results show that the CNN-based IDS models achieved significant increases in the intrusion detection system accuracy of 97.51% and 95.43% compared with the scores before the adversarial scenarios were applied. Furthermore, it was revealed that the adversarial attacks caused the models’ accuracy to significantly decrease from one attack scenario to another. The Auto-PGD and BIM attacks had the strongest effect against the CNN-based IDS models, with accuracy drops of 2.92% and 3.46%, respectively. Third, this research applied the adversarial perturbation elimination with generative adversarial nets (APE_GAN++) defense method to enhance the accuracy of the CNN-based IDS models after they were affected by adversarial attacks, which was shown to increase after the adversarial attacks in an intelligible way, with accuracy scores ranging between 78.12% and 89.40%.
References
More filters
[...]
TL;DR: Deep learning is making major advances in solving problems that have resisted the best attempts of the artificial intelligence community for many years, and will have many more successes in the near future because it requires very little engineering by hand and can easily take advantage of increases in the amount of available computation and data.
Abstract: Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics. Deep learning discovers intricate structure in large data sets by using the backpropagation algorithm to indicate how a machine should change its internal parameters that are used to compute the representation in each layer from the representation in the previous layer. Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on sequential data such as text and speech.
46,982 citations
01 Jan 1998
TL;DR: In this article, a graph transformer network (GTN) is proposed for handwritten character recognition, which can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters.
Abstract: Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.
42,067 citations
Book•
[...]
TL;DR: Deep learning as mentioned in this paper is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts, and it is used in many applications such as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames.
Abstract: Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.
38,208 citations
Posted Content•
TL;DR: This work proposes a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit and derives a robust initialization method that particularly considers the rectifier nonlinearities.
Abstract: Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU improves model fitting with nearly zero extra computational cost and little overfitting risk. Second, we derive a robust initialization method that particularly considers the rectifier nonlinearities. This method enables us to train extremely deep rectified models directly from scratch and to investigate deeper or wider network architectures. Based on our PReLU networks (PReLU-nets), we achieve 4.94% top-5 test error on the ImageNet 2012 classification dataset. This is a 26% relative improvement over the ILSVRC 2014 winner (GoogLeNet, 6.66%). To our knowledge, our result is the first to surpass human-level performance (5.1%, Russakovsky et al.) on this visual recognition challenge.
11,866 citations
Proceedings Article•
01 Jan 2014TL;DR: It is found that there is no distinction between individual highlevel units and random linear combinations of high level units, according to various methods of unit analysis, and it is suggested that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks.
Abstract: Deep neural networks are highly expressive models that have recently achieved state of the art performance on speech and visual recognition tasks. While their expressiveness is the reason they succeed, it also causes them to learn uninterpretable solutions that could have counter-intuitive properties. In this paper we report two such properties.
First, we find that there is no distinction between individual high level units and random linear combinations of high level units, according to various methods of unit analysis. It suggests that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks.
Second, we find that deep neural networks learn input-output mappings that are fairly discontinuous to a significant extend. We can cause the network to misclassify an image by applying a certain imperceptible perturbation, which is found by maximizing the network's prediction error. In addition, the specific nature of these perturbations is not a random artifact of learning: the same perturbation can cause a different network, that was trained on a different subset of the dataset, to misclassify the same input.
9,561 citations