Researchers are increasingly interested to study novel attacks on machine learning models. The classifiers are fooled by making small perturbation to the input or by learning patches that can be applied to objects. In this paper we present an iterative approach to generate a patch that when digitally placed on the face can successfully fool the facial recognition system. We focus on dodging attack where a target face is misidentified as any other face. The proof of concept is show-cased using FGSM and FaceNet face recognition system under the white-box attack. The framework is generic and it can be extended to other noise model and recognition system. It has been evaluated for different - patch size, noise strength, patch location, number of patches and dataset. The experiments shows that the proposed approach can significantly lower the recognition accuracy. Compared to state of the art digital-world attacks, the proposed approach is simpler and can generate inconspicuous natural looking patch with comparable fool rate and smallest patch size. 

On Fooling Facial Recognition Systems using Adversarial Patches

Given a set of points in the Euclidean space $\mathbb{R}^\ell$ with $\ell>1$, the pairwise distances between the points are determined by their spatial location and the metric $d$ that we endow $\mathbb{R}^\ell$ with. Hence, the distance $d(\mathbf x,\mathbf y)=\delta$ between two points is fixed by the choice of $\mathbf x$ and $\mathbf y$ and $d$. We study the related problem of fixing the value $\delta$, and the points $\mathbf x,\mathbf y$, and ask if there is a topological metric $d$ that computes the desired distance $\delta$. We demonstrate this problem to be solvable by constructing a metric to simultaneously give desired pairwise distances between up to $O(\sqrt\ell)$ many points in $\mathbb{R}^\ell$. We then introduce the notion of an $\varepsilon$-semimetric $\tilde{d}$ to formulate our main result: for all $\varepsilon>0$, for all $m\geq 1$, for any choice of $m$ points $\mathbf y_1,\ldots,\mathbf y_m\in\mathbb{R}^\ell$, and all chosen sets of values $\{\delta_{ij}\geq 0: 1\leq i<j\leq m\}$, there exists an $\varepsilon$-semimetric $\tilde{\delta}:\mathbb{R}^\ell\times \mathbb{R}^\ell\to\mathbb{R}$ such that $\tilde{d}(\mathbf y_i,\mathbf y_j)=\delta_{ij}$, i.e., the desired distances are accomplished, irrespectively of the topology that the Euclidean or other norms would induce. We showcase our results by using them to attack unsupervised learning algorithms, specifically $k$-Means and density-based (DBSCAN) clustering algorithms. These have manifold applications in artificial intelligence, and letting them run with externally provided distance measures constructed in the way as shown here, can make clustering algorithms produce results that are pre-determined and hence malleable. This demonstrates that the results of clustering algorithms may not generally be trustworthy, unless there is a standardized and fixed prescription to use a specific distance function.

Metricizing the Euclidean Space towards Desired Distance Relations in Point Clouds

Researchers are increasingly interested to study novel attacks on machine learning models. The classifiers are fooled by making small perturbation to the input or by learning patches that can be applied to objects. In this paper we present an iterative approach to generate a patch that when digitally placed on the face can successfully fool the facial recognition system. We focus on dodging attack where a target face is misidentified as any other face. The proof of concept is show-cased using FGSM and FaceNet face recognition system under the white-box attack. The framework is generic and it can be extended to other noise model and recognition system. It has been evaluated for different - patch size, noise strength, patch location, number of patches and dataset. The experiments shows that the proposed approach can significantly lower the recognition accuracy. Compared to state of the art digital-world attacks, the proposed approach is simpler and can generate inconspicuous natural looking patch with comparable fool rate and smallest patch size.

The state-of-the-art techniques create adversarial examples with a very low-intensity noise making the detection very hard. In the proposed work, we explore adding extra noise and filtering operations to differentiate between benign and adversarial examples. We hypothesize that adding lightweight noise affects the classification probability of adversarial examples more than benign ones. The proposed architecture uses them as features to train a binary classifier and detect adversarial examples in high-resolution, real-world images. Specifically, we look at beneficial noise generated through targeted adversarial attacks and noise from JPEG compression to perturb adversarial examples. Our standard classifier was able to distinguish benign and adversarial for BIM, PGD, and DeepFool in, approximately, 96.5%, 97%, and 85% of the cases, respectively, on high-resolution images from the ImageNet dataset.

Defense Against Adversarial Examples Using Beneficial Noise

The state-of-the-art techniques create adversarial examples with a very low-intensity noise making the detection very hard. In the proposed work, we explore adding extra noise and filtering operations to differentiate between benign and adversarial examples. We hypothesize that adding lightweight noise affects the classification probability of adversarial examples more than benign ones. The proposed architecture uses them as features to train a binary classifier and detect adversarial examples in high-resolution, real-world images. Specifically, we look at beneficial noise generated through targeted adversarial attacks and noise from JPEG compression to perturb adversarial examples. Our standard classifier was able to distinguish benign and adversarial for BIM, PGD, and DeepFool in, approximately, 96.5%, 97%, and 85% of the cases, respectively, on high-resolution images from the ImageNet dataset. 

Adversarial example(AE) aims at fooling a Convolution Neural Network by introducing small perturbations in the input image. The proposed work uses the magnitude and phase of the Fourier Spectrum and the entropy of the image to defend against AE. We demonstrate the defense in two ways: by training an adversarial detector and denoising the adversarial effect. Experiments were conducted on the low-resolution CIFAR-10 and high-resolution ImageNet datasets. The adversarial detector has 99% accuracy for FGSM and PGD attacks on the CIFAR-10 dataset. However, the detection accuracy falls to 50% for sophisticated DeepFool and Carlini & Wagner attacks on ImageNet. We overcome the limitation by using autoencoder and show that 70% of AEs are correctly classified after denoising.

Frequency Centric Defense Mechanisms against Adversarial Examples

Adversarial example (AE) aims at fooling a Convolution Neural Network by introducing small perturbations in the input image.The proposed work uses the magnitude and phase of the Fourier Spectrum and the entropy of the image to defend against AE. We demonstrate the defense in two ways: by training an adversarial detector and denoising the adversarial effect. Experiments were conducted on the low-resolution CIFAR-10 and high-resolution ImageNet datasets. The adversarial detector has 99% accuracy for FGSM and PGD attacks on the CIFAR-10 dataset. However, the detection accuracy falls to 50% for sophisticated DeepFool and Carlini & Wagner attacks on ImageNet. We overcome the limitation by using autoencoder and show that 70% of AEs are correctly classified after denoising.

Sanket B. Shah

Papers

Frequency Centric Defense Mechanisms against Adversarial Examples

Frequency Centric Defense Mechanisms against Adversarial Examples.