Detecting GANs and Retouching Based Digital Alterations via DAD-HCNN
14 Jun 2020-pp 2870-2879
TL;DR: A hierarchical approach termed as DAD-HCNN which performs two-fold task: it differentiates between digitally generated images and digitally retouched images from the original unaltered images, and to increase the explainability of the decision, it also identifies the GAN architecture used to create the image.
Abstract: While image generation and editing technologies such as Generative Adversarial Networks and Photoshop are being used for creative and positive applications, the misuse of these technologies to create negative applications including Deep-nude and fake news is also increasing at a rampant pace. Therefore, detecting digitally created and digitally altered images is of paramount importance. This paper proposes a hierarchical approach termed as DAD-HCNN which performs two-fold task: (i) it differentiates between digitally generated images and digitally retouched images from the original unaltered images, and (ii) to increase the explainability of the decision, it also identifies the GAN architecture used to create the image. The effectiveness of the model is demonstrated on a database generated by combining face images generated from four different GAN architectures along with the retouched images and original images from existing benchmark databases.
Citations
More filters
TL;DR: This survey provides a thorough review of techniques for manipulating face images including DeepFake methods, and methods to detect such manipulations, with special attention to the latest generation of DeepFakes.
502 citations
Cites background from "Detecting GANs and Retouching Based..."
...In fact, these fingerprints seem to be dependent not only of the GAN architecture, but also of the different instances of it [49]–[51]....
[...]
14 Apr 2021
TL;DR: The proposed MIPGAN is derived from the StyleGAN with a newly formulated loss function exploiting perceptual quality and identity factor to generate a high quality morphed facial image with minimal artefacts and with high resolution.
Abstract: Face morphing attacks target to circumvent Face Recognition Systems (FRS) by employing face images derived from multiple data subjects (e.g., accomplices and malicious actors). Morphed images can be verified against contributing data subjects with a reasonable success rate, given they have a high degree of facial resemblance. The success of morphing attacks is directly dependent on the quality of the generated morph images. We present a new approach for generating strong attacks extending our earlier framework for generating face morphs. We present a new approach using an Identity Prior Driven Generative Adversarial Network, which we refer to as MIPGAN (Morphing through Identity Prior driven GAN) . The proposed MIPGAN is derived from the StyleGAN with a newly formulated loss function exploiting perceptual quality and identity factor to generate a high quality morphed facial image with minimal artefacts and with high resolution. We demonstrate the proposed approach’s applicability to generate strong morphing attacks by evaluating its vulnerability against both commercial and deep learning based Face Recognition System (FRS) and demonstrate the success rate of attacks. Extensive experiments are carried out to assess the FRS’s vulnerability against the proposed morphed face generation technique on three types of data such as digital images, re-digitized (printed and scanned) images, and compressed images after re-digitization from newly generated MIPGAN Face Morph Dataset . The obtained results demonstrate that the proposed approach of morph generation poses a high threat to FRS.
73 citations
Cites background from "Detecting GANs and Retouching Based..."
...Additionally, we investigate recent works about general face manipulation detection [54] [55] [56] and some results are shown in the supplementary material....
[...]
TL;DR: A new approach aimed to extract a Deepfake fingerprint from images is proposed, based on the Expectation-Maximization algorithm trained to detect and extract a fingerprint that represents the Convolutional Traces left by GANs during image generation.
Abstract: Advances in Artificial Intelligence and Image Processing are changing the way people interacts with digital images and video. Widespread mobile apps like FACEAPP make use of the most advanced Generative Adversarial Networks (GAN) to produce extreme transformations on human face photos such gender swap, aging, etc. The results are utterly realistic and extremely easy to be exploited even for non-experienced users. This kind of media object took the name of Deepfake and raised a new challenge in the multimedia forensics field: the Deepfake detection challenge. Indeed, discriminating a Deepfake from a real image could be a difficult task even for human eyes but recent works are trying to apply the same technology used for generating images for discriminating them with preliminary good results but with many limitations: employed Convolutional Neural Networks are not so robust, demonstrate to be specific to the context and tend to extract semantics from images. In this paper, a new approach aimed to extract a Deepfake fingerprint from images is proposed. The method is based on the Expectation-Maximization algorithm trained to detect and extract a fingerprint that represents the Convolutional Traces (CT) left by GANs during image generation. The CT demonstrates to have high discriminative power achieving better results than state-of-the-art in the Deepfake detection task also proving to be robust to different attacks. Achieving an overall classification accuracy of over 98%, considering Deepfakes from 10 different GAN architectures not only involved in images of faces, the CT demonstrates to be reliable and without any dependence on image semantic. Finally, tests carried out on Deepfakes generated by FACEAPP achieving 93% of accuracy in the fake detection task, demonstrated the effectiveness of the proposed technique on a real-case scenario.
49 citations
Cites methods from "Detecting GANs and Retouching Based..."
...[21] proposed a work known as DAD-HCNN, a new framework based on a hierarchical classification pipeline composed of three levels to distinguish respectively real Vs altered images (first level), retouched Vs GAN’s generated images (second level) and finally, the specific GAN architecture (third level)....
[...]
Posted Content•
TL;DR: In this paper, the authors provide a thorough review of techniques for manipulating face images including DeepFake methods, and methods to detect such manipulations, including entire face synthesis, identity swap (DeepFakes), attribute manipulation and expression swap.
Abstract: The free access to large-scale public databases, together with the fast progress of deep learning techniques, in particular Generative Adversarial Networks, have led to the generation of very realistic fake content with its corresponding implications towards society in this era of fake news. This survey provides a thorough review of techniques for manipulating face images including DeepFake methods, and methods to detect such manipulations. In particular, four types of facial manipulation are reviewed: i) entire face synthesis, ii) identity swap (DeepFakes), iii) attribute manipulation, and iv) expression swap. For each manipulation group, we provide details regarding manipulation techniques, existing public databases, and key benchmarks for technology evaluation of fake detection methods, including a summary of results from those evaluations. Among all the aspects discussed in the survey, we pay special attention to the latest generation of DeepFakes, highlighting its improvements and challenges for fake detection.
In addition to the survey information, we also discuss open issues and future trends that should be considered to advance in the field.
42 citations
Posted Content•
TL;DR: A novel approach to detect, attribute and localize GAN generated images that combines image features with deep learning methods is proposed.
Abstract: Recent advances in Generative Adversarial Networks (GANs) have led to the creation of realistic-looking digital images that pose a major challenge to their detection by humans or computers. GANs are used in a wide range of tasks, from modifying small attributes of an image (StarGAN [14]), transferring attributes between image pairs (CycleGAN [91]), as well as generating entirely new images (ProGAN [36], StyleGAN [37], SPADE/GauGAN [64]). In this paper, we propose a novel approach to detect, attribute and localize GAN generated images that combines image features with deep learning methods. For every image, co-occurrence matrices are computed on neighborhood pixels of RGB channels in different directions (horizontal, vertical and diagonal). A deep learning network is then trained on these features to detect, attribute and localize these GAN generated/manipulated images. A large scale evaluation of our approach on 5 GAN datasets comprising over 2.76 million images (ProGAN, StarGAN, CycleGAN, StyleGAN and SPADE/GauGAN) shows promising results in detecting GAN generated images.
23 citations
References
More filters
09 Sep 2013
TL;DR: The results show that state-of-the-art algorithms are sufficiently robust to deal with some alterations whereas other kinds of degradation can significantly affect the accuracy, thus requiring the adoption of proper detection mechanisms.
Abstract: This work is framed into the context of automatic face recognition in electronic identity documents. In particular we study the impact of digital alteration of the face images used for enrollment on the recognition accuracy. Alterations can be produced both unintentionally (e.g., by the acquisition or printing device) or intentionally (e.g., people modify images to appear more attractive). Our results show that state-of-the-art algorithms are sufficiently robust to deal with some alterations whereas other kinds of degradation can significantly affect the accuracy, thus requiring the adoption of proper detection mechanisms.
18 citations
"Detecting GANs and Retouching Based..." refers methods in this paper
...Retouching using both handcrafted methods and GANs can affect the biometric identification process as well [5, 10]....
[...]
Posted Content•
TL;DR: In this article, a supervised deep learning algorithm using Convolutional Neural Networks (CNNs) was proposed to detect synthetically altered images and achieved an accuracy of 99.65% on detecting retouching on the ND-IIITD dataset.
Abstract: Digitally retouching images has become a popular trend, with people posting altered images on social media and even magazines posting flawless facial images of celebrities. Further, with advancements in Generative Adversarial Networks (GANs), now changing attributes and retouching have become very easy. Such synthetic alterations have adverse effect on face recognition algorithms. While researchers have proposed to detect image tampering, detecting GANs generated images has still not been explored. This paper proposes a supervised deep learning algorithm using Convolutional Neural Networks (CNNs) to detect synthetically altered images. The algorithm yields an accuracy of 99.65% on detecting retouching on the ND-IIITD dataset. It outperforms the previous state of the art which reported an accuracy of 87% on the database. For distinguishing between real images and images generated using GANs, the proposed algorithm yields an accuracy of 99.83%.
17 citations
TL;DR: In this article, the authors introduce a new multi-demographic retouched faces (MDRF) dataset, which contains images belonging to two genders, male and female, and three ethnicities, Indian, Chinese and Caucasian.
Abstract: Digital retouching of face images is becoming more widespread due to the introduction of software packages that automate the task. Several researchers have introduced algorithms to detect whether a face image is original or retouched. However, previous work on this topic has not considered whether or how accuracy of retouching detection varies with the demography of face images. In this paper, we introduce a new Multi-Demographic Retouched Faces (MDRF) dataset, which contains images belonging to two genders, male and female, and three ethnicities, Indian, Chinese, and Caucasian. Further, retouched images are created using two different retouching software packages. The second major contribution of this research is a novel semi-supervised autoencoder incorporating “sub-class” information to improve classification. The proposed approach outperforms existing state-of-the-art detection algorithms for the task of generalized retouching detection. Experiments conducted with multiple combinations of ethnicities show that accuracy of retouching detection can vary greatly based on the demographics of the training and testing images.
16 citations
01 Jan 2009
TL;DR: The authors found that exposure to physical-ideal advertisements did not appear to affect body satisfaction, self-esteem, or internalization, and the level of internalization increased as the level for social comparison increased.
Abstract: The current study explored the effects of media exposure on men and women's body satisfaction, self-esteem, level of internalization of sociocultural ideals, and level of social comparison. Male and female undergraduates (N = 32) were exposed to television advertisements either with muscular men and thin women (sociocultural ideal group) or without those types of men and women (neutral advertisement group). Men were more satisfied with their bodies than women, and they internalized ideals less. Self-esteem and social comparison levels were similar for both men and women. In addition, exposure to physical-ideal advertisements did not appear to effect body satisfaction, self-esteem, or internalization. Also, the level of internalization increased as the level of social comparison increased. According to the U.S. Census Bureau (2006) Americans will spend approximately 3,592 hours this year on media usage with 1,704 of those hours being spent watching television. That is equivalent to spending five months using some form of media and watching about two and a half months worth of TV. It is only plausible to assume that something we spend so much of our time on has some affect on us. This idea has been the focus of a number of research studies (e.g., Bessenoff, 2006; Hawkins, Richards, Granley, & Stein, 2004). Many of these research studies have looked specifically at the ideals that the media portrays, and how those ideals (such as the thin-ideal) have affected how one views themselves. Both self-esteem and body satisfaction have been shown to be negatively effected by the media. That is why it is so important to understand the specific role that the media plays, what affect it has on one's wellbeing. Once the media's role is understood then ways can be found to reduce those damaging effects. In the following literature review the research studies that describe the media and its effects on men and women are explored. First, studies that address the general influence of the media and its separate influence on men and women are examined. Internalization of sociocultural ideals are described, as well as studies that address the media's effect on selfesteem and weight concerns. Also, body satisfaction in reference to how it is influenced by the media is presented, along with social comparison theory and its connection with the media's influence. Influence of the Media Many forms of media can be found all over the world. American society is especially involved in the use of the media. Americans are confronted with images of beautiful people, expensive must-haves, and the latest fashion almost everywhere they go. The mass media is the most powerful way to spread these images that represent sociocultural ideals (Tiggemann, 2003). One of most influential ideals spread by the media is society's ideal of beauty and attractiveness. The ultra-thin beautiful woman and the handsome muscular man are seen everywhere. And as the influence of media increases, the pressure to adhere to these ideals becomes greater. The standard of female attractiveness that the media portrays is becoming increasingly harder for women to live up to. Hawkins et al. (2004) stated that most of the women portrayed in the media are 15% below the average weight of women, and there has been evidence that these women have become increasingly thinner over the years. Not only are women pressured to be thin because of the beauty standard, but positive traits are also linked to this ideal of attractiveness (Greenberg & Worrell, 2005). Both men and women are confronted with pressures from the media to conform to society's attractiveness ideal. Unlike women, men are not pressured to be thin, but rather to be muscular. Media exposure has been shown to increase men's concern about muscularity and make them feel pressured to become more muscular (Botta, 2003; Hatoum, & Belle, 2004). The media often portrays that it is the strong handsome men who are popular and who get the beautiful women. Both men and women are faced with the demands that society places on them to conform to these ideals of attractiveness. Internalization of Sociocultural Ideals Sociocultural ideals, like the thin-ideal, are most influential when they are internalized. Internalization of sociocultural ideals of attractiveness is accepting or agreeing with social standards of beauty. Sometimes these ideals are internalized without one realizing that they are. Since media has a large part in spreading society's ideals it is no surprise that researchers have found that increased media consumption leads to increased internalization of the thin-ideal (e.g., Miller & Halberstadt, 2005; Tiggemann, 2003). Once that ideal in internalized it affects how one views their body and comparisons are made between their body and what society's standards are. Yet not everyone is affected by the thin-ideal. Some do not internalize this ideal, and therefore exposure to the media does not affect them as greatly. Posavac, Posavac, and Posavac (1998) suggested that there are two reasons why the media is not as influential on some people. They suggest that (a) their body is not much different than those of models presented in the media, or (b) physical attractiveness is not as important to them because they are confident in their skills and abilities. Even though some men and women are not affected by the thin-ideal there are still many that are, and their self-esteem can be a good indicator of the extent that the thin-ideal influences their lives. Self-Esteem and Weight Concern Self-esteem, a measure of how one feel's about oneself, can also effect how one feels about his or her weight. Having a high level of self-esteem can also help prevent the negative effects of the media's influence. Low self-esteem on the other hand can cause one to be more susceptible to media images. Research has shown that low levels of self-esteem in both men and women are predictive of more weight concern compared to those who have high levels of selfesteem (Hatoum & Belle, 2004; Posavac & Posavac, 2002). Not only does self-esteem affect weight concern and the media's level of influence, but self-esteem itself can be affected by the media. Research indicates that being exposed to thinideal images lowers self-esteem and increases the drive for thinness in women and the drive for muscularity in men (Bessenoff, 2006; Dohnt & Tiggemann, 2006; Hawkins et al., 2004; Hobza, Walker, Yakushko, & Peugh, 2007). Body Satisfaction and Self-Esteem Self-esteem levels have also been linked to body satisfaction. Usually when one is low the other is low as well. How an individual feels about his or her body is an important part of how he or she feels about themselves. Many individuals in today's society are not satisfied with their bodies. Researchers have found that this body dissatisfaction increases as self-esteem decreases (Bessenoff, 2006; Dohnt & Tiggemann, 2006; Hawkins et al., 2004). Since self-esteem and body satisfaction are related, it is no surprise that the media has just as strong an influence on body satisfaction as it does on self-esteem. Watson and Vaughn (2006) stated that sociocultural pressures to adhere to the ideal body image, as is reinforced by the media, is the cause for the large amount of body dissatisfaction found in many individuals, especially women. Exposure to media can not only cause body dissatisfaction, but body dissatisfaction can also cause one to be more apt to expose oneself to certain types of media that feed that dissatisfaction (Aubrey, 2006). Body dissatisfaction and self-esteem are also both affected by how an individual compares themselves to those media images. Social Comparison Theory Social comparison theory was first suggested by Festinger in 1954 (as cited in Wykes & Gunter, 2005). It states that individuals make comparisons between themselves and others who posses certain desired qualities or traits, and these comparisons help the individuals to establish their identity. Also, individuals differ in their tendencies to compare themselves to others. Those who are more likely to choose inappropriate comparison targets or to take part in upward comparisons are also more likely to be influenced by sociocultural ideals, especially those dealing with appearance (Wykes & Gunter, 2005). The more an individual engages in social comparison the more negative the media's influence will be. Social comparison theory can be seen as the chain that links together the media's effects on" internalization of ideals, self-esteem, and body dissatisfaction. Miller and Halberstadt (2005) found that men and women who were predisposed to social comparison were more aware of thinness norms and were more likely to internalize those norms. Researchers have also discovered that those who used social comparison where more affected by exposure to the thin-ideal and therefore had more weight concerns, more body dissatisfaction, and lower selfesteem (Bessenoff, 2006; Botta, 2003; Posavac & Posavac, 2OO2). The problem then rests in an individual's dependency upon comparison to others in an attempt to define themselves. Most of the research studies found similar effects from media exposure. Yet there were some limitations to some of the research that was done. Most of the research focused on white heterosexual women of collage or adolescent age. While there were some studies that used male participants, they were not nearly as numerous as the ones that had used female participants. Another limitation was the nature of the media sources that were used. Magazines were used the most as opposed to other types of media. The last major limitation was the lack of experimental research. Most of the research that was done used survey methods. It would be beneficial to use a design that is more experimental as opposed to only using surveys and questionnaires. The current research problem addressed the issue of whether or not social comparison leads to the int
16 citations
"Detecting GANs and Retouching Based..." refers background in this paper
...Some of these illintended applications of the society not only has an effect on law enforcement scenarios but can also lead to significant psychological and sociological implications [31]....
[...]
01 Jan 2020
TL;DR: This publication describes systems and techniques to detect if a camera application is applying an automated mechanism to beautify a human face present in a photograph.
Abstract: This publication describes systems and techniques to detect if a camera application is applying an automated mechanism to beautify a human face present in a photograph. Some camera applications may automatically adjust characteristics of a face to “beautify” them, such as by changing a skin tone. Although provided as a feature, this “beautification” potentially can be interpreted as cultural insensitivity or can result in mental health concerns. Accordingly, enabling a user to at least have knowledge of automated beautification can be beneficial. Detecting beautification is accomplished by determining if a camera application adjusts pixels corresponding to a face but not pixels corresponding to a rearranged face. Facial recognition software recognizes the face but fails to recognize the rearranged face. A tile from the face is compared to a corresponding tile from the rearranged face. If the two tiles are sufficiently dissimilar, then the system infers that the camera application has adjusted the pixels of the tile corresponding to the original facial image responsive to recognizing those pixels as part of a face. In this manner, the automated facial beautification can be detected.
1 citations
"Detecting GANs and Retouching Based..." refers background in this paper
...[29] detects whether a camera is performing automatic face beautification by comparing camera captured face and rearranged face images....
[...]