Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain
Honggu Liu,Xiaodan Li,Wenbo Zhou,Yuefeng Chen,Yuan He,Hui Xue,Weiming Zhang,Nenghai Yu +7 more
- pp 772-781
TLDR
Wang et al. as mentioned in this paper proposed a spatial-phase shallow learning (SPSL) method, which combines spatial image and phase spectrum to capture the up-sampling artifacts of face forgery to improve the transferability.Abstract:
The remarkable success in face forgery techniques has received considerable attention in computer vision due to security concerns. We observe that up-sampling is a necessary step of most face forgery techniques, and cumulative up-sampling will result in obvious changes in the frequency domain, especially in the phase spectrum. According to the property of natural images, the phase spectrum preserves abundant frequency components that provide extra information and complement the loss of the amplitude spectrum. To this end, we present a novel Spatial-Phase Shallow Learning (SPSL) method, which combines spatial image and phase spectrum to capture the up-sampling artifacts of face forgery to improve the transferability, for face forgery detection. And we also theoretically analyze the validity of utilizing the phase spectrum. Moreover, we notice that local texture information is more crucial than high-level semantic information for the face forgery detection task. So we reduce the receptive fields by shallowing the network to suppress high-level features and focus on the local region. Extensive experiments show that SPSL can achieve the state-of-the-art performance on cross-datasets evaluation as well as multi-class classification and obtain comparable results on single dataset evaluation.read more
Citations
More filters
Proceedings ArticleDOI
Detecting Deepfakes with Self-Blended Images
TL;DR: Novel synthetic training data called self-blended images (SBIs) to detect deepfakes are presented and extensive experiments show that the method improves the model generalization to unknown manipulations and scenes.
Proceedings ArticleDOI
Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection
TL;DR: This work addresses the generalizable deepfake detection from a simple principle: a generalizable representation should be sensitive to diverse types of forgeries and synthesize augmented forgeries with a pool of forgery configurations and strengthen the “sensitivity” to the forgeries by enforcing the model to predict the forgery configuration.
Proceedings ArticleDOI
End-to-End Reconstruction-Classification Learning for Face Forgery Detection
TL;DR: This paper proposes a forgery detection frame-work emphasizing the common compact representations of genuine faces based on reconstruction-classification learning, and builds bipartite graphs over the encoder and decoder features in a multi-scale fashion.
Proceedings ArticleDOI
Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection
TL;DR: This paper harnesses the natural correspondence between the visual and auditory modalities in real videos to learn temporally dense video representations that capture factors such as facial movements, expression, and identity, and suggests that leveraging natural and unlabelled videos is a promising direction for the development of more robust face forgery detectors.
Journal ArticleDOI
FInfer: Frame Inference-Based Deepfake Detection for High-Visual-Quality Videos
TL;DR: A frame inference-based detection framework (FInfer) to solve the problem of high-visual-quality Deepfake detection by first learning the referenced representations of the current and future frames’ faces and utilizing an autoregressive model.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Journal ArticleDOI
Generative Adversarial Nets
Ian Goodfellow,Jean Pouget-Abadie,Mehdi Mirza,Bing Xu,David Warde-Farley,Sherjil Ozair,Aaron Courville,Yoshua Bengio +7 more
TL;DR: A new framework for estimating generative models via an adversarial process, in which two models are simultaneously train: a generative model G that captures the data distribution and a discriminative model D that estimates the probability that a sample came from the training data rather than G.
Journal Article
Visualizing Data using t-SNE
TL;DR: A new technique called t-SNE that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map, a variation of Stochastic Neighbor Embedding that is much easier to optimize, and produces significantly better visualizations by reducing the tendency to crowd points together in the center of the map.
Proceedings ArticleDOI
Xception: Deep Learning with Depthwise Separable Convolutions
TL;DR: This work proposes a novel deep convolutional neural network architecture inspired by Inception, where Inception modules have been replaced with depthwise separable convolutions, and shows that this architecture, dubbed Xception, slightly outperforms Inception V3 on the ImageNet dataset, and significantly outperforms it on a larger image classification dataset.