Face forgery detection is essential in combating malicious digital face attacks. Previous methods mainly rely on prior expert knowledge to capture specific forgery clues, such as noise patterns, blending boundaries, and frequency artifacts. However, these methods tend to get trapped in local optima, resulting in limited robustness and generalization capability. To address these issues, we propose a novel Critical Forgery Mining (CFM) framework, which can be flexibly assembled with various backbones to boost their generalization and robustness performance. Specifically, we first build a fine-grained triplet and suppress specific forgery traces through prior knowledge-agnostic data augmentation. Subsequently, we propose a fine-grained relation learning prototype to mine critical information in forgeries through instance and local similarity-aware losses. Moreover, we design a novel progressive learning controller to guide the model to focus on principal feature components, enabling it to learn critical forgery features in a coarse-to-fine manner. The proposed method achieves state-of-the-art forgery detection performance under various challenging evaluation settings.

Beyond the Prior Forgery Knowledge: Mining Critical Clues for General Face Forgery Detection

Advancements in computer vision and deep learning have made it difficult to distinguish generated Deepfake media in visual. While existing detection frameworks have achieved significant performance on the challenging Deepfake datasets, these approaches consider a single perspective. More importantly, in urban scenes, neither complex scenarios can be covered by a single view, nor the correlation between multiple information is well utilized. In this paper, to mine the new view for Deepfake detection and utilize the correlation of multi-view information contained in images, we propose a novel triple complementary streams detector, namely TCSD. Specifically, firstly, a novel depth estimator is designed to perceive depth information (DI) which is not utilized by previous methods. Then, to supplement the depth information for obtaining comprehensive forgery clues, we consider the incoherence between image foreground and background information (FBI) and the inconsistency between local and global information (LGI). In addition, attention-based multi-scale feature extraction (MsFE) module is designed to perceive more complementary features from DI, FBI and LGI. Finally, two attention-based feature fusion modules are proposed to adaptively fuse information. Extensive experiment results show that the proposed approach achieves the state-of-the-art performance on detecting Deepfake.

TCSD: Triple Complementary Streams Detector for Comprehensive Deepfake Detection

Advancements in computer vision and deep learning have led to difficulty in distinguishing the generated Deepfake media. In addition, recent forgery techniques also modify the audio information based on the forged video, which brings new challenges. However, due to the cross-modal bias, recent multimodal detection methods do not well explore the intra-modal and cross-modal forgery clues, which leads to limited detection performance. In this paper, we propose a novel audio-visual aware multimodal Deepfake detection framework to magnify intra-modal and cross-modal forgery clues. Firstly, to capture temporal intra-modal defects, Forgery Clues Magnification Transformer (FCMT) module is proposed to magnify forgery clues based on sequence-level relationships. Then, the Distribution Difference based Inconsistency Computing (DDIC) module based on Jensen–Shannon divergence is designed to adaptively align multimodal information for further magnifying the cross-modal inconsistency. Next, we further explore spatial artifacts by connecting multi-scale feature representation to provide comprehensive information. Finally, a feature fusion module is designed to adaptively fuse features to generate a more discriminative feature. Experiments demonstrate that the proposed framework outperforms independently trained models, and at the same time, yields superior generalization capability on unseen types of Deepfake. 

Xiaolong Liu

Papers

TCSD: Triple Complementary Streams Detector for Comprehensive Deepfake Detection

Magnifying multimodal forgery clues for Deepfake detection