Nowadays, full face synthesis and partial face manipulation by virtue of the generative adversarial networks (GANs) have raised wide public concerns. In the multi-media forensics area, detecting and ultimately locating the image forgery have become imperative. We investigated the architecture of existing GAN-based face manipulation methods and observed that the imperfection of upsampling methods therewithin could be served as an important asset for GAN-synthesized fake images detection and forgery localization. Based on this basic observation, we have proposed a novel approach to obtain high localization accuracy, at full resolution, on manipulated facial images. To the best of our knowledge, this is the very first attempt to solve the GAN-based fake localization problem with a gray-scale fakeness prediction map that preserves more information of fake regions. To improve the universality of FakeLocator across multifarious facial attributes, we introduce an attention mechanism to guide the training of the model. Experimental results on the CelebA and FFHQ databases with seven different state-of-the-art GAN-based face generation methods show the effectiveness of our method. Compared with the baseline, our method performs two times better on various metrics. Moreover, the proposed method is robust against various real-world facial image degradations such as JPEG compression, low-resolution, noise, and blur.

FakeLocator: Robust Localization of GAN-Based Face Manipulations

The chest X-ray plays a key role in screening and diagnosis of many lung diseases including the COVID-19. More recently, many works construct deep neural networks (DNNs) for chest X-ray images to realize automated and efficient diagnosis of lung diseases. However, bias field caused by the improper medical image acquisition process widely exists in the chest X-ray images while the robustness of DNNs to the bias field is rarely explored, which definitely poses a threat to the X-ray-based automated diagnosis system. In this paper, we study this problem based on the recent adversarial attack and propose a brand new attack, i.e., the adversarial bias field attack where the bias field instead of the additive noise works as the adversarial perturbations for fooling the DNNs. This novel attack posts a key problem: how to locally tune the bias field to realize high attack success rate while maintaining its spatial smoothness to guarantee high realisticity. These two goals contradict each other and thus has made the attack significantly challenging. To overcome this challenge, we propose the adversarial-smooth bias field attack that can locally tune the bias field with joint smooth & adversarial constraints. As a result, the adversarial X-ray images can not only fool the DNNs effectively but also retain very high level of realisticity. We validate our method on real chest X-ray datasets with powerful DNNs, e.g., ResNet50, DenseNet121, and MobileNet, and show different properties to the state-of-the-art attacks in both image realisticity and attack transferability. Our method reveals the potential threat to the DNN-based X-ray automated diagnosis and can definitely benefit the development of bias-field-robust automated diagnosis system.

Bias Field Poses a Threat to DNN-based X-Ray Recognition.

Deep learning is used to address a wide range of challenging issues including large data analysis, image processing, object detection, and autonomous control. In the same way, deep learning techniques are also used to develop software and techniques that pose a danger to privacy, democracy, and national security. Fake content in the form of images and videos using digital manipulation with artificial intelligence (AI) approaches has become widespread during the past few years. Deepfakes, in the form of audio, images, and videos, have become a major concern during the past few years. Complemented by artificial intelligence, deepfakes swap the face of one person with the other and generate hyper-realistic videos. Accompanying the speed of social media, deepfakes can immediately reach millions of people and can be very dangerous to make fake news, hoaxes, and fraud. Besides the well-known movie stars, politicians have been victims of deepfakes in the past, especially US presidents Barak Obama and Donald Trump, however, the public at large can be the target of deepfakes. To overcome the challenge of deepfake identification and mitigate its impact, large efforts have been carried out to devise novel methods to detect face manipulation. This study also discusses how to counter the threats from deepfake technology and alleviate its impact. The outcomes recommend that despite a serious threat to society, business, and political institutions, they can be combated through appropriate policies, regulation, individual actions, training, and education. In addition, the evolution of technology is desired for deepfake identification, content authentication, and deepfake prevention. Different studies have performed deepfake detection using machine learning and deep learning techniques such as support vector machine, random forest, multilayer perceptron, k-nearest neighbors, convolutional neural networks with and without long short-term memory, and other similar models. This study aims to highlight the recent research in deepfake images and video detection, such as deepfake creation, various detection algorithms on self-made datasets, and existing benchmark datasets.

/pdf/a-review-of-image-processing-techniques-for-deepfakes-2crtukio.pdf

A Review of Image Processing Techniques for Deepfakes

In this paper, we develop a privacy-preserving UAV system that does not infringe on the privacy of people in the videos taken by UAVs. Instead of blurring or masking the face parts of the videos, we want to exquisitely modify only the face parts so that the people in the modified videos still look like humans, but they become anonymous. Doing so, the semantic information of the videos can be preserved even with the anonymization. Specifically, based on the latest generative adversarial network architecture, we propose a deep learning-based face-anonymization scheme so that each modified face part looks like the face of a person who does not actually exist. The trained face-anonymizer is then mounted on the UAV system we have implemented. Through experiments, we confirm that the developed privacy-preserving UAV system anonymizes UAV’s first-person videos so that the people in the video are not recognized as anyone in the dataset used. In addition, we show that even with such anonymized videos, the perception performance required for performing UAV’s essential functions such as simultaneous localization and mapping is not degraded.

Development of a Privacy-Preserving UAV System With Deep Learning-Based Face Anonymization

Recently, Deepfake arises as a powerful tool to fool the existing real-world face detection systems, which has received wide attention in both academia and society. Most existing forgery face detection methods use heuristic clues to build a binary forgery detector, which mainly takes advantage of the empirical observation based on abnormal texture, blending clues, or high-frequency noise, etc.. However, heuristic clues only reflect certain aspects of the forgery, which might lead to model bias or sub-optimization. Our recent observations indicate that most of the forgery clues are hidden in the informative region, which can be measured quantitatively by the classic information maximization theory. Motivated by this, we make the first attempt to introduce the self-information metric to enhance the feature representation for forgery detection. The proposed metric can be formulated as a plug-and-play block, termed self-information attention (SIA) module, which can be integrated with most of the top-performance deep models to boost their detection performance. The SIA module can explicitly help the model locate the informative regions and recalibrate channel-wise feature responses, which improves both model’s performance and generalization with few additional parameters. Extensive experiments on several large-scale benchmarks demonstrate the superiority of the proposed method against the state-of-the-art competitors. 

An Information Theoretic Approach for Attention-Driven Face Forgery Detection

The creation and the manipulation of facial appearance via deep generative approaches, known as DeepFake, have achieved significant progress and promoted a wide range of benign and malicious applications. The evil side of this new technique poses another popular study, i.e., DeepFake detection aiming to identify the fake faces from the real ones. With the rapid development of the DeepFake-related studies in the community, both sides (i.e., DeepFake generation and detection) have formed the relationship of the battleground, pushing the improvements of each other and inspiring new directions, e.g., the evasion of DeepFake detection. Nevertheless, the overview of such battleground and the new direction is unclear and neglected by recent surveys due to the rapid increase of related publications, limiting the in-depth understanding of the tendency and future works. 
To fill this gap, in this paper, we provide a comprehensive overview and detailed analysis of the research work on the topic of DeepFake generation, DeepFake detection as well as evasion of DeepFake detection, with more than 191 research papers carefully surveyed. We present the taxonomy of various DeepFake generation methods and the categorization of various DeepFake detection methods, and more importantly, we showcase the battleground between the two parties with detailed interactions between the adversaries (DeepFake generation) and the defenders (DeepFake detection). The battleground allows fresh perspective into the latest landscape of the DeepFake research and can provide valuable analysis towards the research challenges and opportunities as well as research trends and directions in the field of DeepFake generation and detection. We also elaborately design interactive diagrams (this http URL) to allow researchers to explore their own interests on popular DeepFake generators or detectors.

Countering Malicious DeepFakes: Survey, Battleground, and Horizon.

The creation or manipulation of facial appearance through deep generative approaches, known as DeepFake, have achieved significant progress and promoted a wide range of benign and malicious applications, e.g., visual effect assistance in movie and misinformation generation by faking famous persons. The evil side of this new technique poses another popular study, i.e., DeepFake detection aiming to identify the fake faces from the real ones. With the rapid development of the DeepFake-related studies in the community, both sides (i.e., DeepFake generation and detection) have formed the relationship of battleground, pushing the improvements of each other and inspiring new directions, e.g., the evasion of DeepFake detection. Nevertheless, the overview of such battleground and the new direction is unclear and neglected by recent surveys due to the rapid increase of related publications, limiting the in-depth understanding of the tendency and future works. To fill this gap, in this paper, we provide a comprehensive overview and detailed analysis of the research work on the topic of DeepFake generation, DeepFake detection as well as evasion of DeepFake detection, with more than 318 research papers carefully surveyed. We present the taxonomy of various DeepFake generation methods and the categorization of various DeepFake detection methods, and more importantly, we showcase the battleground between the two parties with detailed interactions between the adversaries (DeepFake generation) and the defenders (DeepFake detection). The battleground allows fresh perspective into the latest landscape of the DeepFake research and can provide valuable analysis towards the research challenges and opportunities as well as research trends and future directions. We also elaborately design interactive diagrams (http://www.xujuefei.com/dfsurvey) to allow researchers to explore their own interests on popular DeepFake generators or detectors. 

/pdf/countering-malicious-deepfakes-survey-battleground-and-114whpf3.pdf

Countering Malicious DeepFakes: Survey, Battleground, and Horizon

There exist hydraulic parametric uncertainty and unknown external load in electro-hydraulic system (EHS), which inevitably decline the tracking performance and the stability of the closed-loop system. In this study, an optimal robust controller (ORC) is presented to solve this problem. Different from the traditional backstepping control, the ORC constructs the Hamilton–Jacobi–Bellman (HJB) equation based on the tracking error model of EHS to optimize the performance cost function as well as derive the optimal control variable. Furthermore, a critical neural network is adopted to estimate the optimal control solution of HJB equation such that guarantee the system state error of EHS uniformly ultimate boundary. Since the model uncertainty is considered in the HJB solution, the ORC has satisfactory robustness by using the network weight updation rather than direct parametric estimation or disturbance observer. The effectiveness of the proposed controller is verified by comparative simulation and experimental results. 

Optimal Robust Control of Electro-Hydraulic System Based on Hamilton–Jacobi–Bellman Solution With Backstepping Iteration

Human-exoskeleton coupling dynamics in the swing of lower limb

The control synchronization of multiple electrohydraulic actuators (MEHAs) is initially discussed to ensure the consensus of every electrohydraulic actuator (EHA) with three-order isomorphic dynamics. First, the EHA model is linearized using the Lie derivative method to obtain the state-space model of MEHAs. Then, the disturbance observer is used to estimate and compensate for the unknown external load caused by the driving force of a motion plant. Via the Lyapunov technique, this protocol asymptotically achieves consensus to a zero neighborhood with the ultimate boundaries of the MEHAs’ state errors. The effectiveness of the synchronous control protocol is verified by both simulation and experimental benches with two-node EHAs. 

/pdf/control-synchronization-design-of-a-multiple-196kdnm3.pdf

Josef Vedral

Papers

Countering Malicious DeepFakes: Survey, Battleground, and Horizon.

Countering Malicious DeepFakes: Survey, Battleground, and Horizon

Optimal Robust Control of Electro-Hydraulic System Based on Hamilton–Jacobi–Bellman Solution With Backstepping Iteration

Human-exoskeleton coupling dynamics in the swing of lower limb

Control Synchronization Design of a Multiple Electrohydraulic Actuator System with Linearization Dynamics and an External Disturbance Observer