“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

IEEE Transactions on Pattern Analysis and Machine Intelligence

In the real world, a realistic setting for computer vision or multimedia recognition problems is that we have some classes containing lots of training data and many classes contain a small amount of training data. Therefore, how to use frequent classes to help learning rare classes for which it is harder to collect the training data is an open question. Learning with Shared Information is an emerging topic in machine learning, computer vision and multimedia analysis. There are different level of components that can be shared during concept modeling and machine learning stages, such as sharing generic object parts, sharing attributes, sharing transformations, sharing regularization parameters and sharing training examples, etc. Regarding the specific methods, multi-task learning, transfer learning and deep learning can be seen as using different strategies to share information. These learning with shared information methods are very effective in solving real-world large-scale problems. This special issue aims at gathering the recent advances in learning with shared information methods and their applications in computer vision and multimedia analysis. Both state-of-the-art works, as well as literature reviews, are welcome for submission. Papers addressing interesting real-world computer vision and multimedia applications are especially encouraged. Topics of interest include, but are not limited to:  • Multi-task learning or transfer learning for large-scale computer vision and multimedia analysis • Deep learning for large-scale computer vision and multimedia analysis • Multi-modal approach for large-scale computer vision and multimedia analysis • Different sharing strategies, e.g., sharing generic object parts, sharing attributes, sharing transformations, sharing regularization parameters and sharing training examples, • Real-world computer vision and multimedia applications based on learning with shared information, e.g., event detection, object recognition, object detection, action recognition, human head pose estimation, object tracking, location-based services, semantic indexing. • New datasets and metrics to evaluate the benefit of the proposed sharing ability for the specific computer vision or multimedia problem. • Survey papers regarding the topic of learning with shared information.  Authors who are unsure whether their planned submission is in scope may contact the guest editors prior to the submission deadline with an abstract, in order to receive feedback.

IEEE transactions on pattern analysis and machine intelligence

A survey on deep learning and its applications

Deep neural network concepts for background subtraction:A systematic review and comparative evaluation

Background estimation is a fundamental step in many high-level vision applications, such as tracking and surveillance. Existing background estimation techniques suffer from performance degradation in the presence of challenges such as dynamic backgrounds, photometric variations, camera jitters, and shadows. To handle these challenges for the purpose of accurate background estimation, we propose a unified method based on Generative Adversarial Network (GAN) and image inpainting. The proposed method is based on a context prediction network, which is an unsupervised visual feature learning hybrid GAN model. Context prediction is followed by a semantic inpainting network for texture enhancement. We also propose a solution for arbitrary region inpainting using the center region inpainting method and Poisson blending technique. The proposed algorithm is compared with the existing state-of-the-art methods for background estimation and foreground segmentation and outperforms the compared methods by a significant margin.

Unsupervised deep context prediction for background estimation and foreground segmentation

Moving object detection is the fundamental step for various computer vision tasks. Many existing methods are still limited in accurately detecting the moving objects because of complex background scenes such as illumination condition, color saturation, and shadows etc. RPCA models have shown potential for moving object detection, where input data matrix is decomposed into a low-rank matrix representing the background image and a sparse component identifying moving objects. However, RPCA methods are not ideal for real-time processing because of the batch processing issues. These methods also show a performance degradation without encoding spatiotemporal and depth information. To address these problems, we investigate the performance of online Spatiotemporal RPCA (SRPCA) algorithm [1] for moving object detection using RGB-D videos. SRPCA is a graph regularized algorithm which preserves the low-rank spatiotemporal information in the form of dual spectral graphs. This graph regularized information is then encoded into the objective function which is solved using online optimization. Experiments show competitive results as compared to four state-of-the-art subspace learning methods.

Moving Object Detection on RGB-D Videos Using Graph Regularized Spatiotemporal RPCA

Moving object detection (MOD) is a fundamental step in many high-level vision-based applications, such as human activity analysis, visual object tracking, autonomous vehicles, surveillance, and security. Most of the existing MOD algorithms observe performance degradation in the presence of complex scenes containing camouflage objects, shadows, dynamic backgrounds, and varying illumination conditions, and captured by static cameras. To appropriately handle these challenges, we propose a Generative Adversarial Network (GAN) based on a moving object detection algorithm, called MOD_GAN. In the proposed algorithm, scene-specific GANs are trained in an unsupervised MOD setting, thereby enabling the algorithm to learn generating background sequences using input from uniformly distributed random noise samples. In addition to adversarial loss, during training, norm-based loss in the image space and discriminator feature-space is also minimized between the generated images and the training data. The additional losses enable the generator to learn subtle background details, resulting in a more realistic complex scene generation. During testing, a novel back-propagation based algorithm is used to generate images with statistics similar to the test images. More appropriate random noise samples are searched by directly minimizing the loss function between the test and generated images both in the image and discriminator feature-spaces. The network is not updated in this step; only the input noise samples are iteratively modified to minimize the loss function. Moreover, motion information is used to ensure that this loss is only computed on small-motion pixels. A novel dataset containing outdoor time-lapsed images from dawn to dusk with a full illumination variation cycle is also proposed to better compare the MOD algorithms in outdoor scenes. Accordingly, extensive experiments on five benchmark datasets and comparison with 30 existing methods demonstrate the strength of the proposed algorithm.

Unsupervised Moving Object Detection in Complex Scenes Using Adversarial Regularizations

Dynamic Background Subtraction (BS) is a fundamental problem in many vision-based applications. BS in real complex environments has several challenging conditions like illumination variations, shadows, camera jitters, and bad weather. In this study, we aim to address the challenges of BS in complex scenes by exploiting conditional least squares adversarial networks. During training, a scene-specific conditional least squares adversarial network with two additional regularizations including L 1 -Loss and Perceptual-Loss is employed to learn the dynamic background variations. The given input to the model is video frames conditioned on corresponding ground truth to learn the dynamic changes in complex scenes. Afterwards, testing is performed on unseen test video frames so that the generator would conduct dynamic background subtraction. The proposed method consisting of three loss-terms including least squares adversarial loss, L 1 -Loss and Perceptual-Loss is evaluated on two benchmark datasets CDnet2014 and BMC. The results of our proposed method show improved performance on both datasets compared with 10 existing state-of-the-art methods.

Maryam Sultana

Papers

Deep neural network concepts for background subtraction:A systematic review and comparative evaluation

Unsupervised deep context prediction for background estimation and foreground segmentation

Moving Object Detection on RGB-D Videos Using Graph Regularized Spatiotemporal RPCA

Unsupervised Moving Object Detection in Complex Scenes Using Adversarial Regularizations

Dynamic Background Subtraction Using Least Square Adversarial Learning