“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

IEEE Transactions on Pattern Analysis and Machine Intelligence

In the real world, a realistic setting for computer vision or multimedia recognition problems is that we have some classes containing lots of training data and many classes contain a small amount of training data. Therefore, how to use frequent classes to help learning rare classes for which it is harder to collect the training data is an open question. Learning with Shared Information is an emerging topic in machine learning, computer vision and multimedia analysis. There are different level of components that can be shared during concept modeling and machine learning stages, such as sharing generic object parts, sharing attributes, sharing transformations, sharing regularization parameters and sharing training examples, etc. Regarding the specific methods, multi-task learning, transfer learning and deep learning can be seen as using different strategies to share information. These learning with shared information methods are very effective in solving real-world large-scale problems. This special issue aims at gathering the recent advances in learning with shared information methods and their applications in computer vision and multimedia analysis. Both state-of-the-art works, as well as literature reviews, are welcome for submission. Papers addressing interesting real-world computer vision and multimedia applications are especially encouraged. Topics of interest include, but are not limited to:  • Multi-task learning or transfer learning for large-scale computer vision and multimedia analysis • Deep learning for large-scale computer vision and multimedia analysis • Multi-modal approach for large-scale computer vision and multimedia analysis • Different sharing strategies, e.g., sharing generic object parts, sharing attributes, sharing transformations, sharing regularization parameters and sharing training examples, • Real-world computer vision and multimedia applications based on learning with shared information, e.g., event detection, object recognition, object detection, action recognition, human head pose estimation, object tracking, location-based services, semantic indexing. • New datasets and metrics to evaluate the benefit of the proposed sharing ability for the specific computer vision or multimedia problem. • Survey papers regarding the topic of learning with shared information.  Authors who are unsure whether their planned submission is in scope may contact the guest editors prior to the submission deadline with an abstract, in order to receive feedback.

IEEE transactions on pattern analysis and machine intelligence

A survey on deep learning and its applications

Deep neural network concepts for background subtraction:A systematic review and comparative evaluation

Conventional neural networks show a powerful framework for background subtraction in video acquired by static cameras. Indeed, the well-known SOBS method and its variants based on neural networks were the leader methods on the largescale CDnet 2012 dataset during a long time. Recently, convolutional neural networks which belong to deep learning methods were employed with success for background initialization, foreground detection and deep learned features. Currently, the top current background subtraction methods in CDnet 2014 are based on deep neural networks with a large gap of performance in comparison on the conventional unsupervised approaches based on multi-features or multi-cues strategies. Furthermore, a huge amount of papers was published since 2016 when Braham and Van Droogenbroeck published their first work on CNN applied to background subtraction providing a regular gain of performance. In this context, we provide the first review of deep neural network concepts in background subtraction for novices and experts in order to analyze this success and to provide further directions. For this, we first surveyed the methods used background initialization, background subtraction and deep learned features. Then, we discuss the adequacy of deep neural networks for background subtraction. Finally, experimental results are presented on the CDnet 2014 dataset.

Deep Neural Network Concepts for Background Subtraction: A Systematic Review and Comparative Evaluation

Video object segmentation is a fundamental step in many advanced vision applications. Most existing algorithms are based on handcrafted features such as HOG, super-pixel segmentation or texture-based techniques, while recently deep features have been found to be more efficient. Existing algorithms observe performance degradation in the presence of challenges such as illumination variations, shadows, and color camouflage. To handle these challenges we propose a fusion based moving object segmentation algorithm which exploits color as well as depth information using GAN to achieve more accuracy. Our goal is to segment moving objects in the presence of challenging background scenes, in real environments. To address this problem, GAN is trained in an unsupervised manner on color and depth information independently with challenging video sequences. During testing, the trained GAN generates backgrounds similar to that in the test sample. The generated background samples are then compared with the test sample to segment moving objects. The final result is computed by fusion of object boundaries in both modalities, RGB and the depth. The comparison of our proposed algorithm with five state-of-the-art methods on publicly available dataset has shown the strength of our algorithm for moving object segmentation in videos in the presence of challenging real scenarios.

Unsupervised RGBD Video Object Segmentation Using GANs.

Dynamic Background Modeling (DBM) is a crucial task in many computer vision based applications such as human activity analysis, traffic monitoring, surveillance, and security. DBM is extremely challenging in scenarios like illumination changes, camouflage, intermittent object motion or shadows. In this study, we proposed an end-to-end framework based on Generative Adversarial Network, which can generate dynamic background information for the task of DBM in an unsupervised manner. Our proposed model can handle the problem of DBM in the presence of the challenges mentioned above by generating data similar to the desired information. The primary aim of our proposed model during training is to learn all the dynamic changes in a scene-specific background information. While, during testing, inverse mapping of data to latent space representation in our model generates dynamic backgrounds similar to test data. The comparative analysis of our proposed model upon experimental evaluations on SBM.net and SBI benchmark datasets has outperformed eight existing methods for DBM in many challenging scenarios.

Unsupervised Adversarial Learning for Dynamic Background Modeling

Facial expression recognition (FER) is an important task for various computer vision applications The task becomes challenging when it requires the detection and encoding of macro- and micropatterns of facial expressions We present a two-stage texture feature extraction framework based on the local binary pattern (LBP) variants and evaluate its significance in recognizing posed and nonposed facial expressions We focus on the parametric limitations of the LBP variants and investigate their effects for optimal FER The size of the local neighborhood is an important parameter of the LBP technique for its extraction in images To make the LBP adaptive, we exploit the granulometric information of the facial images to find the local neighborhood size for the extraction of center-symmetric LBP (CS-LBP) features Our two-stage texture representations consist of an LBP variant and the adaptive CS-LBP features Among the presented two-stage texture feature extractions, the binarized statistical image features and adaptive CS-LBP features were found showing high FER rates Evaluation of the adaptive texture features shows competitive and higher performance than the nonadaptive features and other state-of-the-art approaches, respectively

Local binary pattern variants-based adaptive texture features analysis for posed and nonposed facial expression recognition

In many advanced video based applications background modeling is a pre-processing step to eliminate redundant data, for instance in tracking or video surveillance applications. Over the past years background subtraction is usually based on low level or hand-crafted features such as raw color components, gradients, or local binary patterns. The background subtraction algorithms performance suffer in the presence of various challenges such as dynamic backgrounds, photometric variations, camera jitters, and shadows. To handle these challenges for the purpose of accurate background modeling we propose a unified framework based on the algorithm of image inpainting. It is an unsupervised visual feature learning hybrid Generative Adversarial algorithm based on context prediction. We have also presented the solution of random region inpainting by the fusion of center region inpaiting and random region inpainting with the help of poisson blending technique. Furthermore we also evaluated foreground object detection with the fusion of our proposed method and morphological operations. The comparison of our proposed method with 12 state-of-the-art methods shows its stability in the application of background estimation and foreground detection.

Maryam Sultana

Papers

Deep Neural Network Concepts for Background Subtraction: A Systematic Review and Comparative Evaluation

Unsupervised RGBD Video Object Segmentation Using GANs.

Unsupervised Adversarial Learning for Dynamic Background Modeling

Local binary pattern variants-based adaptive texture features analysis for posed and nonposed facial expression recognition

Unsupervised Deep Context Prediction for Background Foreground Separation.