In the real world, a realistic setting for computer vision or multimedia recognition problems is that we have some classes containing lots of training data and many classes contain a small amount of training data. Therefore, how to use frequent classes to help learning rare classes for which it is harder to collect the training data is an open question. Learning with Shared Information is an emerging topic in machine learning, computer vision and multimedia analysis. There are different level of components that can be shared during concept modeling and machine learning stages, such as sharing generic object parts, sharing attributes, sharing transformations, sharing regularization parameters and sharing training examples, etc. Regarding the specific methods, multi-task learning, transfer learning and deep learning can be seen as using different strategies to share information. These learning with shared information methods are very effective in solving real-world large-scale problems. This special issue aims at gathering the recent advances in learning with shared information methods and their applications in computer vision and multimedia analysis. Both state-of-the-art works, as well as literature reviews, are welcome for submission. Papers addressing interesting real-world computer vision and multimedia applications are especially encouraged. Topics of interest include, but are not limited to:  • Multi-task learning or transfer learning for large-scale computer vision and multimedia analysis • Deep learning for large-scale computer vision and multimedia analysis • Multi-modal approach for large-scale computer vision and multimedia analysis • Different sharing strategies, e.g., sharing generic object parts, sharing attributes, sharing transformations, sharing regularization parameters and sharing training examples, • Real-world computer vision and multimedia applications based on learning with shared information, e.g., event detection, object recognition, object detection, action recognition, human head pose estimation, object tracking, location-based services, semantic indexing. • New datasets and metrics to evaluate the benefit of the proposed sharing ability for the specific computer vision or multimedia problem. • Survey papers regarding the topic of learning with shared information.  Authors who are unsure whether their planned submission is in scope may contact the guest editors prior to the submission deadline with an abstract, in order to receive feedback.

IEEE transactions on pattern analysis and machine intelligence

Colorization of grayscale images has been a hot topic in computer vision. Previous research mainly focuses on producing a colored image to match the original one. However, since many colors share the same gray value, an input grayscale image could be diversely colored while maintaining its reality. In this paper, we design a novel solution for unsupervised diverse colorization. Specifically, we leverage conditional generative adversarial networks to model the distribution of real-world item colors, in which we develop a fully convolutional generator with multi-layer noise to enhance diversity, with multi-layer condition concatenation to maintain reality, and with stride 1 to keep spatial information. With such a novel network architecture, the model yields highly competitive performance on the open LSUN bedroom dataset. The Turing test of 80 humans further indicates our generated color schemes are highly convincible.

Unsupervised Diverse Colorization via Generative Adversarial Networks

Colorization of grayscale images is a hot topic in computer vision. Previous research mainly focuses on producing a color image to recover the original one in a supervised learning fashion. However, since many colors share the same gray value, an input grayscale image could be diversely colorized while maintaining its reality. In this paper, we design a novel solution for unsupervised diverse colorization. Specifically, we leverage conditional generative adversarial networks to model the distribution of real-world item colors, in which we develop a fully convolutional generator with multi-layer noise to enhance diversity, with multi-layer condition concatenation to maintain reality, and with stride 1 to keep spatial information. With such a novel network architecture, the model yields highly competitive performance on the open LSUN bedroom dataset. The Turing test on 80 humans further indicates our generated color schemes are highly convincible.

Violent interaction detection is of vital importance in some video surveillance scenarios like railway stations, prisons or psychiatric centres. Existing vision-based methods are mainly based on hand-crafted features such as statistic features between motion regions, leading to a poor adaptability to another dataset. En lightened by the development of convolutional networks on common activity recognition, we construct a FightNet to represent the complicated visual violence interaction. In this paper, a new input modality, image acceleration field is proposed to better extract the motion attributes. Firstly, each video is framed as RGB images. Secondly, optical flow field is computed using the consecutive frames and acceleration field is obtained according to the optical flow field. Thirdly, the FightNet is trained with three kinds of input modalities, i.e., RGB images for spatial networks, optical flow images and acceleration images for temporal networks. By fusing results from different inputs, we conclude whether a video tells a violent event or not. To provide researchers a common ground for comparison, we have collected a violent interaction dataset (VID), containing 2314 videos with 1077 fight ones and 1237 no-fight ones. By comparison with other algorithms, experimental results demonstrate that the proposed model for violent interaction detection shows higher accuracy and better robustness.

https://iopscience.iop.org/article/10.1088/1742-6596/844/1/012044/pdf

Violent Interaction Detection in Video Based on Deep Learning

Action recognition has become a hot topic within computer vision However, the action recognition community has focused mainly on relatively simple actions like clapping, walking, jogging, etc The detection of specific events with direct practical use such as fights or in general aggressive behavior has been comparatively less studied Such capability may be extremely useful in some video surveillance scenarios like prisons, psychiatric centers or even embedded in camera phones As a consequence, there is growing interest in developing violence detection algorithms Recent work considered the well-known Bag-of-Words framework for the specific problem of fight detection Under this framework, spatio-temporal features are extracted from the video sequences and used for classification Despite encouraging results in which high accuracy rates were achieved, the computational cost of extracting such features is prohibitive for practical applications This work proposes a novel method to detect violence sequences Features extracted from motion blobs are used to discriminate fight and non-fight sequences Although the method is outperformed in accuracy by state of the art, it has a significantly faster computation time thus making it amenable for real-time applications

Fast fight detection.

This work proposes a novel system for Violent Scenes Detection, which is based on the combination of visual and audio features with machine learning at segment-level. Multiple Kernel Learning is applied so that multimodality of videos can be maximized. In particular, Mid-level Violence Clustering is proposed in order for mid-level concepts to be implicitly learned, without using manually tagged annotations. Finally a violence-score for each shot is calculated. The whole system is trained ona dataset from MediaEval 2013 Affect Task and evaluated by its official metric. The obtained results outperformed its best score.

/pdf/violent-scenes-detection-using-mid-level-violence-clustering-v8mmxwuhct.pdf

Violent Scenes Detection Using Mid-Level Violence Clustering

In this paper, we propose Motion Dense Sampling (MDS) for action recognition, which detects very informative interest points from video frames. MDS has three advantages compared to other existing methods. The first advantage is that MDS detects only interest points which belong to action regions of all regions of a video frame. The second one is that it can detect the constant number of points even when the size of action region in an image drastically changes. The Third one is that MDS enables to describe scale invariant features by computing sampling scale for each frame based on the size of action regions. Thus, our method detects much more informative interest points from videos unlike other methods. We also propose Category Clustering and Component Clustering, which generate the very effective codebook for action recognition. Experimental results show a significant improvement over existing methods on YouTube dataset. Our method achieves 87.5 % accuracy for video classification by using only one descriptor.

https://link.springer.com/content/pdf/10.1007%2Fs11042-014-2112-1.pdf

Motion dense sampling and component clustering for action recognition

Automatic image colorization is one of the attractive research topics in image processing. The most crucial task in this field is how to design an algorithm to define appropriate color from the reference image(s) for propagating to the target image. In other words, we need to determine whether two pixels in reference and target images have similar color. In previous methods, many approaches have been introduced mostly based on local feature matching algorithms. However, they still have some defects as well as time-consuming. In this paper, we will present a novel automatic image colorization method based on Feature Lines. Feature Lines is our new concept, which enhances the concept of Color Lines. It represents the distribution of each pixel feature vector as being elongated around the lines so that we are able to assemble the similar feature pixels into one feature line. By introducing this new technique, pixel matching between reference and target images performs precisely. The experimental achievements show our proposed method achieves smoother, evener and more natural color assignment than the previous methods.

Automatic Image Colorization based on Feature Lines

/pdf/color-segmentation-based-depth-adjustment-for-3d-model-ajcev897v9.pdf

Color Segmentation Based Depth Adjustment for 3D Model Reconstruction from a Single Input Image

In this paper, we propose the motion dense sampling (MDS) for video classification, which detects very informative interest points from video frames. MDS has two advantages compared to other existing methods. The first advantage is that MDS detects only interest points which belong to foreground regions of all regions of a video frame. Also it can detect the constant number of points even when the size of foreground region in an image drastically changes. The Second one is that MDS enable to describe scale invariable features by computing sampling scale for each frame based on the size of foreground regions. Thus, our method detects much more informative interest points from videos than other methods. Experimental results show a significant improvement over existing methods on YouTube dataset. Our method achieves 86.8% accuracy for video classification by using only one descriptor.

Terumasa Aoki

Papers

Violent Scenes Detection Using Mid-Level Violence Clustering

Motion dense sampling and component clustering for action recognition

Automatic Image Colorization based on Feature Lines

Color Segmentation Based Depth Adjustment for 3D Model Reconstruction from a Single Input Image

Motion Dense Sampling for Video Classification