https://hal.archives-ouvertes.fr/hal-01724317/document

On the role and the importance of features for background modeling and foreground detection

Proceedings of the British Machine Vision Conference

Annually, deep learning algorithms have proven their effectiveness in many vision-based applications, such as autonomous driving, traffic, and congestion monitoring, and so on. In computer vision, accurate traffic sign recognition and semantic road detection are vital challenges for increased safety, which are becoming a major research topic for intelligent transport systems community. In this paper, a deep learning-based driving assistance system has been proposed. To this end, we present hybrid 2D-3D CNN models based on the transfer learning paradigm to achieve better performance on benchmark real-world datasets. The primary goal of transfer learning is to improve the learning process in the target domain while transferring relevant knowledge from the source domain. We combine a pre-trained deep 2D CNN and a shallow 3D CNN to significantly reduce complexity and speed-up the training algorithm. The first model, called Hybrid-TSR, is intended to effectively address the task of traffic sign recognition. Hybrid-SRD is the second architecture that allows the semantic detection of road space through a combination of up-sampling and deconvolutional operations. The experimental results show that the proposed methods have considerable relevance in terms of efficiency and accuracy.

Transfer learning based hybrid 2D-3D CNN for traffic sign recognition and semantic road detection applied in advanced driver assistance systems

Background subtraction is an effective method of choice when it comes to detection of moving objects in videos and has been recognized as a breakthrough for the wide range of applications of intelligent video analytics (IVA). In recent years, a number of video datasets intended for background subtraction have been created to address the problem of large realistic datasets with accurate ground truth. The use of these datasets enables qualitative as well as quantitative comparisons and allows benchmarking of different algorithms. Finding the appropriate dataset is generally a cumbersome task for an exhaustive evaluation of algorithms. Therefore, we systematically survey standard video datasets and list their applicability for different applications. This paper presents a comprehensive account of public video datasets for background subtraction and attempts to cover the lack of a detailed description of each dataset. The video datasets are presented in chronological order of their appearance. Current trends of deep learning in background subtraction along with top-ranked background subtraction methods are also discussed in this paper. The survey introduced in this paper will assist researchers of the computer vision community in the selection of appropriate video dataset to evaluate their algorithms on the basis of challenging scenarios that exist in both indoor and outdoor environments.

/pdf/a-comprehensive-survey-of-video-datasets-for-background-2b8dcnw8xt.pdf

A Comprehensive Survey of Video Datasets for Background Subtraction

To address the issues encountered when using traditional airplane detection methods, including the low accuracy rate, high false alarm rate, and low detection speed due to small object sizes in aerial remote sensing images, we propose a remote sensing image airplane detection method that uses multilayer feature fusion in fully convolutional neural networks. The shallow layer and deep layer features are fused at the same scale after sampling to overcome the problems of low dimensionality in the deep layer and the inadequate expression of small objects. The sizes of candidate regions are modified to fit the size of the actual airplanes in the remote sensing images. The fully connected layers are replaced with convolutional layers to reduce the network parameters and adapt to different input image sizes. The region proposal network shares convolutional layers with the detection network, which ensures high detection efficiency. The simulation results indicate that, when compared to typical airplane detection methods, the proposed method is more accurate and has a lower false alarm rate. Additionally, the detection speed is considerably faster and the method can accurately and rapidly complete airplane detection tasks in aerial remote sensing images.

Rapid Airplane Detection in Remote Sensing Images Based on Multilayer Feature Fusion in Fully Convolutional Neural Networks.

Action recognition in video is widely applied in video indexing, intelligent surveillance, multimedia understanding, and other fields. A typical human action contains the spatiotemporal information from various scales. Learning and fusing the multi-temporal-scale information make action recognition more reliable in terms of recognition accuracy. To demonstrate this argument, in this paper, we use Res3D, a 3D Convolution Neural Network (CNN) architecture, to extract information in multiple temporal scales. And in each temporal scale, we transfer the knowledge learned from RGB to 3-channel optical flow (OF) and learn information from RGB and OF fields. We also propose Parallel Pair Discriminant Correlation Analysis (PPDCA) to fuse the multi-temporal-scale information into action representation with a lower dimension. Experimental results show that compared with single-temporal-scale method, the proposed multi-temporal-scale method gains higher recognition accuracy, and spends more time on feature extraction, but less time on classification due to the representation with lower dimension. Moreover, the proposed method achieves recognition performance comparable to that of the state-of-the-art methods. The source code and 3D filter animations are available online: https://github.com/JerryYaoGl/multi-temporal-scale
 .

Learning multi-temporal-scale deep information for action recognition

Background subtraction (BS) is one of the most commonly encountered tasks in video analysis and tracking systems. It distinguishes the foreground (moving objects) from the video sequences captured by static imaging sensors. Background subtraction in remote scene infrared (IR) video is important and common to lots of fields. This paper provides a Remote Scene IR Dataset captured by our designed medium-wave infrared (MWIR) sensor. Each video sequence in this dataset is identified with specific BS challenges and the pixel-wise ground truth of foreground (FG) for each frame is also provided. A series of experiments were conducted to evaluate BS algorithms on this proposed dataset. The overall performance of BS algorithms and the processor/memory requirements were compared. Proper evaluation metrics or criteria were employed to evaluate the capability of each BS algorithm to handle different kinds of BS challenges represented in this dataset. The results and conclusions in this paper provide valid references to develop new BS algorithm for remote scene IR video sequence, and some of them are not only limited to remote scene or IR video sequence but also generic for background subtraction. The Remote Scene IR dataset and the foreground masks detected by each evaluated BS algorithm are available online: https://github.com/JerryYaoGl/BSEvaluationRemoteSceneIR.

Comparative Evaluation of Background Subtraction Algorithms in Remote Scene Videos Captured by MWIR Sensors.

Spatial-temporal interaction learning based two-stream network for action recognition

Considering the high requirements of current kiwifruit picking recognition systems for mobile devices, including the small number of available features for image targets and small-scale aggregation, an enhanced YOLOX-S target detection algorithm for kiwifruit picking robots is proposed in this study. This involved designing a new multi-scale feature integration structure in which, with the aim of providing a small and lightweight model, the feature maps used for detecting large targets in the YOLOX model are eliminated, the feature map of small targets is sampled through the nearest neighbor values, the superficial features are spliced with the final features, the gradient of the SiLU activation function is perturbed, and the loss function at the output is optimized. The experimental results show that, compared with the original YOLOX-S, the enhanced model improved the detection average precision (AP) of kiwifruit images by 6.52%, reduced the number of model parameters by 44.8%, and improved the model detection speed by 63.9%. Hence, with its outstanding effectiveness and relatively light weight, the proposed model is capable of effectively providing data support for the 3D positioning and automated picking of kiwifruit. It may also successfully provide solutions in similar fields related to small target detection.

/pdf/lightweight-detection-algorithm-of-kiwifruit-based-on-piafhxi7.pdf

Lightweight Detection Algorithm of Kiwifruit Based on Improved YOLOX-S

Temporal action detection in long, untrimmed videos is an important yet challenging task that requires not only recognizing the categories of actions in videos, but also localizing the start and end times of each action. Recent years, artificial neural networks, such as Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) improve the performance significantly in various computer vision tasks, including action detection. In this paper, we make the most of different granular classifiers and propose to detect action from fine to coarse granularity, which is also in line with the people’s detection habits. Our action detection method is built in the ‘proposal then classification’ framework. We employ several neural network architectures as deep information extractor and segment-level (fine granular) and window-level (coarse granular) classifiers. Each of the proposal and classification steps is executed from the segment to window level. The experimental results show that our method not only achieves detection performance that is comparable to that of state-of-the-art methods, but also has a relatively balanced performance for different action categories.

Ping Jiang

Papers

Learning multi-temporal-scale deep information for action recognition

Comparative Evaluation of Background Subtraction Algorithms in Remote Scene Videos Captured by MWIR Sensors.

Spatial-temporal interaction learning based two-stream network for action recognition

Lightweight Detection Algorithm of Kiwifruit Based on Improved YOLOX-S

Temporal Action Detection in Untrimmed Videos from Fine to Coarse Granularity