5分で分かる!? 有名論文ナナメ読み：Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

How do humans recognize the action "opening a book" ? We argue that there are two important cues: modeling temporal shape dynamics and modeling functional relationships between humans and objects. In this paper, we propose to represent videos as space-time region graphs which capture these two important cues. Our graph nodes are defined by the object region proposals from different frames in a long range video. These nodes are connected by two types of relations: (i) similarity relations capturing the long range dependencies between correlated objects and (ii) spatial-temporal relations capturing the interactions between nearby objects. We perform reasoning on this graph representation via Graph Convolutional Networks. We achieve state-of-the-art results on both Charades and Something-Something datasets. Especially for Charades, we obtain a huge 4.4% gain when our model is applied in complex environments.

Videos as Space-Time Region Graphs.

Event-based cameras can measure intensity changes (called ‘events’) with microsecond accuracy under high-speed motion and challenging lighting conditions. With the active pixel sensor (APS), the event camera allows simultaneous output of the intensity frames. However, the output images are captured at a relatively low frame-rate and often suffer from motion blur. A blurry image can be regarded as the integral of a sequence of latent images, while the events indicate the changes between the latent images. Therefore, we are able to model the blur-generation process by associating event data to a latent image. In this paper, we propose a simple and effective approach, the Event-based Double Integral (EDI) model, to reconstruct a high frame-rate, sharp video from a single blurry frame and its event data. The video generation is based on solving a simple non-convex optimization problem in a single scalar variable. Experimental results on both synthetic and real images demonstrate the superiority of our EDI model and optimization method in comparison to the state-of-the-art.

Bringing a Blurry Frame Alive at High Frame-Rate With an Event Camera

In recent years, the research community has approached the problem of vehicle re-identification (re-id) with attention-based models, specifically focusing on regions of a vehicle containing discriminative information. These re-id methods rely on expensive key-point labels, part annotations, and additional attributes including vehicle make, model, and color. Given the large number of vehicle re-id datasets with various levels of annotations, strongly-supervised methods are unable to scale across different domains. In this paper, we present Self-supervised Attention for Vehicle Re-identification (SAVER), a novel approach to effectively learn vehicle-specific discriminative features. Through extensive experimentation, we show that SAVER improves upon the state-of-the-art on challenging VeRi, VehicleID, Vehicle-1M and VERI-Wild datasets.

The Devil Is in the Details: Self-supervised Attention for Vehicle Re-identification

Event-based cameras can measure intensity changes (called `{\it events}') with microsecond accuracy under high-speed motion and challenging lighting conditions. With the active pixel sensor (APS), the event camera allows simultaneous output of the intensity frames. However, the output images are captured at a relatively low frame-rate and often suffer from motion blur. A blurry image can be regarded as the integral of a sequence of latent images, while the events indicate the changes between the latent images. Therefore, we are able to model the blur-generation process by associating event data to a latent image. In this paper, we propose a simple and effective approach, the \textbf{Event-based Double Integral (EDI)} model, to reconstruct a high frame-rate, sharp video from a single blurry frame and its event data. The video generation is based on solving a simple non-convex optimization problem in a single scalar variable. Experimental results on both synthetic and real images demonstrate the superiority of our EDI model and optimization method in comparison to the state-of-the-art.

Bringing a Blurry Frame Alive at High Frame-Rate with an Event Camera

We present a solution for the goal of extracting a video from a single motion blurred image to sequentially reconstruct the clear views of a scene as beheld by the camera during the time of exposure. We first learn motion representation from sharp videos in an unsupervised manner through training of a convolutional recurrent video autoencoder network that performs a surrogate task of video reconstruction. Once trained, it is employed for guided training of a motion encoder for blurred images. This network extracts embedded motion information from the blurred image to generate a sharp video in conjunction with the trained recurrent video decoder. As an intermediate step, we also design an efficient architecture that enables real-time single image deblurring and outperforms competing methods across all factors: accuracy, speed, and compactness. Experiments on real scenes and standard datasets demonstrate the superiority of our framework over the state-of-the-art and its ability to generate a plausible sequence of temporally consistent sharp frames.

/pdf/bringing-alive-blurred-moments-1c2aou6nk5.pdf

Bringing Alive Blurred Moments

Vehicle re-identification and anomaly detection are useful tools in traffic analytics applications. Vehicle reidentification is particularly challenging due to variations in viewpoint, illumination and occlusion. Moreover, the reality of multiple vehicles having the same make and model hinders the design of traditional deep network-based solutions. In this work, we leverage an attention-based model which learns to focus on different parts of a vehicle by conditioning the feature maps on visible key-points. We use triplet embedding to reduce the dimensionality of the features obtained from the ensemble of networks trained using different datasets. To address the problem of anomaly detection, we design an unsupervised algorithm to detect and localize anomalies in traffic scenes. To handle moving cameras, we use the results obtained from tracking to generate anomaly proposals which are then filtered in successive steps. We show the effectiveness of our method on the Nvidia AI City vehicle re-identification dataset, where we obtain mean Average Precision (mAP) score of 60.78% placing us at the 8th position out of 84 participating teams. In addition, we achieved the S3 score of 22.07% for vehicle anomaly detection.

https://openaccess.thecvf.com/content_CVPRW_2019/papers/AI City/Khorramshahi_Attention_Driven_Vehicle_Re-identification_and_Unsupervised_Anomaly_Detection_for_Traffic_CVPRW_2019_paper.pdf

Attention Driven Vehicle Re-identification and Unsupervised Anomaly Detection for Traffic Understanding

This paper addresses the problem of obtaining a blur-based segmentation map from a single image affected by motion or defocus blur Since traditional hand-designed priors have fundamental limitations, we utilise deep neural networks to learn features related to blur and enable a pixel-level blur classification Our approach mitigates the ambiguities present in blur detection task by introducing joint learning of global context and local features into the framework Specifically, we train two sub-networks to perform the task at global (image) and local (patch) levels We aggregate the pixel-level probabilities estimated by two networks and feed them to a MRF based framework which returns a refined and dense segmentation-map of the image with respect to blur We also demonstrate via both qualitative and quantitative evaluation, that our approach performs favorably against state-of-the-art blur detection or segmentation works, and show its utility to applications of automatic image matting and blur magnification

Learning Based Single Image Blur Detection and Segmentation

Most human action recognition systems typically consider static appearances and motion as independent streams of information. In this paper, we consider the evolution of human pose and propose a method to better capture interdependence among skeleton joints. Our model extracts motion information from each joint independently, reweighs the information and finally performs inter-joint reasoning. The effectiveness of pose and joint-based representations is strengthened using a geometry-aware data augmentation technique which jitters pose heatmaps while retaining the dynamics of the action. Our best model gives an absolute improvement of 8.19% on JHMDB, 4.31% on HMDB and 1.55 mAP on Charades datasets over state-of-the-art methods using pose heat-maps alone. Fusing with RGB and flow streams leads to improvement over state-of-the-art. Our model also outperforms the baseline on Mimetics, a dataset with out-of-context videos by 1.14% while using only pose heatmaps. Further, to filter out clips irrelevant for action recognition, we re-purpose our model for clip selection guided by pose information and show improved performance using fewer clips.

Anshul Shah

Papers

Bringing Alive Blurred Moments

Attention Driven Vehicle Re-identification and Unsupervised Anomaly Detection for Traffic Understanding

Bringing Alive Blurred Moments

Learning Based Single Image Blur Detection and Segmentation

Pose And Joint-Aware Action Recognition.