“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Despite active research and significant progress in the last 30 years, eye detection and tracking remains challenging due to the individuality of eyes, occlusion, variability in scale, location, and light conditions. Data on eye location and details of eye movements have numerous applications and are essential in face detection, biometric identification, and particular human-computer interaction tasks. This paper reviews current progress and state of the art in video-based eye detection and tracking in order to identify promising techniques as well as issues to be further addressed. We present a detailed review of recent eye models and techniques for eye detection and tracking. We also survey methods for gaze estimation and compare them based on their geometric properties and reported accuracies. This review shows that, despite their apparent simplicity, the development of a general eye detection technique involves addressing many challenges, requires further theoretical developments, and is consequently of interest to many other domains problems in computer vision and beyond.

/pdf/in-the-eye-of-the-beholder-a-survey-of-models-for-eyes-and-oxe7gfw5jq.pdf

In the Eye of the Beholder: A Survey of Models for Eyes and Gaze

Recognizing faces in unconstrained videos is a task of mounting importance. While obviously related to face recognition in still images, it has its own unique characteristics and algorithmic requirements. Over the years several methods have been suggested for this problem, and a few benchmark data sets have been assembled to facilitate its study. However, there is a sizable gap between the actual application needs and the current state of the art. In this paper we make the following contributions. (a) We present a comprehensive database of labeled videos of faces in challenging, uncontrolled conditions (i.e., ‘in the wild’), the ‘YouTube Faces’ database, along with benchmark, pair-matching tests1. (b) We employ our benchmark to survey and compare the performance of a large variety of existing video face recognition techniques. Finally, (c) we describe a novel set-to-set similarity measure, the Matched Background Similarity (MBGS). This similarity is shown to considerably improve performance on the benchmark tests.

/pdf/face-recognition-in-unconstrained-videos-with-matched-3tbbtju4in.pdf

Face recognition in unconstrained videos with matched background similarity

The demand for various multimedia applications is rapidly increasing due to the recent advance in the computing and network infrastructure, together with the widespread use of digital video technology. Among the key elements for the success of these applications is how to effectively and efficiently manage and store a huge amount of audio visual information, while at the same time providing user-friendly access to the stored data. This has fueled a quickly evolving research area known as video abstraction. As the name implies, video abstraction is a mechanism for generating a short summary of a video, which can either be a sequence of stationary images (keyframes) or moving images (video skims). In terms of browsing and navigation, a good video abstract will enable the user to gain maximum information about the target video sequence in a specified time constraint or sufficient information in the minimum time. Over past years, various ideas and techniques have been proposed towards the effective abstraction of video contents. The purpose of this article is to provide a systematic classification of these works. We identify and detail, for each approach, the underlying components and how they are addressed in specific works.

Video abstraction: A systematic review and classification

We propose a principled method for designing high level features forphoto quality assessment. Our resulting system can classify between high quality professional photos and low quality snapshots. Instead of using the bag of low-level features approach, we first determine the perceptual factors that distinguish between professional photos and snapshots. Then, we design high level semantic features to measure the perceptual differences. We test our features on a large and diverse dataset and our system is able to achieve a classification rate of 72% on this difficult task. Since our system is able to achieve a precision of over 90% in low recall scenarios, we show excellent results in a web image search application.

/pdf/the-design-of-high-level-features-for-photo-quality-3yiqez1ezj.pdf

The Design of High-Level Features for Photo Quality Assessment

We propose an unsupervised Local and Global feature extraction paradigm to approach the problem of facial expression recognition in thermal images. Starting from local, low-level features computed at interest point locations, our approach combines the localization of facial features with the holistic approach. The detailed steps are as follows: First, face localization using bi-modal thresholding is accomplished in order to localize facial features by way of a novel interest point detection and clustering approach. Second, we compute representative Eigenfeatures for feature extraction. Third, facial expression classification is made with a Support Vector Machine Committiee. Finally, the experiments over the IRIS data-set show that automation was achieved with good feature localization and classification performance.

Automatic Feature Localization in Thermal Images for Facial Expression Recognition

Visual learning of texture descriptors for facial expression recognition in thermal imagery

We describe two advanced video analysis techniques, including video-indexed by voice annotations (VIVA) and multi-media indexing and explorer (MINER). VIVA utilizes analyst call-outs (ACOs) in the form of chat messages (voice-to-text) to associate labels with video target tracks, to designate spatial-temporal activity boundaries and to augment video tracking in challenging scenarios. Challenging scenarios include low-resolution sensors, moving targets and target trajectories obscured by natural and man-made clutter. MINER includes: (1) a fusion of graphical track and text data using probabilistic methods; (2) an activity pattern learning framework to support querying an index of activities of interest (AOIs) and targets of interest (TOIs) by movement type and geolocation; and (3) a user interface to support streaming multi-intelligence data processing. We also present an activity pattern learning framework that uses the multi-source associated data as training to index a large archive of full-motion videos (FMV). VIVA and MINER examples are demonstrated for wide aerial/overhead imagery over common data sets affording an improvement in tracking from video data alone, leading to 84% detection with modest misdetection/false alarm results due to the complexity of the scenario. The novel use of ACOs and chat Sensors 2014, 14 19844 messages in video tracking paves the way for user interaction, correction and preparation of situation awareness reports.

Automatic Association of Chats and Video Tracks for Activity Learning and Recognition in Aerial Video Surveillance

Information Fusion consists of organizing a set of data for correlation in time, association over multimodal collections, and estimation in space. There exist many methods for object tracking and classification; however video tracking suffers from exact methods in object labeling, the ability to correlate tracks through dropouts, and determination of intent. Our novel solution is to fuse video data with text data for better simultaneous tracking and identification. The need for such solutions resides in answering user queries, linking information over different collections, and providing meaningful product reports. For example, text data can establish that a certain person should be visible in a video. Together, video-to-text (V2T) enhances situation awareness, provides situation understanding, and affords situation assessment. V2T is an example of hard (e.g., video) and soft (i.e., text) data fusion that links Level 5 User Refinement to Level 1 object tracking and characterization. A demonstrated example for multimodal text and video sensing is shown where context provides the means for associating the multimode data aligned in space and time.

Context aided video-to-text information fusion

In this paper we introduce a system to track pedestrians using a combined input from RGB and thermal cameras. Two major contributions are presented here. First is the novel probabilistic model of the scene background where each pixel is represented as a multi-modal distribution with the changing number of modalities for both color and thermal input. We demonstrate how to eliminate the influence of shadows with this type of fusion. Second, based on our background model we introduce a pedestrian tracker designed as a particle filter. We further develop a number of informed reversible transformations to sample the model probability space in order to maximize our model posterior probability. The novelty of our tracking approach also comes from a way we formulate observation likelihoods to account for 3D locations of the bodies with respect to the camera and occlusions by other tracked human bodies as well as static objects. The results of tracking on color and thermal sequences demonstrate that our algorithm is robust to illumination noise and performs well in the outdoor environments.

Riad I. Hammoud

Papers

Automatic Feature Localization in Thermal Images for Facial Expression Recognition

Visual learning of texture descriptors for facial expression recognition in thermal imagery

Automatic Association of Chats and Video Tracks for Activity Learning and Recognition in Aerial Video Surveillance

Context aided video-to-text information fusion

Pedestrian tracking by fusion of thermal-visible surveillance videos