Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

http://cvssp-data.eps.surrey.ac.uk/Personal/AdrianHilton/Publications_files/survey_techrep.pdf

A survey of advances in vision-based human motion capture and analysis

This paper aims to develop a method than can accurately estimate the crowd count from an individual image with arbitrary crowd density and arbitrary perspective. To this end, we have proposed a simple but effective Multi-column Convolutional Neural Network (MCNN) architecture to map the image to its crowd density map. The proposed MCNN allows the input image to be of arbitrary size or resolution. By utilizing filters with receptive fields of different sizes, the features learned by each column CNN are adaptive to variations in people/head size due to perspective effect or image resolution. Furthermore, the true density map is computed accurately based on geometry-adaptive kernels which do not need knowing the perspective map of the input image. Since exiting crowd counting datasets do not adequately cover all the challenging situations considered in our work, we have collected and labelled a large new dataset that includes 1198 images with about 330,000 heads annotated. On this challenging new dataset, as well as all existing datasets, we conduct extensive experiments to verify the effectiveness of the proposed model and method. In particular, with the proposed simple MCNN model, our method outperforms all existing methods. In addition, experiments show that our model, once trained on one dataset, can be readily transferred to a new dataset.

/pdf/single-image-crowd-counting-via-multi-column-convolutional-2h5l5iatou.pdf

Single-Image Crowd Counting via Multi-Column Convolutional Neural Network

A video surveillance system extracts video primitives and extracts event occurrences from the video primitives using event discriminators. The system can undertake a response, such as an alarm, based on extracted event occurrences.

Video surveillance system employing video primitives

Automatic video analysis from urban surveillance cameras is a fast-emerging field based on computer vision techniques. We present here a comprehensive review of the state-of-the-art computer vision for traffic video with a critical analysis and an outlook to future research directions. This field is of increasing relevance for intelligent transport systems (ITSs). The decreasing hardware cost and, therefore, the increasing deployment of cameras have opened a wide application field for video analytics. Several monitoring objectives such as congestion, traffic rule violation, and vehicle interaction can be targeted using cameras that were typically originally installed for human operators. Systems for the detection and classification of vehicles on highways have successfully been using classical visual surveillance techniques such as background estimation and motion tracking for some time. The urban domain is more challenging with respect to traffic density, lower camera angles that lead to a high degree of occlusion, and the variety of road users. Methods from object categorization and 3-D modeling have inspired more advanced techniques to tackle these challenges. There is no commonly used data set or benchmark challenge, which makes the direct comparison of the proposed algorithms difficult. In addition, evaluation under challenging weather conditions (e.g., rain, fog, and darkness) would be desirable but is rarely performed. Future work should be directed toward robust combined detectors and classifiers for all road users, with a focus on realistic conditions during evaluation.

A Review of Computer Vision Techniques for the Analysis of Urban Traffic

There have been a number of techniques that have demonstrated the generation of multimedia data for one modality at a time using GANs, such as the ability to generate images, videos, and audio. However, so far, the task of multi-modal generation of data, specifically for audio and videos both, has not been sufficiently well-explored. Towards this, we propose a method that demonstrates that we are able to generate naturalistic samples of video and audio data by the joint correlated generation of audio and video modalities. The proposed method uses multiple discriminators to ensure that the audio, video, and the joint output are also indistinguishable from real-world samples. We present a dataset for this task and show that we are able to generate realistic samples. This method is validated using various standard metrics such as Inception Score, Frechet Inception Distance (FID) and through human evaluation.

Collaborative Learning to Generate Audio-Video Jointly

Since its inception in computer graphics community light field has drawn a lot of attention and interest. By densly sampling the plenoptic function light fields present an alternative way to represent and produce a faithful reconstruction of 3D scenes. But acquisition of densely sampled light fields require camera arrays, robotic arms or newly developed plenoptic cameras. The light fields captured using the existential technologies are limited to scenes containing limited complexity. In this paper we propose to use unmanned aerial vehicle for acquisition of larger unstructured aerial light fields. We aim to capture light fields of larger objects and scenes which are not possible by traditional light field acquisition setup. We combine the data from IMU and state estimated using homography with a Kalman filter framework. Frames which gives a minimum error (approximation of free form camera surface to traditional parameterization) are selected as perspective images of light fields. Rendering algorithm is devised to support the unstructured camera surface and to avoid rebinning of image data.

Acquisition of Aerial Light Fields

The prohibitive amounts of time required to review the large amounts of data captured by surveillance and other cameras has brought into question the very utility of large scale video logging. Yet, one recognizes that such logging and analysis are indispensable to security applications. The only way out of this paradox is to devise expedited browsing, by the creation of hyperlapse. We address the hyperlapse problem for the very challenging category of intensive egomotion which makes the hyperlapse highly jerky. We propose an economical approach for trajectory estimation based on Visual Odometry and implement cost functions to penalize pose and path deviations. Also, this is implemented on data taken by omni-directional camera, so that the viewer can opt to observe any direction while browsing. This requires many innovations, including handling the massive radial distortions and implementing scene stabilization that need to be operated upon the least distorted region of the omni view.

Visual Odometry Based Omni-directional Hyperlapse

Tracking a human in a video sequence is an important function of a human-following robot. This paper aims at developing a robust vision-based algorithm using point-based features, like SURF, which can track a human under challenging conditions including variation in illumination, pose change, full or partial occlusion and abrupt camera motion. Since the point-based methods use tracking-by-detection framework, the major problem lies in finding sufficient number of descriptors in subsequent frames as the target undergoes the above mentioned variations. This paper looks into the problem of constructing an object model which can evolve over time to deal with short-term changes while maintaining stability on a longer term. The object model is updated by propagating useful descriptors from past templates onto the current template using affine transformation. An SVM classifier along with a Kalman Filter predictor is used to differentiate between a case of pose change and a case of occlusion (partial/full). An attempt is also made to detect pose change due to out-of-plane rotations which is a difficult problem and lead to frequent tracking failures. The efficacy of the algorithm is demonstrated through various simulation and experimental results.

SURF-based human tracking algorithm for a human-following mobile robot

Shot and scene boundary detection is a primary and an important step towards all the video condensation techniques. In this paper a simple but automatic and efficient scene and shot boundary detection is proposed based on energy minimization approach. A shot differs in its background from another. The background has a temporal continuity throughout the shot, we have invoked this fact in our algorithm. We formulate the extraction of Video shot as an energy minimization problem in which the desire to extract the shot with the common background is represented by an energy term. The energy minimized approach optimizes to minimize the discontinuity in the background edges and color. We have proposed different algorithms for different kinds of shot transitions(cut, fade-in, fade-out, dissolve and wipe) to help us to get a better solution for shot boundary detection problem. We have demonstrated our method with a variety of videos which is capable of accurately detecting shot transitions.

K. S. Venkatesh

Papers

Collaborative Learning to Generate Audio-Video Jointly

Acquisition of Aerial Light Fields

Visual Odometry Based Omni-directional Hyperlapse

SURF-based human tracking algorithm for a human-following mobile robot

An Energy Minimization Approach for Automatic Video Shot and Scene Boundary Detection