In this paper, we suggest a variational model for optic flow computation based on non-linearised and higher order constancy assumptions. Besides the common grey value constancy assumption, also gradient constancy, as well as the constancy of the Hessian and the Laplacian are proposed. Since the model strictly refrains from a linearisation of these assumptions, it is also capable to deal with large displacements. For the minimisation of the rather complex energy functional, we present an efficient numerical scheme employing two nested fixed point iterations. Following a coarse-to-fine strategy it turns out that there is a theoretical foundation of so-called warping techniques hitherto justified only on an experimental basis. Since our algorithm consists of the integration of various concepts, ranging from different constancy assumptions to numerical implementation issues, a detailed account of the effect of each of these concepts is included in the experimental section. The superior performance of the proposed method shows up by significantly smaller estimation errors when compared to previous techniques. Further experiments also confirm excellent robustness under noise and insensitivity to parameter variations.

Highly accurate optic flow computation with theoretically justified warping

: Supervoxel segmentation has strong potential to be incorporated into early video analysis as superpixel segmentation has in image analysis. However, there are many plausible supervoxel methods and little understanding as to when and where each is most appropriate. Indeed, we are not aware of a single comparative study on supervoxel segmentation. To that end, we study five supervoxel algorithms in the context of what we consider to be a good supervoxel: namely, spatiotemporal uniformity, object/region boundary detection, region compression and parsimony. For the evaluation we propose a comprehensive suite of 3D volumetric quality metrics to measure these desirable supervoxel characteristics. We use three benchmark video data sets with a variety of content-types and varying amounts of human annotations. Our findings have led us to conclusive evidence that the hierarchical graph-based and segmentation by weighted aggregation methods perform best and almost equally-well on nearly all the metrics and are the methods of choice given our proposed assumptions.

Evaluation of Super Voxel Methods for Early Video Processing (Author's Manuscript)

We segment moving objects in videos by ranking spatio-temporal segment proposals according to “moving objectness”; how likely they are to contain a moving object. In each video frame, we compute segment proposals using multiple figure-ground segmentations on per frame motion boundaries. We rank them with a Moving Objectness Detector trained on image and motion fields to detect moving objects and discard over/under segmentations or background parts of the scene. We extend the top ranked segments into spatio-temporal tubes using random walkers on motion affinities of dense point trajectories. Our final tube ranking consistently outperforms previous segmentation methods in the two largest video segmentation benchmarks currently available, for any number of proposals. Further, our per frame moving object proposals increase the detection rate up to 7% over previous state-of-the-art static proposal methods.

/pdf/learning-to-segment-moving-objects-in-videos-fqxt213vww.pdf

Learning to segment moving objects in videos

We segment moving objects in videos by ranking spatio-temporal segment proposals according to "moving objectness": how likely they are to contain a moving object. In each video frame, we compute segment proposals using multiple figure-ground segmentations on per frame motion boundaries. We rank them with a Moving Objectness Detector trained on image and motion fields to detect moving objects and discard over/under segmentations or background parts of the scene. We extend the top ranked segments into spatio-temporal tubes using random walkers on motion affinities of dense point trajectories. Our final tube ranking consistently outperforms previous segmentation methods in the two largest video segmentation benchmarks currently available, for any number of proposals. Further, our per frame moving object proposals increase the detection rate up to 7\% over previous state-of-the-art static proposal methods.

Learning to Segment Moving Objects in Videos

We present an unsupervised approach that generates a diverse, ranked set of bounding box and segmentation video object proposals—spatio-temporal tubes that localize the foreground objects—in an unannotated video. In contrast to previous unsupervised methods that either track regions initialized in an arbitrary frame or train a fixed model over a cluster of regions, we instead discover a set of easy-togroup instances of an object and then iteratively update its appearance model to gradually detect harder instances in temporally-adjacent frames. Our method first generates a set of spatio-temporal bounding box proposals, and then refines them to obtain pixel-wise segmentation proposals. We demonstrate state-of-the-art segmentation results on the SegTrack v2 dataset, and bounding box tracking results that perform competitively to state-of-the-art supervised tracking methods.

/pdf/track-and-segment-an-iterative-unsupervised-approach-for-3h3pi0hmb9.pdf

Track and Segment: An Iterative Unsupervised Approach for Video Object Proposals

As early stage of video processing, we introduce an iterative trajectory merging algorithm that produces a region-based and hierarchical representation of the video sequence, called the Trajectory Binary Partition Tree (BPT). From this representation, many analysis and graph cut techniques can be used to extract partitions or objects that are useful in the context of specific applications. In order to define trajectories and to create a precise merging algorithm, color and motion cues have to be used. Both types of informations are very useful to characterize objects but present strong differences of behavior in the spatial and the temporal dimensions. On the one hand, scenes and objects are rich in their spatial color distributions, but these distributions are rather stable over time. Object motion, on the other hand, presents simple structures and low spatial variability but may change from frame to frame. The proposed algorithm takes into account this key difference and relies on different models and associated metrics to deal with color and motion information. We show that the proposed algorithm outperforms existing hierarchical video segmentation algorithms and provides more stable and precise regions.

/pdf/hierarchical-video-representation-with-trajectory-binary-1h747rbzmk.pdf

Hierarchical Video Representation with Trajectory Binary Partition Tree

This paper proposes a system that relates objects in an image using occlusion cues and arranges them according to depth. The system does not rely on a priori knowledge of the scene structure and focuses on detecting special points, such as T-junctions and highly convex contours, to infer the depth relationships between objects in the scene. The system makes extensive use of the binary partition tree as hierarchical region-based image representation jointly with a new approach for candidate T-junction estimation. Since some regions may not involve T-junctions, occlusion is also detected by examining convex shapes on region boundaries. Combining T-junctions and convexity leads to a system which only relies on low level depth cues and does not rely on semantic information. However, it shows a similar or better performance with the state-of-the-art while not assuming any type of scene. As an extension of the automatic depth ordering system, a semi-automatic approach is also proposed. If the user provides the depth order for a subset of regions in the image, the system is able to easily integrate this user information to the final depth order for the complete image. For some applications, user interaction can naturally be integrated, improving the quality of the automatically generated depth map.

/pdf/monocular-depth-ordering-using-t-junctions-and-convexity-1mu0znqnmm.pdf

Monocular Depth Ordering Using T-Junctions and Convexity Occlusion Cues

This paper proposes a system to relate objects in an image using occlusion cues and arrange them according to depth. The system does not rely on any a priori knowledge of the scene structure and focuses on detecting specific points, such as T-junctions, to infer the depth relationships between objects in the scene. The system makes extensive use of the Binary Partition Tree (BPT) as the segmentation tool jointly with a new approach for T-junction estimation. Following a bottomup strategy, regions (initially individual pixels) are iteratively merged until only one region is left. At each merging step, the system estimates the probability of observing a T-junction which is a cue of occlusion when three regions meet. When the BPT is constructed and the pruning is performed, this information is used for depth ordering. Although the proposed system only relies on one low-level depth cue and does not involve any learning process, it shows similar performances than the state of the art.

Occlusion-based depth ordering on monocular images with Binary Partition Tree

Performance of multiple-input multiple-output (MIMO) orthogonal frequency division multiplexing (OFDM) is analyzed on an experimental shallow water acoustic channel. Different modulation levels, numbers of subcarriers and transmitters were tested over a period of two weeks. The objectives in doing so were (a) to assess the effect of environmental conditions on the system performance, (b) to determine the performance limits and the data rate supported by the existing detection methods, and (c) to investigate the possibility to push these limits by employing methods for inter-carrier interference (ICI) compensation.

/pdf/underwater-acoustic-mimo-ofdm-an-experimental-analysis-2nedjbp9m1.pdf

Underwater acoustic MIMO OFDM: An experimental analysis

In this paper, we propose a system to obtain a depth ordered segmentation of a single image based on low level cues. The algorithm first constructs a hierarchical, region-based image representation of the image using a Binary Partition Tree (BPT). During the building process, T-junction depth cues are detected, along with high convex boundaries. When the BPT is built, a suitable segmentation is found and a global depth ordering is found using a probabilistic framework. Results are compared with state of the art depth ordering and figure/ground labeling systems. The advantage of the proposed approach compared to systems based on a training procedure is the lack of assumptions about the scene content. Moreover, it is shown that the system outperforms previously low-level cue based systems, while offering similar results to a priori trained figure/ground labeling algorithms.

/pdf/from-local-occlusion-cues-to-global-monocular-depth-220e01c5v9.pdf

Guillem Palou

Papers

Hierarchical Video Representation with Trajectory Binary Partition Tree

Monocular Depth Ordering Using T-Junctions and Convexity Occlusion Cues

Occlusion-based depth ordering on monocular images with Binary Partition Tree

Underwater acoustic MIMO OFDM: An experimental analysis

From local occlusion cues to global monocular depth estimation