Book ChapterDOI
A naturalistic open source movie for optical flow evaluation
Daniel J. Butler,Jonas Wulff,Garrett B. Stanley,Michael J. Black +3 more
- pp 611-625
Reads0
Chats0
TLDR
A new optical flow data set derived from the open source 3D animated short film Sintel is introduced, which has important features not present in the popular Middlebury flow evaluation: long sequences, large motions, specular reflections, motion blur, defocus blur, and atmospheric effects.Abstract:
Ground truth optical flow is difficult to measure in real scenes with natural motion As a result, optical flow data sets are restricted in terms of size, complexity, and diversity, making optical flow algorithms difficult to train and test on realistic data We introduce a new optical flow data set derived from the open source 3D animated short film Sintel This data set has important features not present in the popular Middlebury flow evaluation: long sequences, large motions, specular reflections, motion blur, defocus blur, and atmospheric effects Because the graphics data that generated the movie is open source, we are able to render scenes under conditions of varying complexity to evaluate where existing flow algorithms fail We evaluate several recent optical flow algorithms and find that current highly-ranked methods on the Middlebury evaluation have difficulty with this more complex data set suggesting further research on optical flow estimation is needed To validate the use of synthetic data, we compare the image- and flow-statistics of Sintel to those of real films and videos and show that they are similar The data set, metrics, and evaluation website are publicly availableread more
Citations
More filters
Proceedings ArticleDOI
FlowNet: Learning Optical Flow with Convolutional Networks
Alexey Dosovitskiy,Philipp Fischery,Eddy Ilg,Philip Häusser,Caner Hazirbas,Vladimir Golkov,Patrick van der Smagt,Daniel Cremers,Thomas Brox +8 more
TL;DR: In this paper, the authors propose and compare two architectures: a generic architecture and another one including a layer that correlates feature vectors at different image locations, and show that networks trained on this unrealistic data still generalize very well to existing datasets such as Sintel and KITTI.
Proceedings ArticleDOI
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
TL;DR: The concept of end-to-end learning of optical flow is advanced and it work really well, and faster variants that allow optical flow computation at up to 140fps with accuracy matching the original FlowNet are presented.
Proceedings ArticleDOI
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
TL;DR: PWC-Net as discussed by the authors uses the current optical flow estimate to warp the CNN features of the second image, which is processed by a CNN to estimate the optical flow, and achieves state-of-the-art performance on the MPI Sintel final pass and KITTI 2015 benchmarks.
Proceedings ArticleDOI
A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation
Nikolaus Mayer,Eddy Ilg,Philip Häusser,Philipp Fischer,Daniel Cremers,Alexey Dosovitskiy,Thomas Brox +6 more
TL;DR: In this article, a large-scale synthetic stereo video dataset is proposed to enable training and evaluation of optical flow estimation with a convolutional network and disparity estimation with CNNs.
Proceedings ArticleDOI
A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation
Federico Perazzi,Jordi Pont-Tuset,Brian McWilliams,L. Van Gool,Markus Gross,Alexander Sorkine-Hornung +5 more
TL;DR: This work presents a new benchmark dataset and evaluation methodology for the area of video object segmentation, named DAVIS (Densely Annotated VIdeo Segmentation), and provides a comprehensive analysis of several state-of-the-art segmentation approaches using three complementary metrics.
References
More filters
Proceedings ArticleDOI
Are we ready for autonomous driving? The KITTI vision benchmark suite
TL;DR: The autonomous driving platform is used to develop novel challenging benchmarks for the tasks of stereo, optical flow, visual odometry/SLAM and 3D object detection, revealing that methods ranking high on established datasets such as Middlebury perform below average when being moved outside the laboratory to the real world.
Journal ArticleDOI
Determining optical flow
TL;DR: In this paper, a method for finding the optical flow pattern is presented which assumes that the apparent velocity of the brightness pattern varies smoothly almost everywhere in the image, and an iterative implementation is shown which successfully computes the Optical Flow for a number of synthetic image sequences.
Proceedings ArticleDOI
Determining Optical Flow
TL;DR: In this article, a method for finding the optical flow pattern is presented which assumes that the apparent velocity of the brightness pattern varies smoothly almost everywhere in the image, and an iterative implementation is shown which successfully computes the Optical Flow for a number of synthetic image sequences.
Journal ArticleDOI
Performance of optical flow techniques
TL;DR: These comparisons are primarily empirical, and concentrate on the accuracy, reliability, and density of the velocity measurements; they show that performance can differ significantly among the techniques the authors implemented.
Journal ArticleDOI
Relations between the statistics of natural images and the response properties of cortical cells.
TL;DR: The results obtained with six natural images suggest that the orientation and the spatial-frequency tuning of mammalian simple cells are well suited for coding the information in such images if the goal of the code is to convert higher-order redundancy into first- order redundancy.