Degraf-Flow: Extending Degraf Features for Accurate and Efficient Sparse-To-Dense Optical Flow Estimation
Summary (2 min read)
1. INTRODUCTION
- Optical flow estimation is the recovery of the motion fields between temporally adjacent images within a sequence.
- Such matching of image points can accurately recover long range motions, however, as the matches are sparse (only cover a fraction of the image) they can lead to loss of accuracy at motion boundaries [3] in the refined dense optical flow.
- This shows improved results compared against previous interpolation methods [19] and is currently used as a popular post-processing step in contemporary state-of-the-art dense optical flow estimation methods [20, 21, 22].
- By contrast, recent work on the use of Dense Gradient Based Features [1] address these above issues.
- This approach provides spatially tunable feature density and uniformity in addition to sub-pixel accuracy.
2. APPROACH
- Dense optical flow is recovered from two temporally adjacent images (Figure 2A) using a three step process: Point detection on the first image is carried out by calculation of an even grid of DeGraF points [1] shown in Figure 2B.
- A sliding window is passed over the image with step size of δ.
- For an image region I of dimensions w × h containing grayscale pixels, two centroids, Cpos and Cneg are defined which define a gradient vector −−−−−−→ CposCneg for the region.
- The key-point in each region is taken as the location of the most stable centroid, i.e if Sneg >.
- The sparse optical flow vectors recovered are shown in Figure 2C.
3. EVALUATION
- Statistical accuracy is measured using the established End-Point Error (EPE) metric [10, 13].
- The authors method (DeGraF / DeGraF-Flow) was implemented in C++ and all experiments run on a Core i7 using four CPU cores.
- All timings reported are for the run-time of the algorithm excluding image input/output and display.
- For RLOF, the global motion and illumination models are used as per [2] with the adaptive cross based support region described in [29].
3.1. Comparison of Feature Detectors
- To justify the use of DeGraF points, the authors compare with established feature detectors for dense flow computation on KITTI 2012 [10].
- As is shown in [26], when using RLOF and interpolation to recover dense optical flow, a uniform grid of points is a superior input compared to other feature point detectors.
- Here the authors repeat the experiment of [26] but with the addition of DeGraF (Table 1).
- To allow meaningful comparison, each detector is tuned to ensure a comparable number of points are detected.
- DeGraF shows the best performing EPE and efficient detection, equal to FAST.
3.2. Benchmark Comparison - KITTI 2012
- Semi dense (50%) ground truth, calculated using a LiDAR, is provided.
- The result of standalone Pyramidal Lucas-Kanade (PLK) [24] is shown as a baseline reference.
- The first two columns give the percentage of estimated flow vectors that have an EPE of more than 3px.
- DeGraF-Flow shows promising results in terms of balancing computational efficiency (run-time, Table 2) and accuracy.
- In Table 2 it ranks thirteenth in accuracy (Out-Noc) and is shown to be the fourth fastest CPU method that has a percentage outlier of non occluded pixels of less than 10%.
3.3. Benchmark Comparison - KITTI 2015
- These exhibit far larger pixel displacements in some areas resulting in lower algorithm performance on KITTI 2015 (Table 3) than on KITTI 2012 (Table 2).
- Table 3 shows the results of DeGraF-Flow on the test set compared to DeepFlow [18] and EpicFlow [3] (which both use DeepMatch as the sparse matching technique).
- The percentage of flow vectors with an EPE greater than 3px are shown, with fg and bg referring to foreground objects and the background scene respectively.
- As with the KITTI 2012 benchmark results their approach shows comparable accuracy with significantly reduced runtime over EpicFlow and DeepFlow.
- In terms of accuracy it places 13th (out of 22) from CPU methods that take less than than 10 seconds to process an image pair.
4. CONCLUSION
- A novel optical flow estimation method.the authors.
- -Flow uses a rapidly computed grid of Dense Gradient based Features and then combines an existing state of the art sparse point tracker (RLOF [2]) and interpolator (EPIC [3]) to recover dense flow.
- With only minimal impact on accuracy (within 2% of EPICFlow [3] and DeepFlow [18] across all metrics) their approach offers significant gains in computational performance for dense optic flow estimation.
- On the KITTI 2012 and 2015 benchmarks [10, 13] their method shows competitive run-time and comparable accuracy with other CPU methods.
Did you find this useful? Give us your feedback
Citations
10 citations
References
46,906 citations
14,708 citations
13,993 citations
"Degraf-Flow: Extending Degraf Featu..." refers background in this paper
...Common feature choices are Harris [27] or FAST key-points [28] due to their relative speed....
[...]
...[27] C. Harris and M. Stephens, “A Combined Corner and Edge Detector,” Procedings Alvey Vis....
[...]
12,944 citations
12,449 citations
Related Papers (5)
Frequently Asked Questions (11)
Q2. What have the authors stated for future works in "Degraf-flow: extending degraf features for accurate and efficient sparse-to-dense optical flow estimation" ?
Future work will exploit the tracking of DeGraF features for applications in an autonomous vehicle setting.
Q3. What is the key to achieving accurate dense optical flow recovery?
sparse flow vectors with uniform spatial coverage is an ideal for accurate dense optical flow recovery [26] making uniform feature distribution across the scene a key conduit to success.
Q4. What data sets are used for the evaluation of dense optical flow?
Evaluation is carried out on the KITTI optic flow estimation benchmark data sets (denoted as KITTI 2012 [10] and KITTI 2015 [13]).
Q5. What is the benchmark for KITTI 2015?
The KITTI 2015 benchmark [13] comprises 200 training and 200 test image pairs (1242 × 375 pixels) with the increasedchallenges of dynamic scene objects (vehicles).
Q6. How is the optical flow recovered from two temporally adjacent images?
Dense optical flow is recovered from two temporally adjacent images (Figure 2A) using a three step process:Point detection on the first image is carried out by calculation of an even grid of DeGraF points [1] shown in Figure 2B.
Q7. What is the definition of dense optical flow?
To cope with such challenges, contemporary optical flow methods use a sparse-to-dense estimation scheme, whereby a sparse set of points on a video frame are matched to points in the subsequent frame.
Q8. Why is the corresponding centroid more robust?
This choice is made because the larger value from Sneg and Spos is less sensitive to noise and so the corresponding centroid is more robust.
Q9. How many pixels are measured in the deGraF-Flow algorithm?
For a predicted optical flow vector up at every pixel with corresponding ground flow truth vector ugt, the EPE is then defined as the average difference between the predicted and ground truth vectors over the image:EPE = 1N ∑ i ‖upi − u gt i ‖ 2, (3)where N is the number of pixels and EPE is hence measured in pixels.
Q10. What is the process of detecting deGraF points in a sequence?
Given two temporally adjacent images in a sequence, DeGraF points are detected in the first image and then efficiently tracked to the subsequent image using RLOF [2].
Q11. What is the percentage of flow vectors with an EPE greater than 3px?
The percentage of flow vectors with an EPE greater than 3px are shown, with fg and bg referring to foreground objects and the background scene respectively.