A Computational Framework and an Algorithm for the Measurement of Visual

TLDR: This paper describes a hierarchical computational framework for the determination of dense displacement fields from a pair of images, and an algorithm consistent with that framework, based on a scale-based separation of the image intensity information and the process of measuring motion.

Abstract: THE ROBUST MEASUREMENT OF VISUAL MOTION FROM DIGITIZED IMAGE SEQUENCES HAS BEEN AN IMPORTANT BUT DIFFICULT PROBLEM IN COMPUTER VISION. THIS PAPER DESCRIBES A HIERARCHICAL COMPUTATIONAL FRAMEWORK FOR THE DETERMINATION OF DENSE DISPLACEMENT FIELDS FROM A PAIR OF IMAGES, AND AN ALGORITHM CONSIST- ENT WITH THAT FRAMEWORK. OUR FRAMEWORK IS BASED ON THE SEPARATION OF THE IMAGE INTENSITY INFORMATION AS WELL AS THE PROCESS OF MEASURING MOTION ACCORDING TO SCALE. THE LARGE SCALE INTENSITY INFORMATION IS FIRST USED TO OBTAIN ROUGH ESTIMATES OF IMAGE MOTION, WHICH ARE THEN REFINED BY USING INTENSITY INFORMATION AT SMALLER SCALES. THE ESTIMATES ARE IN THE FORM OF DISPLACEMENT (OR VELOCITY) VECTORS FOR PIXELS AND ARE ACCOMPANIED BY A DIRECTION-DEPENDENT CONFIDENCE MEASURE. A SMOOTHNESS CONSTRAINT IS EMPLOYED TO PROPAGATE THE MEASUREMENTS WITH HIGH CONFIDENCE TO THEIR NEIGBORING AREAS WHERE THE CONFIDENCES ARE LOW. AT ALL LEVELS, THE COMPUTATIONS ARE PIXEL-PARALLEL, UNIFORM ACROSS THE IMAGE, AND BASED ON INFORMATION FROM A SMALL NEIGHBORHOOD OF A PIXEL. FOR OUR ALGORITHM, THE LOCAL DISPLACEMENT VECTORS ARE DETERMIND BY MINI- MIZING THE SUM-OF-SQUARED DIFFERENCES (SSD) OF INTENSITIES, THE CONFIDENCE MEASURES ARE DERIVED FROM THE SHAPE OF THE SSD SURFACE, AND THE SMOOTHNESS CONSTRAINT IS CAST IN THE FORM OF ENERGY MINIMIZATION. RESULTS OF APPLYING OUR ALGORITHM TO PAIRS OF REAL IMAGES ARE INCLUDED. IN ADDITION TO OUR OWN