Patent•
Time-space consistency segmentation method for binocular stereoscopic video
07 Aug 2013-
TL;DR: In this article, a time-space consistency segmentation method for a binocular stereoscopic video is proposed, which comprises the steps of conducting video preprocessing to obtain video clips and a corresponding initial light flow diagram sequence and disparity map sequence, conducting pixel-based segmentation to the video clips to obtain a first-layer segmentation result; according to the first layer segmentation results, smoothing the initial LFD sequence and the disparity map sequences to obtain an optimized light flow diagrams and disparity maps.
Abstract: The invention provides a time-space consistency segmentation method for a binocular stereoscopic video The method comprises the steps of conducting video preprocessing to obtain video clips and a corresponding initial light flow diagram sequence and disparity map sequence; according to the initial light flow diagram sequence and the disparity map sequence, conducting pixel-based segmentation to the video clips to obtain a first-layer segmentation result; according to the first-layer segmentation result, smoothing the initial light flow diagram sequence and the disparity map sequence to obtain an optimized light flow diagram sequence and disparity map sequence; and according to the optimized light flow diagram sequence and disparity map sequence and the first-layer segmentation result, conducting superpixel-based segmentation to the video clips to obtain a multilayer segmentation result By introducing the disparity information of the binocular stereoscopic video and in combination with information such as textures and motion, more semantic time-space consistency segmentation can be obtained, the multilayer segmentation result provides multiple segmentation levels from over-segmentation to sparse segmentation close to semantic expression and bases are provided for different later-stage video processing and application
Citations
More filters
Patent•
28 Jan 2015
TL;DR: In this article, a method for detecting vehicle motion information based on integration of binocular stereoscopic vision and optical flow is proposed, which comprises the following steps of: analyzing and processing left and right image sequences taken by a binocular stereo camera, and acquiring motion information of the three-dimensional space velocity and angle velocity of the car body through feature matching, 3D reconstruction, feature extraction, and estimation of optical flow.
Abstract: The invention relates to a method for detecting vehicle motion information based on integration of binocular stereoscopic vision and optical flow, and the method comprises the following steps of: analyzing and processing left and right image sequences taken by a binocular stereoscopic camera, and acquiring motion information of the three-dimensional space velocity and angle velocity of the car body through feature matching, three-dimensional reconstruction, feature extraction and optical flow estimation, in order to estimate motion information of the car body As a new method for detecting car motion parameters, it is simply installed, applied to the actual operating environment of the car, high in precision and strong in real time By the method, inaccurate reading numbers and measuring errors occurred by environmental factors can be effectively reduced, and the visual sensor has the advantages of simple structure and much information and can provide a mass of other information The system selects ground points as interested feature points to effectively reduce interference of the motion object and improve estimation precision At the same time, by the method, steadiness and robustness of the algorithm can be enhanced by fully utilizing depth information
16 citations
Patent•
06 Apr 2018
TL;DR: In this paper, an integrated correction method based on stereo vision and a low-beam lidar and applied to unmanned driving is proposed. But the method is limited to the case of a single camera and a single Lidar, and it only needs to substitute a parallax image acquired by the binocular vision into the error compensation function.
Abstract: The invention relates to an integrated correction method based on stereo vision and a low-beam lidar and applied to unmanned driving. According to the method, a binocular camera and the lidar are registered in air domain and time sequence; the binocular camera, aiming at a target, acquires an image, and at the same time, the lidar emits a beam to the target and acquires data; the data of the lidarare converted into a parallax image which is used for correcting the parallax error of the binocular vision; an error compensation function is obtained based on the distribution of the parallax error; in the subsequent process, it only needs to substitute a parallax image acquired by the binocular vision into the error compensation function, so that a corrected parallax image with extremely smallparallax error can be obtained; since the parallax of the binocular vision is corrected, visual accuracy is improved; a parallax image which has been subjected to semantic segmentation and a compensated full-pixel parallax image, both adopted as input quantities, are inputted to a deep learning network; finally a parallax image which has been trained by the deep learning network training is obtained; and therefore, the precision of binocular vision in the unmanned driving can be further improved.
8 citations
Patent•
18 May 2016
TL;DR: In this article, a motion estimation method and apparatus is presented, which consists of an initial light stream of each frame of image in an inputted image sequence, and according to color information, each frame is divided into at least one segment; initial motion models of all segments in each frame in each image are obtained; motion segmentation time-sequence matching is carried out on initial motion segments of each segment in each video frame, so that the same object in the image sequence has the same mark in each time frame in the video sequence.
Abstract: The invention, which belongs to the technical field of the digital signal processing, discloses a motion estimation method and apparatus. The method comprises: an initial light stream of each frame of image in an inputted image sequence is obtained; according to color information, each frame of image is divided into at least one segment; initial motion models of all segments in each frame of image are obtained; motion segmentation time-sequence matching is carried out on initial motion models of all segments in each frame of image, so that the same object in the image sequence has the same mark in each frame in the image sequence; and according to the time-sequence matching result, a motion segment and an output light stream are obtained. According to the invention, the motion segment and motion estimation of a long image sequence can be processed and the motion segment and motion estimation results with good time-space coherence can be obtained, so that the time-space coherence of the motion segments can be improved effectively.
5 citations
Patent•
03 Dec 2014
TL;DR: In this article, a field moving object fining method is proposed, which comprises the following steps that: a sequence image including a moving object is obtained, and the sequence image is preprocessed; the pre-processed sequence image are subjected to frame-by-frame differentiation, and is then segmented into a plurality of grids, a located moving region of the object is determined according to feature values of the grids, and a grid method is used for extracting a moving region for further reducing an object range.
Abstract: The invention relates to a field moving object fining extraction method, which comprises the following steps that: a sequence image including a moving object is obtained, and the sequence image is preprocessed; the preprocessed sequence image is subjected to frame-by-frame differentiation, and is then segmented into a plurality of grids, a located moving region of the object is determined according to feature values of the grids, and a grid method is used for extracting a moving region of the object for further reducing an object range; the background is subjected to modeling in the moving region of the object, a binaryzation chart of the object is obtained through a background subtraction method, and the binaryzation chart is subjected to feedback pixel processing; the processed binaryzation chart is mapped to a located color chart region of the object, and the color chart region is subjected to super pixel segmentation; and the segmentation result is fused with the binaryzation chart, the confidence degree of each super pixel is calculated according to the fusion result, and after the thresholding, the fining moving object is finally obtained. The field moving object fining extraction method can realize the real-time robust and fine field moving object extraction under complicated backgrounds.
4 citations
Patent•
19 Jan 2018
TL;DR: In this article, a binocular stereoscopic visual system-based foreground segmentation method was proposed, which comprises the following steps of: obtaining a left view and a right view of a same object by utilizing binocular stereo-vision visual system; separating the foreground and background of the left view; defining an energy equation; and endowing foreground pixel points in the foregrounds and background pixels in the background with different labels by utilizing the energy equation so as to separate the foreground's and the backgrounds.
Abstract: The invention discloses a binocular stereoscopic visual system-based foreground segmentation method. The method comprises the following steps of: obtaining a left view and a right view of a same object by utilizing a binocular stereoscopic visual system; separating the foreground and background of the left view and separating the foreground and background of the right view; defining an energy equation; and endowing foreground pixel points in the foregrounds and background pixel points in the backgrounds with different labels by utilizing the energy equation so as to separate the foregrounds and the backgrounds. The method has the beneficial effects that foreground pixel points and the background pixel points in a picture are endowed with different labels through defining the energy equation, so that the foreground pixel points and the background pixel points can be correctly separated and then the foreground segmentation correctness is improved; and according to the method, the foreground segmentation of binocular color images can be realized, so that the method is wide in application range.
2 citations
References
More filters
Patent•
14 Jun 2004TL;DR: In this article, a system for estimating orientation of a target based on real-time video data using depth data included in the video to determine the estimated orientation is presented, which includes a time-of-flight camera capable of depth sensing within a depth window.
Abstract: A system for estimating orientation of a target based on real-time video data uses depth data included in the video to determine the estimated orientation. The system includes a time-of-flight camera capable of depth sensing within a depth window. The camera outputs hybrid image data (color and depth). Segmentation is performed to determine the location of the target within the image. Tracking is used to follow the target location from frame to frame. During a training mode, a target-specific training image set is collected with a corresponding orientation associated with each frame. During an estimation mode, a classifier compares new images with the stored training set to determine an estimated orientation. A motion estimation approach uses an accumulated rotation/translation parameter calculation based on optical flow and depth constrains. The parameters are reset to a reference value each time the image corresponds to a dominant orientation.
327 citations
09 Jan 2012
TL;DR: A novel real-time superpixel segmentation algorithm is presented which usesreal-time stereo and realtime optical flow to obtain a more accurate and at the same time faster segmentation by incorporating color, depth and optical flow.
Abstract: The use of depth is becoming increasingly popular in real-time computer vision applications. However, when using real-time stereo for depth, the quality of the disparity image is usually insufficient for reliable segmentation. The aim of this paper is to obtain a more accurate and at the same time faster segmentation by incorporating color, depth and optical flow. A novel real-time superpixel segmentation algorithm is presented which uses real-time stereo and realtime optical flow. The presented system provides superpixels which represent suggested object boundaries based on color, depth and motion. Each outputted superpixel has a 3D location and a motion vector, and thus allows for straightforward segmentation of objects by 3D position and by motion direction. In particular, it enables reliable segmentation of persons, and of moving hands or arms. We show that our method is competitive with the state of the art while approaching real-time performance.
26 citations
Patent•
30 Jun 2010
TL;DR: Wang et al. as discussed by the authors proposed a depth representation method based on light stream and image segmentation, which comprises the following steps: (1) for each frame image in an original 2D video, referring an adjacent latter 2D image in the time direction, carrying out light stream analysis, and obtaining a light stream map of the current image; (2) carrying out image segmentations on each frame in the original two-dimensional video to obtain a segmentation map.
Abstract: The invention discloses a depth representing method based on light stream and image segmentation, which comprises the following steps: (1) for each frame image in an original two-dimensional video, referring an adjacent latter frame image in the time direction, carrying out light stream analysis, and obtaining a light stream map of the current image; (2) carrying out image segmentation on each frame image in the original two-dimensional video to obtain a segmentation map; and (3) combining the light stream map and the segmentation map of each frame image in the original two-dimensional video to obtain the depth map for representing the three-dimensional video. The invention extracts the motion information of the two-dimensional video by a light stream analyzing method, is more accurate compared with a window matching method, simultaneously combines the image segmentation method to generate the depth map for representing the three-dimensional video, effectively smoothes the noise, and modifies the edge contour of an object.
16 citations
Patent•
24 Oct 2012
TL;DR: In this article, a binocular video depth map obtaining method based on image segmentation and motion estimation is presented. But the method requires the use of a stereo matching algorithm to calculate the depth map.
Abstract: The invention discloses a binocular video depth map obtaining method based on image segmentation and motion estimation. The method includes the following steps: respectively performing single-frame image segmentation for two images; performing depth calculating by aid of the stereo matching algorithm based on image segmentation; and rectifying depth extracting results by aid of motion estimation. A depth map extracted in the binocular video depth map obtaining method based on image segmentation and motion estimation is accurate in edges, an ideal effect can also be obtained on shielded portions, accuracy and timeliness are both considered, and the distance relation between scene objects is truly reflected.
16 citations
Patent•
27 Apr 2011
TL;DR: In this paper, the authors proposed a method to estimate the motion information and depth information of a main motion target using a gradient descent method and evolving a break curve by using a level set method.
Abstract: The invention discloses a motion segmentation and three-dimensional (3D) expression method for a single view image sequence. The method comprises the following steps of: acquiring the single view image sequence, estimating the motion information and depth information of a main motion target by using a gradient descent method and evolving a break curve by using a level set method; verifying the estimated depth information and correcting unreliable depth information; minimizing an energy function by using the corrected depth information and the obtained break curve and estimating the motion information and depth information of the main motion target; and fixing the break curve, minimizing the energy function once again, verifying and correcting the reliability of the depth information and minimizing the energy function once again so as to obtain the depth information and motion information of every motion target. Motion segmentation and 3D expression can be performed at the same time and the number of targets in the image sequence does not need to be known in advance. The method has wide applicability.
4 citations