Apparatus and methods are disclosed for modifying video based on user input and or face detection data received with a mobile device to generate foreground regions (e.g., to separate a user image from background in the video). According to one disclosed embodiment, a method comprises receiving user input and/or face regions generated with a mobile device, producing an initial representation for segmenting input video into a plurality of portions based on the user input, where the initial representation includes probabilities for one or more regions of the input video being designated as foreground regions or background regions. Based on the initial representation, input video is segmented by designating one or more of the regions of the input video as foreground regions or background regions.

Modifying video regions using mobile device input

A system for background image subtraction includes a computing device coupled with a 3D video camera, a processor of the device programmed to receive a video feed from the camera containing images of one or more subject that include depth information. The processor, for an image: segments pixels and corresponding depth information into three different regions including foreground (FG), background (BG), and unclear (UC); categorizes UC pixels as FG or BG using a function that considers the color and background history (BGH) information associated with the UC pixels and the color and BGH information associated with pixels near the UC pixels; examines the pixels marked as FG and applies temporal and spatial filters to smooth boundaries of the FG regions; constructs a new image by overlaying the FG regions on top of a new background; displays a video feed of the new image in a display device; and continually maintains the BGH.

System for background subtraction with 3D camera

Aspects of a method and system for change detection in localization and tracking of objects in a smart video camera are provided. A programmable surveillance video camera comprises processors for detecting objects in a video signal based on an object mask. The processors may generate a textual representation of the video signal by utilizing a description language to indicate characteristics of the detected objects, such as shape, texture, color, and/or motion, for example. The object mask may be based on a detection field value generated for each pixel in the video signal by comparing a first observation field and a second observation field associated with each of the pixels. The first observation field may be based on a difference between an input video signal value and an estimated background value while the second observation field may be based on a temporal difference between first observation fields.

Method and system for background estimation in localization and tracking of objects in a smart video camera

A method of image processing, comprising receiving at least one video frame of a video sequence, the at least one video frame comprising at least one foreground subject and a background, and processing the at least one video frame so as to separate said at least one foreground subject from said background. The processing includes: generating a pixel mask indicating whether a pixel of the at least one video frame belongs to the foreground subject or to the background, applying morphological closing to the pixel mask, wherein the applying morphological closing comprises, for each pixel of the pixel mask, conditioning a pixel value in the mask to values of neighboring pixels. The conditioning comprises: determining at least edges of the at least one foreground subject in the at least one video frame; and, for the generic pixel under processing, determining the neighboring pixels based on the determined edges.

Edge-guided morphological closing in segmentation of video sequences

A method of image processing includes: receiving at least one video frame of a video sequence, the at least one video frame including at least one foreground subject and a background; and processing the at least one video frame so as to separate the at least one foreground subject from the background. The processing includes: generating a pixel mask containing information indicating, for each pixel of the at least one video frame, whether the pixel belongs to the foreground subject or to the background, and determining contours of the at least one foreground subject on the pixel mask. The determining of the contours includes for each pixel in the at least one video frame; based on the information included in the pixel mask, determining whether at least one pixel border belongs to a contour of the at least one foreground subject, the at least one pixel border separating the pixel from a respective at least one adjacent pixel.

Contour finding in segmentation of video sequences

A method of image processing, comprising: receiving at least one video frame of a video sequence, the at least one video frame comprising at least one foreground subject and a background; and processing the at least one video frame so as to separate the at least one foreground subject from said background. The processing includes: obtaining a reference image including the background; comparing the at least one video frame to the reference image; and generating a pixel mask as a result of the comparison, the pixel mask indicating whether a pixel of the at least one video frame belongs to the foreground subject or to the background. The method further comprises at least partially determining edges of the at least one foreground subject in the at least one video frame, and modifying the pixel mask based on the determined foreground subject edges.

Edge comparison in segmentation of video sequences

A method of image processing, comprises: receiving at least one video frame of a video sequence, the at least one video frame comprising at least one foreground subject and a background; and processing the at least one video frame so as to separate the at least one foreground subject from said background. The processing includes: obtaining a reference image including the background; comparing theat least one video frame to the reference image; and generating a pixel mask as a result of the comparison, the pixel mask indicating whether a pixel of the at least one video frame belongs to the foreground subject or to the background. The method further comprises at least partially determining edges of the at least one foreground subject in the at least one video frame, and modifying the pixelmask based on the determined foreground subject edges.

Andrea Varesio

Papers

Edge comparison in segmentation of video sequences

Edge comparison in video sequence partition