Showing papers on "Residual frame published in 2017"

Moving object detection based on frame difference and W4

Abstract: Moving object detection is a basic and important task on automated video surveillance systems, because it gives the focus of attention for further examination. Frame differencing and W4 algorithm can be individually employed to detect the moving objects. However, the detected results of the individual approach are not accurate due to foreground aperture and ghosting problems. We propose an approach to segment the moving objects using both the frame differencing and W4 algorithm to overcome the above problems. Here first we compute the difference between consecutive frames using histogram-based frame differencing technique, next W4 algorithm is applied on frame sequences, and subsequently, the outcomes of the frame differencing and W4 algorithm are combined using logical ‘OR’ operation. Finally, morphological operation with connected component labeling is employed to detect the moving objects. The experimental results and performance evaluation on real video datasets demonstrate the effectiveness of our approach in comparison with existing techniques.

A C3D-Based Convolutional Neural Network for Frame Dropping Detection in a Single Video Shot

Abstract: Frame dropping is a type of video manipulation where consecutive frames are deleted to omit content from the original video. Automatically detecting dropped frames across a large archive of videos while maintaining a low false alarm rate is a challenging task in digital video forensics. We propose a new approach for forensic analysis by exploiting the local spatio-temporal relationships within a portion of a video to robustly detect frame removals. In this paper, we propose to adapt the Convolutional 3D Neural Network (C3D) for frame drop detection. In order to further suppress the errors due by the network, we produce a refined video-level confidence score and demonstrate that it is superior to the raw output scores from the network. We conduct experiments on two challenging video datasets containing rapid camera motion and zoom changes. The experimental results clearly demonstrate the efficacy of the proposed approach.

A Fast and Robust Key Frame Extraction Method for Video Copyright Protection

Abstract: The paper proposes a key frame extraction method for video copyright protection. The fast and robust method is based on frame difference with low level features, including color feature and structure feature. A two-stage method is used to extract accurate key frames to cover the content for the whole video sequence. Firstly, an alternative sequence is got based on color characteristic difference between adjacent frames from original sequence. Secondly, by analyzing structural characteristic difference between adjacent frames from the alternative sequence, the final key frame sequence is obtained. And then, an optimization step is added based on the number of final key frames in order to ensure the effectiveness of key frame extraction. Compared with the previous methods, the proposed method has advantage in computation complexity and robustness on several video formats, video resolution, and so on.

Detecting video frame rate up-conversion based on frame-level analysis of average texture variation

Abstract: Frame rate up-conversion (FRUC) refers to frame interpolation between adjacent video frames to increase the motion continuity of low frame rate video, which can improve the visual quality on hand-held displays. However, FRUC can also be used for video forgery purposes such as splicing two videos with different frame-rates. We found that most FRUC approaches introduce visual artifacts into texture regions of interpolated frames. Based on this observation, a two-stage blind detection approach is proposed for video FRUC based on the frame-level analysis of average texture variation (ATV). First, the ATV value is computed for each frame to obtain an ATV curve of candidate video. Second, the ATV curve is further processed to highlight its periodic property, which indicates the existence of FRUC operation and further estimates the original frame rate. Thus, the positions of interpolated frames can be inferred as well. Extensive experimental results show that the proposed forensics approach is efficient and effective for the detection of existing typical FRUC approaches such as linear frame averaging and motion-compensated interpolation (MCI). The detection performance is superior to the existing approaches in terms of time efficiency and detection accuracy.

Apparatus of Inter Prediction for Spherical Images and Cubic Images

Abstract: Methods and apparatus of video encoding and decoding for a spherical image sequence and a cubic image sequence using circular Inter prediction are disclosed. For the spherical image sequence, the search window includes an area outside or crossing a vertical frame boundary of the reference frame for at least one block of the current spherical image to be encoded. Candidate reference blocks within the search window are determined, where if a given candidate reference block is outside or crossing one vertical frame boundary, the reference pixels are accessed circularly from the reference frame in a horizontal direction crossing the vertical frame boundary of the reference frame. For the cubic image sequence, circular edges of the cubic frame are determined. The search window includes an area outside or crossing a circular edge of the reference frame for at least one block of the current cubic frame to be encoded.

A block-based background model for moving object detection

Abstract: Detecting the moving objects in a video sequence using a stationary camera is an important task for many computer vision applications. This paper proposes a background subtraction approach. As first step, the background is initialized using the block-based analysis before being updated in each incoming frame. Our background frame is generated by collecting the blocks background candidates. The block candidate selection is based on probability density function (pdf) computation. After that, the absolute difference between the background frame and each frame of sequence is computed. A noise filter is applied using the Structure/Texture decomposition in order to minimize the noise caused by background subtraction operation. The binary motion mask is formed using an adaptive threshold that was deduced from the weighted mean and variance calculation. To assure the correspondence between the current frame and the background frame, an adaptation of background model in each incoming frame is realized. After comparing results obtained from the proposed method to other existing ones, it was shown that our approach attains a higher degree of efficacy

Real-time video encoder rate control using dynamic resolution switching

Abstract: Dynamic resolution switching achieves a target bitrate for single-pass and two-pass encoding of a video stream. A single-pass encoder determines whether an encoding bitrate for a frame meets a target bitrate. If not, a quantization parameter used to encode the frame is compared against minimum and maximum threshold values to determine whether the video resolution needs to be adjusted. A two-pass encoder encodes an input frame using a quantization parameter and determines whether video resolution for encoding the frame during a second pass may be adjusted based on the bitrate at which the input frame is encoded. The resolution may be adjusted based on encoder limitations with respect to a motion search area used to code a frame.

A region-based intra-frame rate control scheme by jointing inter-frame dependency and inter-frame correlation

Abstract: During I frame switching, the subjective quality between I frame and P frames usually have obvious fluctuation due to different coding methods. The periodic temporal visual fluctuation will cause video flicker. According to extensive experiments, we observe that I frame flicker possess a strong regional characteristics and different region have different degree of flicker. Based on this observation, a region-based I frame rate control scheme is proposed to suppress I frame flicker according to the different characteristics of the moving and non-moving regions. Firstly, by jointly considering the inter-frame dependency between I frame and subsequent un-encoded P frames and the inter-frame correlation between I frame and previous encoded P frame, an optimization model is proposed to achieve the optimal QPs for different regions. Secondly, a region-based inter-frame dependency model is proposed to separately describe the inter-frame dependency of different regions, which can accurately describe their description of the inter-frame dependency. The experimental results demonstrate that the proposed scheme can efficiently suppress I frame flicker and maintain the smoothness of subjective quality. Moreover, the proposed scheme can achieve a PSNR gain by 0.26 dB on average when compared with the rate control scheme adopted by the HEVC reference software HM15.0.

An efficient non-selective adaptive motion compensated frame rate up conversion

Abstract: Motion-compensated frame rate up-conversion (MC-FRUC) increases the frame rate of a video by interpolating a new frame from temporally consecutive frames. MC-FRUC consists of two key operations: motion estimation and motion compensated frame interpolation. Motion compensation along unidirectional motion trajectories directly causes overlaps and holes issues. To solve these issues, this paper presents a new algorithm for bidirectional motion compensated frame interpolation. Firstly, the proposed method generates a bidirectional motion vector field from two unidirectional motion vector fields (forward and backward) obtained from the motion estimation. It is done by projecting the forward and backward motion vectors into the interpolated frame. A comprehensive metric as an extension of the distance between a projected block and an interpolated block is proposed to compute weighted coefficients in case the interpolated block has multiple projected ones. Holes are filled based on vector median filter of non-hole available neighbor blocks. The proposed method outperforms existing MC-FRUC methods and removes block artifacts significantly.

Motion-compensated frame interpolation using patch-based sparseland model

Abstract: Using patch-based sparseland model, we design a novel Motion-Compensated Frame Interpolation (MCFI) method. Optical-flow estimation is first adopted to generate a reliable Motion Vector Field (MVF) from the previous frame to the following frame. Then we use patch-based bidirectional motion estimation to generate a smooth Motion Vector Felid (MVF). In the last step, we extract patches from reference frames along the motion trajectories, and perform Principle Component Analysis (PCA) to generate dictionaries that code the patches with various local structures. The sparseland model becomes the prior knowledge of the intermediate frame to fuse into the Motion Compensated Interpolation (MCI) by maximum a posteriori (MAP) criterion. By iterative numerical computing, we solve this sparseland-prior reconstruction model, and acquire a sparsity-preferred interpolated frame. Experimental results show that our method outperforms the existing algorithms in both objective and subjective picture qualities, but it also introduces a high computational complexity in the meantime. We design a patch-based Bidirectional Motion Estimation (BME) to assign a unique Motion Vector (MV) for each patch by using optical-flow ME.We construct the sparse model of the intermediate frame according to the MV of each pixel output by the BME module. This sparseland priori is formulated as a Maximum a Posteriori (MAP) estimation problem under the Bayesian framework, and this MAP estimation problem becomes the non-linear sparseland-prior reconstruction model.

Parameterization of Excitation Signal for Improving the Quality of HMM-Based Speech Synthesis System

Abstract: This paper proposes a new approach of parameterizing the excitation signal for improving the quality of HMM-based speech synthesis system. The proposed method tries to model the excitation or residual signal by segregating the regions of the residual signal based on their perceptual importance. Initially, a study on the characteristics of the residual signal around glottal closure instant (GCI) is performed using principal component analysis (PCA). Based on the present study, and from the previous literature (Adiga and Prasanna in Proceedings of Interspeech, pp 1677–1681, 2013; Cabral in Proceedings of Interspeech, pp 1082–1086, 2013), it is concluded that the segment of the residual signal around GCI which carries perceptually important information is considered as the deterministic component and the remaining part of the residual signal is considered as the noise component. The deterministic component is compactly represented using PCA coefficients (with about 95% accuracy), and the noise component is parameterized in terms of spectral and amplitude envelopes. The proposed excitation modeling approach is incorporated in the HMM-based speech synthesis system. Subjective evaluation results show a significant improvement of quality for both female and male speakers’ speech synthesized by the proposed method, compared to three existing excitation modeling methods. Accurate parameterization of the segment of the residual signal around GCI resulted in the improvement of the quality of the synthesized speech. Synthesized speech samples of the proposed and existing source models are made available online at http://www.sit.iitkgp.ernet.in/~ksrao/parametric-hts/pcd-hts.html .

Method and device for generating binary descriptors in video frames

Abstract: A method generates a binary descriptor associated with a given point in a current frame of a succession of video frames obtained by an apparatus such as an image sensor. The method includes determining a pattern of points pairs around said given point in the current frame, and performing intensity comparison processing between the two points of each pair. The apparatus is likely to move in a rotation between the previous frame and the current frame. The method includes processing the pattern of points of the current frame with tridimensional rotation information representative of the apparatus rotation between the previous frame and the current frame and obtained from inertial measurements provided by at least one inertial sensor.

Low-complexity method for generating synthetic reference frames in video coding

Abstract: Presented herein are techniques for a low-complexity process of generating an artificial frame that can be used for prediction. At least a first reference frame and a second reference frame of a video signal are obtained. A synthetic reference frame is generated from the first reference frame and the second reference frame. Reference blocks from each of the first reference frame and the second reference frame are combined to derive an interpolated block of the synthetic reference frame.

Picture coding method, picture decoding method and apparatuses thereof

Abstract: A frame coding unit generates individual picture signal information from a moving picture signal, and outputs a frame code value which is a numeric value to be obtained as a result of coding picture signal information of each frame. A variable length coding unit decomposes the frame code value into unit frame code values which are basic units of coding, converts the unit frame code values into frame code words using only a single code table, and constructs a frame stream by combining the converted frame code words. A multiplexing unit multiplexes a header stream which is constructed through a method equivalent to a conventional method and the frame stream, and constructs a picture coded signal.

Backward-compatible HDR codecs with temporal scalability

Abstract: A processor for video coding receives a full-frame rate (FFR) HDR video signal and a corresponding FFR SDR video signal. An encoder generates a scalable bitstream that allows decoders to generate half-frame-rate (HFR) SDR, FFR SDR, HFR HDR, or FFR HDR signals. Given odd and even frames of the input FFR SDR signal, the scalable bitstream combines a base layer of coded even SDR frames with an enhancement layer of coded packed frames, where each packed frame includes a downscaled odd SDR frame, a downscaled even HDR residual frame, and a downscaled odd HDR residual frame. In an alternative implementation, the scalable bitstream combines four signals layers: a base layer of even SDR frames, an enhancement layer of odd SDR frames, a base layer of even HDR residual frames and an enhancement layer of odd HDR residual frames. Corresponding decoder architectures are also presented.

Error Correction of Quantum Reference Frame Information

Abstract: The existence of quantum error correcting codes is one of the most counterintuitive and potentially technologically important discoveries of quantum information theory. However, standard error correction refers to abstract quantum information, i.e., information that is independent of the physical incarnation of the systems used for storing the information. There are, however, other forms of information that are physical - one of the most ubiquitous being reference frame information. Here we analyze the problem of error correcting physical information. The basic question we seek to answer is whether or not such error correction is possible and, if so, what limitations govern the process. The main challenge is that the systems used for transmitting physical information, in addition to any actions applied to them, must necessarily obey these limitations. Encoding and decoding operations that obey a restrictive set of limitations need not exist a priori. We focus on the case of erasure errors, and we first show that the problem is equivalent to quantum error correction using group-covariant encodings. We prove a no-go theorem showing that that no finite dimensional, group-covariant quantum codes exist for Lie groups with an infinitesimal generator (e.g., U(1), SU(2), and SO(3)). We then explain how one can circumvent this no-go theorem using infinite dimensional codes, and we give an explicit example of a covariant quantum error correcting code using continuous variables for the group U(1). Finally, we demonstrate that all finite groups have finite dimensional codes, giving both an explicit construction and a randomized approximate construction with exponentially better parameters.

Quality aware error concealment method for video and game streaming and a viewing device employing the same

Abstract: A viewing device, a method of displaying streamed data frames and a client viewing device are disclosed herein. In one embodiment, the video viewing device includes: (1) a screen, (2) a decoder configured to decode a data frame received in a bitstream from a transmitter to provide a decoded data frame, and (3) an error concealer configured to either discard the decoded data frame or select the decoded data frame for display on the screen based on a complexity of the decoded data frame.

Image processing device, image display system and vehicle provided with same, image processing method and recording medium records program for executing same

Abstract: The image processing device includes: a first motion vector detecting section detects a first motion vector indicating a motion from a subsequent frame to the target frame; a second motion vector detecting section detects a second motion vector indicating a motion from a previous frame to the target frame; a first moved image generating section generates data of a first moved image based on data of the subsequent frame and the first motion vector; a second moved image generating section generates data of a second moved image based on data of the previous frame and the second motion vector; and a corrected image generating section generates data of a corrected image, based on data of the target frame, and the data of the first and the second moved images.

Impact of information buffering on a flexible cloud gaming system

Abstract: Cloud gaming will make high quality gaming more accessible to everyone. It not only relieves users of the constant need to upgrade their hardware, but also makes it possible to enjoy high quality gaming on a variety of devices. Although cloud gaming alleviates some of the hardware constraints associated with end user devices, it does come at a cost - an array of performance issues such as increased bandwidth usage, degradation in visual quality, and higher interactive latency. In previous work, we proposed a flexible cloud gaming system that utilizes different combinations of local and remote resources that trades off different performance requirements. A 3D image warping technique is used to help reduce or even eliminate interactive latency by making use of the most recently received residual frame. However, due to occlusion effect and insufficient sampling, some of the pixels will not get the residual information needed to recover their original values. In this paper, we address this issue by applying 3D image warping to multiple recently received and buffered residual frames. Our experimental results show that the impact on visual quality can be positive or negative depending on factors such as interactive latency and base Level of Detail (LOD) models used on the client side.

Video transcoding method and electronic apparatus

Abstract: Disclosed is a video transcoding method which enhances the efficiency of segmentation transcoding, including: performing frame rate conversion analysis on a video to obtain result information of frame rate conversion and position information of an IDR frame, and dividing the video into first video clips according to the position information of the IDR frame; splicing all the first video clips to produce second video clips according to chronological order and preset rule; encoding all the second video clips to obtain statistical file of the video according to the result information of frame rate conversion; determining scene switching position of the video according to predetermined frame type of the statistical file; splicing all the first video clips to produce third video clips according to the scene switching position; encoding and splicing all the third video clips to produce a complete video file according to the result information of frame rate conversion.

Motion compensated frame rate up-conversion using 3D frequency selective extrapolation and a multi-layer consistency check

Abstract: A high temporal resolution is desirable in many applications such as entertainment systems, automotive systems, or video surveillance. Apart from using cameras with a higher temporal resolution, it is also possible to employ frame rate up-conversion methods to obtain an enhanced temporal resolution. In principle, those algorithms can be grouped into approaches that rely on a motion estimation and approaches that do not. Both strategies typically process a video sequence frame by frame and take into account only the directly adjacent frames to compute the intermediate frame. In this paper, we propose a frame rate up-conversion technique that employs a motion compensated three-dimensional reconstruction algorithm. As a result, the proposed method takes into account more than two frames and is capable of jointly reconstructing up to a certain amount of missing frames in a video sequence. Furthermore, we present a multi-layer consistency check to further improve the reconstruction. On average, simulation results show a luminance PSNR gain compared to a conventional frame rate up-conversion method of 0.5 dB. Visual examples substantiate our objective results.

A formulation approach with hybrid wavelet transform for high efficiency video coding

Abstract: Joint Collaborative Team of Video Coding(JCT-VC)collectively refined HEVC as a emerging standard with the compliance of former H.264/AVC. During the HEVC encoding process, the macro blocks of input video file are defined in each frame, between frames the motion vectors are estimated to generate compensated image. By Transform coding frame residuals of inter and intra predictions of HEVC, which possess distinct characters which contemplate residual frame. The residual frame information is performed using transformation, quantization and entropy coder, and the encoded bit stream is decoded to reconstruct the original video. In this paper, by using parallel transformation technique, the residual frame encoding and decoding is implemented for the improvement of HEVC. The results are presented with the use of proposed parallel transformation technique in the framework of the HEVC with algorithm-1 and algorithm-2 approach and performance results of our implementation shown better results in video quality metrics. Considering with the use of these two algorithms on HEVC transform, the performance analysis of PSNR is improved with existing Wavelet Transform.

An efficient 3D video frame interpolation method using color-depth-motion information

Abstract: An efficient frame interpolation method for 3D video is proposed in this paper. First, the motion vector field (MVF) of the interpolated frame is estimated using bidirectional motion estimation method. Then, the blocks in the interpolated frame are classified into occlusion and normal blocks according to the depth information. After that, the motion vector (MV) outliers of the normal blocks are detected and corrected. Furthermore, the MVs of the occlusion blocks are post-processed using a foreground matching method, and one occlusion block is segmented into foreground and background parts, which will be interpolated discriminatively. Simulation results show that the proposed method achieves higher PSNR and SSIM values than traditional motion-compensated frame interpolation (MCFI) methods, and moreover, the visual quality of the interpolated frames using the proposed method is better.

A semi-global motion estimation of a repetition pattern region for frame interpolation

Abstract: In motion compensated frame interpolation, a repetition pattern in an image makes it difficult to derive an accurate motion vector because multiple similar local minima exist in the search space of the matching cost for motion estimation. In order to improve the accuracy of motion estimation in a repetition region, this paper attempts a semi-global approach that exploits both local and global characteristics of a repetition region. Experimental results demonstrate that the proposed method significantly outperforms the previous local approach achieving the PSNR improvement by around 2.59 dB.

A low latency coding scheme for compressing reference frame in video codec

Abstract: Designing ultra-low latency coding algorithms has been the key concern for the development of live video system. In the current video coding standards, the capacities of frame memory and bus bandwidth are the major aspects that affect the performance of encoder and decoder. To address this issue, this paper proposes a low-latency reference frame compression algorithm, which utilizes spatial correlation of frame to improve the throughput of frame memory. The proposed approach makes effective use of spatial correlation between pixels through special compression's order, and employs Golomb-Rice coding approach to encode the magnitude of pixel difference optimally according to its occurring frequency.

Gaming machine and method employing video compression

Abstract: A video compression technique relates to gaming machines which comprise a display to display images; a user interface to receive input from displayed screen images; a memory; and a game controller configured to control display of the images and process input received via the user interface. The game controller is further configured to receive an image sequence comprising a plurality of frames and, for each frame, to divide the frame into a series of blocks; compress each of the blocks using a first compression technique to form a series of compressed blocks; for each compressed block in the frame, identify whether the block is identical to a previous block within the frame and, if so, replace the compressed block with an index to the previous block; for each compressed block in the frame, determine a data difference between the compressed block and a corresponding block in an immediately preceding frame.

Image processing method and apparatus based on screen spliting

Abstract: Disclosed herein is an image processing apparatus based on the immediate transmission and reception of a split screen, including a transmission frame buffer configured to sequentially provide an image frame, an image encoder configured to control a compression method based on the complexity of the image frame, horizontally split the image frame, compress corresponding split image frames in accordance with the compression method, and provide the corresponding split and compressed image frames immediately, a reception frame buffer configured to buffer the corresponding split and compressed image frames, and an image decoder configured to decode the corresponding buffered and split and compressed image frames before buffering regarding the image frame is completed and display the corresponding decoded and split and compressed image frames immediately Accordingly, an image frame can be split, compressed, and transmitted immediately without waiting for the entire image frame to be stored

Methods for generating hdr (high dynamic range) images and apparatuses using the same

Abstract: A method for generating HDR (High Dynamic Range) images, performed by a processing unit, is introduced to at least contain: acquiring a frame 0 and a frame 1; calculating a first MV (Motion Vector) between the frame 0 and the frame 1; acquiring a frame 2; predicting a second MV between the frame 0 and the frame 2 according to the first MV, a time interval between shooting moments for the frames 0 and 1 and a time interval between shooting moments for the frames 0 and 2; generating a first MD (Motion Detection) matrix comprising a plurality of first MD flags according to the second MV; and fusing the frame 0 with the frame 2 according to the first MD flags.