[서평]「Digital Video Processing」

The need for precise (subpixel accuracy) motion estimates in conventional super-resolution has limited its applicability to only video sequences with relatively simple motions such as global translational or affine displacements. In this paper, we introduce a novel framework for adaptive enhancement and spatiotemporal upscaling of videos containing complex activities without explicit need for accurate motion estimation. Our approach is based on multidimensional kernel regression, where each pixel in the video sequence is approximated with a 3-D local (Taylor) series, capturing the essential local behavior of its spatiotemporal neighborhood. The coefficients of this series are estimated by solving a local weighted least-squares problem, where the weights are a function of the 3-D space-time orientation in the neighborhood. As this framework is fundamentally based upon the comparison of neighboring pixels in both space and time, it implicitly contains information about the local motion of the pixels across time, therefore rendering unnecessary an explicit computation of motions of modest size. The proposed approach not only significantly widens the applicability of super-resolution methods to a broad variety of video sequences containing complex motions, but also yields improved overall performance. Using several examples, we illustrate that the developed algorithm has super-resolution capabilities that provide improved optical resolution in the output, while being able to work on general input video with essentially arbitrary motion.

Super-Resolution Without Explicit Subpixel Motion Estimation

This paper reviews the basic background knowledge necessary to design effective no-reference (NR) quality estimators (QEs) for images and video. We describe a three-stage framework for NR QE that encompasses the range of potential use scenarios for the NR QE and allows knowledge of the human visual system to be incorporated throughout. We survey the measurement stage of the framework, considering methods that rely on bitstream, pixels, or both. By exploring both the accuracy requirements of potential uses as well as evaluation criteria to stress-test a QE, we set the stage for our community to make substantial future improvements to the challenging problem of NR quality estimation.

No-reference image and video quality estimation: Applications and human-motivated design

In this paper we will present the concept of a system that allows for an evolutionary introduction of depth perception into the existing 2D digital TV framework. The work is part of the European Information Society Technologies (IST) project “Advanced Three-Dimensional Television System Technologies” (ATTEST), an activity where industries, research centers and universities have joined forces to design a backwardscompatible, flexible and modular broadcast 3D-TV system, where all parts of the 3D processing chain are optimised to one another. This includes content creation, coding and transmission, display and research in human 3D perception, which be will used to guide the development process. The goals of the project comprise the development of a novel broadcast 3D camera, algorithms to convert existing 2D-video material into 3D, a 2Dcompatible coding and transmission scheme for 3D-video using MPEG2/4/7 technologies and the design of two new autostereoscopic displays.

IST project ATTEST - an evolutionary and optimised approach on 3D-TV

Peer to peer.

Scalable video coders provide different scaling options, such as temporal, spatial, and SNR scalabilities, where rate reduction by discarding enhancement layers of different scalability-type results in different kinds and/or levels of visual distortion depend on the content and bitrate. This dependency between scalability type, video content, and bitrate is not well investigated in the literature. To this effect, we first propose an objective function that quantifies flatness, blockiness, blurriness, and temporal jerkiness artifacts caused by rate reduction by spatial size, frame rate, and quantization parameter scaling. Next, the weights of this objective function are determined for different content (shot) types and different bitrates using a training procedure with subjective evaluation. Finally, a method is proposed for choosing the best scaling type for each temporal segment that results in minimum visual distortion according to this objective function given the content type of temporal segments. Two subjective tests have been performed to validate the proposed procedure for content-aware selection of the best scalability type on soccer videos. Soccer videos scaled from 600 kbps to 100 kbps by the proposed content-aware selection of scalability type have been found visually superior to those that are scaled using a single scalability option over the whole sequence.

/pdf/content-aware-scalability-type-selection-for-rate-adaptation-3dy88pojfm.pdf

Content-aware scalability-type selection for rate adaptation of scalable video

Multimedia services for mobile phones are becoming increasingly popular thanks to capabilities brought about
by location awareness, customized programming, interactivity, and portability. With mounting attraction to
these services there is desire to seamlessly expand the mobile multimedia experience to stationary environments
where high-resolution displays can offer significantly better viewing conditions. In this paper, we propose a
fast, high quality super-resolution algorithm that enables high resolution display of low-resolution video. The
proposed algorithm, SWAT, accomplishes sparse reconstructions using directionally warped transforms and spatially
adaptive thresholding. Comparisons are made with some existing techniques in terms of PSNR and visual
quality. Simulation examples show that SWAT significantly outperforms these techniques while staying within
a limited computational complexity envelope.

Fast super-resolution reconstructions of mobile video using warped transforms and adaptive thresholding

The bulk of the video content available today over the Internet and over mobile networks suffers from many
imperfections caused during acquisition and transmission. In the case of user-generated content, which is typically
produced with inexpensive equipment, these imperfections manifest in various ways through noise, temporal
flicker and blurring, just to name a few. Imperfections caused by compression noise and temporal flicker are
present in both studio-produced and user-generated video content transmitted at low bit-rates. In this paper,
we introduce an algorithm designed to reduce temporal flicker and noise in video sequences. The algorithm takes
advantage of the sparse nature of video signals in an appropriate transform domain that is chosen adaptively based
on local signal statistics. When the signal corresponds to a sparse representation in this transform domain, flicker
and noise, which are spread over the entire domain, can be reduced easily by enforcing sparsity. Our results show
that the proposed algorithm reduces flicker and noise significantly and enables better presentation of compressed
videos.

Temporal flicker reduction and denoising in video using sparse directional transforms

A method and apparatus is disclosed herein for reducing at least one of both flicker and noise in video sequences. In one embodiment, the method comprises receiving an input video and performing operations to reduce one or both of noise and flicker in the input video using spatial and temporal processing.

Noise and/or flicker reduction in video sequences using spatial and temporal processing

A method and apparatus is disclosed herein for performing content- based and royalty-based encoding. In one embodiment, the method comprises: selecting one or more encoding tools to encode media based on a royalty cost associated with at least one of the one or more encoding tools and a corresponding decoding tool for each of the one or more encoding tools, decoded media quality that each corresponding decoding tool produces, and one or more transmission bandwidth constraints; encoding the media, in accordance with the media content, using the one or more encoding tools; and transmitting encoded data generated by at least one of the one or more encoding tools.

M. Reha Civanlar

Papers

Content-aware scalability-type selection for rate adaptation of scalable video

Fast super-resolution reconstructions of mobile video using warped transforms and adaptive thresholding

Temporal flicker reduction and denoising in video using sparse directional transforms

Noise and/or flicker reduction in video sequences using spatial and temporal processing

Optimized content-based and royalty-based encoding and distribution of media data