We consider image transformation problems, where an input image is transformed into an output image. Recent methods for such problems typically train feed-forward convolutional neural networks using a per-pixel loss between the output and ground-truth images. Parallel work has shown that high-quality images can be generated by defining and optimizing perceptual loss functions based on high-level features extracted from pretrained networks. We combine the benefits of both approaches, and propose the use of perceptual loss functions for training feed-forward networks for image transformation tasks. We show results on image style transfer, where a feed-forward network is trained to solve the optimization problem proposed by Gatys et al. in real-time. Compared to the optimization-based method, our network gives similar qualitative results but is three orders of magnitude faster. We also experiment with single-image super-resolution, where replacing a per-pixel loss with a perceptual loss gives visually pleasing results.

/pdf/perceptual-losses-for-real-time-style-transfer-and-super-hq4q54uf80.pdf

Perceptual Losses for Real-Time Style Transfer and Super-Resolution

We consider image transformation problems, where an input image is transformed into an output image. Recent methods for such problems typically train feed-forward convolutional neural networks using a \emph{per-pixel} loss between the output and ground-truth images. Parallel work has shown that high-quality images can be generated by defining and optimizing \emph{perceptual} loss functions based on high-level features extracted from pretrained networks. We combine the benefits of both approaches, and propose the use of perceptual loss functions for training feed-forward networks for image transformation tasks. We show results on image style transfer, where a feed-forward network is trained to solve the optimization problem proposed by Gatys et al in real-time. Compared to the optimization-based method, our network gives similar qualitative results but is three orders of magnitude faster. We also experiment with single-image super-resolution, where replacing a per-pixel loss with a perceptual loss gives visually pleasing results.

We have witnessed great interest and a wealth of promise in content-based image retrieval as an emerging technology. While the last decade laid foundation to such promise, it also paved the way for a large number of new techniques and systems, got many new people involved, and triggered stronger association of weakly related fields. In this article, we survey almost 300 key theoretical and empirical contributions in the current decade related to image retrieval and automatic image annotation, and in the process discuss the spawning of related subfields. We also discuss significant challenges involved in the adaptation of existing image retrieval techniques to build systems that can be useful in the real world. In retrospect of what has been achieved so far, we also conjecture what the future may hold for image retrieval research.

/pdf/image-retrieval-ideas-influences-and-trends-of-the-new-age-51whw4w7su.pdf

Image retrieval: Ideas, influences, and trends of the new age

About the Author.Foreword.Preface.Glossary.1. Introduction.2. Video Formats and Quality.3. Video Coding Concepts.4. The MPEG-4 and H.264 Standards.5. MPEG-4 Visual.6. H.264/MPEG-4 Part 10.7. Design and Performance.8. Applications and Directions.Bibliography.Index.

H.264 and MPEG-4 Video Compression: Video Coding for Next-generation Multimedia

International Journal of Computer and Applications

Experimental data are presented that clearly demonstrate the scope of application of peak signal-to-noise ratio (PSNR) as a video quality metric. It is shown that as long as the video content and the codec type are not changed, PSNR is a valid quality measure. However, when the content is changed, correlation between subjective quality and PSNR is highly reduced. Hence PSNR cannot be a reliable method for assessing the video quality across different video contents.

Scope of validity of PSNR in image/video quality assessment

A fast block-matching algorithm for motion estimation is presented. It is based on a logarithmic step where, in each search step, only four locations are tested. For a motion displacement of w pels/frame, this technique requires 5+4 log/sub 2/w computations to locate the best match. Using sequences of CIF standard pictures, the interframe motion compensated prediction error with this technique is compared to the other fast methods. The computational complexity of this algorithm is also compared against those methods. >

The cross-search algorithm for motion estimation (image coding)

Two-layer conditional-replenishment coding of video signals over a variable-bit-rate (VBR) network is described. A slotted-ring network based on an Orwell protocol is assumed, where transmission of certain packets is guaranteed. The two-layer coder produces two output bit streams: the first bit stream contains all the important structural information in the image and is accommodated in the guaranteed capacity of the network, while the second adds the necessary quality finish. The performance of the coder is tested with CIF standard sequences and broadcast-quality pictures. The portion of the VBR channel allocated to the lower layer as guaranteed bandwidth is examined. Using broadcast-quality pictures, statistics were obtained on the performance of this system for different choices of bit rate in the lower layer. The effect of lost packets is shown on CIF standard picture sequences. It is shown that the coder performs well for a guaranteed channel rate as low as 10-20% of the total bit rate. >

Two-layer coding of video signals for VBR networks

This book discusses the growth of digital television technology and the revolution in image and video compression, highlighting the need for standardization in processing static and moving images and their exchange between computer systems.

Standard Codecs: Image Compression to Advanced Video Coding

In this work, transcoding of pre-encoded MPEG-1, 2 video into lower bit rates is realized through altering the coding algorithm into H.261/H.263 standards with lower spatio-temporal resolutions. For this heterogeneous transcoding, we extract and compose a set of candidate motion vectors, from the incoming bit stream, to comply with the encoding format of the output bit stream. For the spatial resolution reduction we generate one motion vector out of a set of input motion vectors operating on the higher spatial resolution image. Finally, for the temporal resolution reduction we compose new motion vectors from the dropped frames motion vectors. Throughout the paper, we discuss the impact of motion estimation refinement on the new motion vectors and show that for all cases a simple half-pixel refinement is sufficient for near-optimum results.

/pdf/heterogeneous-video-transcoding-to-lower-spatio-temporal-1u4dkydd3e.pdf

Mohammed Ghanbari

Papers

Scope of validity of PSNR in image/video quality assessment

The cross-search algorithm for motion estimation (image coding)

Two-layer coding of video signals for VBR networks

Standard Codecs: Image Compression to Advanced Video Coding

Heterogeneous video transcoding to lower spatio-temporal resolutions and different encoding formats