H.264/AVC significantly outperforms previous video coding standards with many new coding tools. However, the better performance comes at the price of the extraordinarily huge computational complexity and memory access requirement, which makes it difficult to design a hardwired encoder for real-time applications. In addition, due to the complex, sequential, and highly data-dependent characteristics of the essential algorithms in H.264/AVC, both the pipelining and the parallel processing techniques are constrained to be employed. The hardware utilization and throughput are also decreased because of the block/MB/frame-level reconstruction loops. In this paper, we describe our techniques to design the H.264/AVC video encoder for HDTV applications. On the system design level, in consideration of the characteristics of the key components and the reconstruction loops, the four-stage macroblock pipelined system architecture is first proposed with an efficient scheduling and memory hierarchy. On the module design level, the design considerations of the significant modules are addressed followed by the hardware architectures, including low-bandwidth integer motion estimation, parallel fractional motion estimation, reconfigurable intrapredictor generator, dual-buffer block-pipelined entropy coder, and deblocking filter. With these techniques, the prototype chip of the efficient H.264/AVC encoder is implemented with 922.8 K logic gates and 34.72-KB SRAM at 108-MHz operation frequency.

/pdf/analysis-and-architecture-design-of-an-hdtv720p-30-frames-s-t9vvgg1ls6.pdf

Analysis and architecture design of an HDTV720p 30 frames/s H.264/AVC encoder

Techniques are described for encoding and decoding digital video data using macroblocks that are larger than the macroblocks prescribed by conventional video encoding and decoding standards. For example, the techniques include encoding and decoding a video stream using macroblocks comprising greater than 16×16 pixels. In one example, an apparatus includes a video encoder configured to encode a coded unit comprising a plurality of video blocks, wherein at least one of the plurality of video blocks comprises a size of more than 16×16 pixels and to generate syntax information for the coded unit that includes a maximum size value, wherein the maximum size value indicates a size of a largest one of the plurality of video blocks in the coded unit. The syntax information may also include a minimum size value. In this manner, the encoder may indicate to a decoder the proper syntax decoder to apply to the coded unit.

Video coding with large macroblocks

The performance evaluation of video transport mechanisms becomes increasingly important as encoded video accounts for growing portions of the network traffic. Compared to the widely studied MPEG-4 encoded video, the recently adopted H.264 video coding standards include novel mechanisms, such as hierarchical B frame prediction structures and highly efficient quality scalable coding, that have important implications for network transport. This tutorial introduces a trace-based evaluation methodology for the network transport of H.264 encoded video. We first give an overview of H.264 video coding, and then present the trace structures for capturing the characteristics of H.264 encoded video. We give an overview of the typical video traffic and quality characteristics of H.264 encoded video. Finally, we explain how to account for the H.264 specific coding mechanisms, such as hierarchical B frames, in networking studies.

/pdf/video-transport-evaluation-with-h-264-video-traces-2ztswbnb2c.pdf

Video Transport Evaluation With H.264 Video Traces

The recently developed H.264/AVC video codec with scalable video coding (SVC) extension, compresses non-scalable (single-layer) and scalable video significantly more efficiently than MPEG-4 Part 2. Since the traffic characteristics of encoded video have a significant impact on its network transport, we examine the bit rate-distortion and bit rate variability-distortion performance of single-layer video traffic of the H.264/AVC codec and SVC extension using long CIF resolution videos. We also compare the traffic characteristics of the hierarchical B frames (SVC) versus classical B frames. In addition, we examine the impact of frame size smoothing on the video traffic to mitigate the effect of bit rate variabilities. We find that compared to MPEG-4 Part 2, the H.264/AVC codec and SVC extension achieve lower average bit rates at the expense of significantly increased traffic variabilities that remain at a high level even with smoothing. Through simulations we investigate the implications of this increase in rate variability on (i) frame losses when transmitting a single video, and (ii) on a bufferless statistical multiplexing scenario with restricted link capacity and information loss. We find increased frame losses, and rate-distortion/rate-variability/encoding complexity tradeoffs. We conclude that solely assessing bit rate-distortion improvements of video encoder technologies is not sufficient to predict the performance in specific networked application scenarios.

Traffic and Quality Characterization of Single-Layer Video Streams Encoded with the H.264/MPEG-4 Advanced Video Coding Standard and Scalable Video Coding Extension

Apparatus and methods of using content information for encoding multimedia data are described. A method of processing multimedia data includes classifying content of multimedia data, and encoding the multimedia data in a first data group and in a second data group based on the content classification, wherein the first data group comprises a coefficient and the second data group comprises a first differential refinement associated with the first data group coefficient. An apparatus for using content information for encoding multimedia data includes a content classifying module configured to classify content of multimedia data and provide content classification data, and an encoder configured to encode the multimedia data in a first data group and in a second data group based on the content classification, wherein the first data group comprises a coefficient and the second data group comprises a first differential refinement associated with the first data group coefficient.

Scalability techniques based on content information

H.264/MPEG-4 AVC is a recently completed video compression standard jointly developed by the ITU-T VCEG and the ISO/IEC MPEG standards committees. The standard promises much higher compression than that possible with earlier standards. It allows coding of non-interlaced and interlaced video very efficiently, and even at high bit rates provides more acceptable visual quality than earlier standards. Further, the standard supports flexibilities in coding as well as organization of coded data that can increase resilience to errors or losses. As might be expected, the increase in coding efficiency and coding flexibility comes at the expense of an increase in complexity with respect to earlier standards. In this paper, we first briefly introduce the video coding tools that the standard supports and how these tools are organized into profiles. As with earlier standards, the mechanism of profiles allows one to implement only a desired subset of the standard and still be interoperable with applications of interest. Next, we discuss how the various video coding tools of the standard work, as well as the related issue of how to perform encoding using these tools. We then evaluate the coding performance in terms of contribution to overall improvement offered by individual tools, options within these tools, and important combinations of tools, on a representative set of video test sequences and movie clips. Next, we discuss a number of additional elements of the standard such as, tools that provide system support, details of levels of profiles, and the issue of encoder and decoder complexity. Finally, we summarize our overview and analysis of this standard, by identifying, based on their performance, promising tools as well as options within various tools. r 2004 Elsevier B.V. All rights reserved.

Atul Puri

Papers

Video coding using the H.264/MPEG-4 AVC compression standard