We provide an overview of an architecture of today's Internet streaming media delivery networks and describe various problems that such systems pose with regard to video coding. We demonstrate that based on the distribution model (live or on-demand), the type of the network delivery mechanism (unicast versus multicast), and optimization criteria associated with particular segments of the network (e.g., minimization of distortion for a given connection rate, minimization of traffic in the dedicated delivery network, etc.), it is possible to identify several models of communication that may require different treatment from both source and channel coding perspectives. We explain how some of these problems can be addressed using a conventional framework of temporal motion-compensated, transform-based video compression algorithm, supported by appropriate channel-adaptation mechanisms in client and server components of a streaming media system. Most of these techniques have already been implemented in RealNetworks(R) RealSystem(R) 8 and its RealVideo(R) 8 codec, which we use throughout the paper to illustrate our results.

Video Coding for Streaming Media Delivery on the Internet

In an H.264/AVC video encoder, integer motion estimation (IME) requires 74.29% computational complexity and 77.49% memory access and becomes the most critical component for low-power applications. According to our analysis, an optimal low-power IME engine should be a parallel hardware architecture supporting fast algorithms and efficient data reuse (DR). In this paper, a hardware-oriented fast algorithm is proposed with the intra-/inter-candidate DR considerations. In addition, based on the systolic array and 2-D adder tree architecture, a ladder-shaped search window data arrangement and an advanced searching flow are proposed to efficiently support inter-candidate DR and reduce latency cycles. According to the implementation results, 97% computational complexity is saved by the proposed fast algorithm. In addition, 77.6% memory bandwidth is further saved with the proposed DR techniques at architecture level. In the ultra-low-power mode, the power consumption is 2.13 mW for real-time encoding CIF 30-fps videos at 13.5-MHz operating frequency

/pdf/fast-algorithm-and-architecture-design-of-low-power-integer-4s68yd5bzl.pdf

Fast Algorithm and Architecture Design of Low-Power Integer Motion Estimation for H.264/AVC

In this paper, we present a new hardware-oriented, modified diamond search (HMDS) algorithm, for fast integer pel, motion estimation in H.264/AVC. We also present our co-designed, low power very large scale integration (VLSI) architecture for HMDS. The goal of HMDS is to enable the support of high quality video on low power mobile devices and low bit rate applications which typically use H.264/AVC baseline profile at levels 1-2. Our experiments use standard test sequences ranging from QCIF to high-definition 1280 × 720p video. The proposed VLSI architecture is prototyped as an field-programable gate array (FPGA)-based field programable system-on-chip. Our results show that HMDS on average has better rate-distortion performance and speedup, compared to previous state-of-the-art fast motion estimation algorithms, while its losses compared to full search motion estimation, are insignificant. Our prototyped architecture is more hardware-efficient than previous FPGA-based architectures in terms of power consumption, area, throughput, and memory utilization. We also show that its performance in terms of maximum frequency, minimum frequency, transistor count, and power consumption are comparable to that of state-of-the-art architectures implemented on application-specific integrated circuits.

Algorithm and Architecture Co-Design of Hardware-Oriented, Modified Diamond Search for Fast Motion Estimation in H.264/AVC

Motion Estimation (ME) is the most computationally intensive part of video compression and video enhancement systems. For the recently available high definition (HD) video formats, the computational complexity of full search (FS) ME algorithm is prohibitively high, whereas the PSNR obtained by fast search ME algorithms is low. Therefore, in this paper, we present dynamically variable step search (DVSS) ME algorithm for processing high definition video formats and a dynamically reconfigurable hardware architecture for efficiently implementing DVSS algorithm. The simulation results showed that DVSS algorithm performs very close to FS algorithm by searching much fewer search locations than FS algorithm and it outperforms successful fast search ME algorithms by searching more search locations than these algorithms. The proposed hardware is implemented in VHDL and is capable of processing high definition video formats in real time. Therefore, it can be used in consumer electronics products for video compression, frame rate up-conversion and de-interlacing.

https://research.sabanciuniv.edu/12393/1/getPDF2.pdf

Dynamically variable step search motion estimation algorithm and a dynamically reconfigurable hardware for its implementation

3-D video will be the most prominent video technology in the next generation. Among the 3-D video technologies, stereo video systems are considered to be realized first in the near future. Stereo video systems require double bandwidth and more than twice the computational complexity relative to mono-video systems. Thus, an efficient coding scheme is necessary for transmitting stereo video. In this paper, a new structure of prediction core in stereo video coding systems is proposed from the algorithm level to the hardware architecture level. The joint prediction algorithm (JPA), which combines three prediction schemes, is proposed for high coding efficiency and low computational complexity. It makes the system outperform MPEG-4 temporal scalability and simple profile by 2-3 dB in rate-distortion performance. Besides, JPA also utilizes the characteristics of stereo video and successfully reduces about 80% computational complexity. Then, a new hardware architecture of the prediction core based on JPA and a modified hierarchical search block-matching algorithm is proposed. With a special data flow, no bubble cycles exist during the block-matching process. The proposed architecture also adopts the near-overlapped candidates reuse scheme to save the heavy burden of data access. Besides, both on-chip memory requirement and off-chip memory bandwidth can be reduced by the proposed new scheduling. Compared with the hardware requirement for the implementation of full search block-matching algorithm, only 11.5% on-chip SRAM and 3.3% processing elements are needed with a tiny PSNR drop, making it area-efficient while maintaining high stereo video quality and processing capability

/pdf/joint-prediction-algorithm-and-architecture-for-stereo-video-439c5xel4p.pdf

Joint Prediction Algorithm and Architecture for Stereo Video Hybrid Coding Systems

We present a novel hardware architecture supporting diamond search and fast full search for block matching motion estimation. It can handle irregular data flow of these fast algorithms without pipeline bubbles, and reduce computation of duplicated search positions. The proposed architecture needs preprocessing with a small amount of computational power while performing fast full search and is suitable for the platform-based video coding system. While the diamond search mode can be applied for real-time requirements, we can choose the fast full search mode, which adapts the processing cycles to picture contents and preserves the same quality of full search block matching (FSBM), for applications of high picture quality or compression ratio. It needs only 9K gates and one additional memory of search range size and is more cost-effective than the conventional systolic array architecture.

https://ieeexplore.ieee.org/iel5/7897/21785/01011032.pdf

A novel hybrid motion estimator supporting diamond search and fast full search

Video streaming with varying transmission bandwidth is becoming increasingly important. In this paper, an interactive video streaming system is proposed. Fine Granularity Scalability (FGS) is applied to be the streaming video format. The computational complexity of FGS coding is analyzed to explore an efficient FGS implementation. A new transmission model is proposed for the realization of a content-aware video streaming. At encoder side, the current MPEG-4 FGS coding flow is reordered such that the picture-level maximum can be acquired in advance and bit-plane data can be dynamically adapted. With these proposed hardware-oriented optimization approaches, a hardwired FGS block-level processing core is proposed to achieve a cost-effective solution to FGS implementation. The streaming server can adaptively decide quality-enhanced region by selective enhancement according to both object information from encoding side and user-defined region from receiver side. From the simulation results, it's demonstrated that the proposed approach can provide better quality in users' interest regions with no bit-rate or complexity overhead.

/pdf/interactive-content-aware-video-streaming-system-with-fine-4dn84m24uh.pdf

Chih-Wei Hsu

Papers

A novel hybrid motion estimator supporting diamond search and fast full search

Interactive Content-aware Video Streaming System with Fine Granularity Scalability

Novel scheme for reducing communication data traffic in advanced metering infrastructure networks