This paper summarizes the results of over 25 research groups or individual researchers that have presented video coding implementations on general-purpose processors with the new single instruction multiple data media instruction set architecture extensions. The extensions are introduced and the fundamentals for extensions, as well as some inherent problems, are explained. The reported attempts to utilize the extensions are divided into kernel- and application-level, as well as platform dependent and independent optimizations. Optimized applications include, in addition to some proprietary methods, all of the major video coding standards such as H.261, H.263, MPEG-4, MPEG-1, and MPEG-2. These optimized implementations include a complete video codec, several decoders, and several encoders. Additionally, a performance comparison is given for four representative encoder implementations based on the reported results. Also included is an overview of future trends for new instructions and architectural speed-up techniques.

Overview of research efforts on media ISA extensions and their usage in video coding

This paper presents an MPEG-4 video codec, called MoVa, for video coding applications that adopts 3G-324M. We designed MoVa to be optimal by embedding a cost-effective ARM7TDMI core and partitioning it into hardwired blocks and firmware blocks to provide a reasonable tradeoff between computational requirements, power consumption, and programmability. Typical hardwired blocks are motion estimation and motion compensation, discrete cosine transform and quantization, and variable length coding and decoding, while intra refresh, rate control, error resilience, error concealment, etc. are implemented by software. MoVa has a pipeline structure and its operation is performed in four stages at encoding and in three stages at decoding. It meets the requirements of MPEG-4 SP@L2 and can perform either 30 frames/s (fps) of QCIF or SQCIF, or 7.5 fps (in codec mode) to 15 fps (in encode/decode mode) of CIF at a maximum clock rate of 27 MHz for 128 kbps or 144 kbps. MoVa can be applied to many video systems requiring a high bit rate and various video formats, such as videophone, videoconferencing, surveillance, news, and entertainment.

Hardware-Software Implementation of MPEG-4 Video Codec

A performance analysis of MPEG-4 encoder and decoder programs on a standard personal computer is presented. The paper first describes the MPEG-4 computational load and discusses related works, then outlines the performance analysis. Experimental results show that while the decoder program can be easily executed in real time, the encoder requires execution times in the order of seconds per frame which call for substantial optimisation to satisfy real-time constraints.

/pdf/performance-analysis-of-mpeg-4-decoder-and-encoder-fr0vu98lyo.pdf

Performance analysis of MPEG-4 decoder and encoder

This paper describes a QVGA/CIF resolution MPEG-4 video codec based on a low-power, general-purpose DSP (NEC /spl mu/PD77210,160 MHz, 80 mW, 1.5 V). To enhance video codec performance, the codec employs fast algorithms, including, in motion estimation, a successive similarity detection algorithm (SSDA; a fast block matching) whose decision timing for termination of block matching is optimized. Further, the use of a software DMA queue reduces the wasteful DSP wait cycles that can result from massive access to external frame memories. The resulting codec executes QVGA /spl times/ 15 fps codec, or CIF /spl times/ 15 fps encoding at 384 kbps, in real time, performance levels sufficient for next-generation wireless videotelephony.

QVGA/CIF resolution MPEG-4 video codec based on a low-power and general-purpose DSP

This paper presents efficient algorithms and architectures for the MPEG-4 object-based video coding. The novel coding tools supporting for object-based coding will much increase the computational burden, in comparison with conventional frame-based video coding system. In order to process these extra computational tasks effectively design techniques both on algorithms and architectures have to be addressed. By the careful examination of computational behavior of algorithms, the best-matched computation models can be derived, and the target architectures can be determined. Based on this design strategy, the computational behavior of the MPEG-4 video coding tools at simple/core profiles and level 2 is explored first. Then, by considering the analysis results, efficient algorithms and architectures are developed to meet the design specifications. The shape coding tool, which is the most computation-extensive task supporting for object-based functionality is selected as our study case. The real-time performance of shape coding can be easily achieved by the proposed architecture approximately running at 7.8 MHz while the optimized software running on a full-speed RISC (Ultra Sparc, 300 MHz) can only have about 1/10 performance.

Efficient algorithms and architectures for MPEG-4 object-based video coding

A PC-based real-time software MPEG-4 video codec with a fast adaptive motion vector search is presented. In a fast adaptive motion estimation (ME) technique, the search order is dynamically changed in accordance with the motion of objects. This technique suppresses load fluctuation in the ME and contributes to the stable real-time work of the codec. MMX instructions are used to increase the codec speed. On a portable PC, the software video codec supports satisfactory mobile visual communication at 64 kbps and 128 kbps, for example, at QCIF 15 fps. The codec on a 450 MHz Pentium II processor can encode and decode 30 CIF frames in real-time.

H. Shinohara

Papers

Real-time software video codec with a fast adaptive motion vector search