The processing time needed for motion estimation usually accounts for a significant part of the overall processing time of the video encoder. To improve the video encoding speed, reducing the execution time for motion estimation process is essential. Parallel implementation of video encoding systems using either the software or the hardware approach has attracted much attention in the area of real time video coding. In this paper, we attempt to implement a video encoder on a bus network. Usually, for such a parallel system, the key concern is associated with partitioning and balancing of the computational load among the processors such that the overall processing time of the video encoder is minimized. With the use of the divisible load theory (DLT) paradigm, a strip-wise load partitioning/balancing scheme, a load distribution strategy, two implementation strategies are developed to exploit the data parallelism inherent in the video encoding process. The striking feature of our design is that,both the granularity of the load partitions and all the associated overheads caused during parallel video encoding process can be explicitly considered. This significantly contributes to the minimization of the overall processing time of the video encoder. Extensive experimental studies are carried out to test the effectiveness of the proposed strategies. The performance of the parallel video encoder is quantified using the metrics speedup and performance gain, respectively. The experimental results show that our strategies are effective for exploiting the available parallelism inherent in the video encoding process and provide a theoretical insight on how to analytically quantify and minimize the overall processing time of a parallel system. The proposed strategies can be easily extended and applied to improve other existing parallel systems.

/pdf/design-and-implementation-of-parallel-video-encoding-313nzlwgo0.pdf

Design and implementation of parallel video encoding strategies using divisible load analysis

Driven by the rapidly increasing demand for audio-visual applications, digital video compression technology has become a mature field, offering several available products based on both hardware and software implementations. Taking advantage of spatial, temporal, and statistical redundancies in video data, a video compression system aims to maximize the compression ratio while maintaining a high picture quality. Despite the tremendous progress in this area, video compression remains a challenging research problem due to its computational requirements and also because of the need for higher picture quality at lower data rates. Designing efficient coding algorithms continues to be a prolific area of research. For circumvent the computational requirement, researchers has resorted to parallel processing with a variety of approaches using dedicated parallel VLSI architectures as well as software on general-purpose available multiprocessor systems. Despite the availability of fast single processors, parallel processing helps to explore advanced algorithms and to build more sophisticated systems. This paper presents an overview of the recent research in video compression using parallel processing. The paper provides a discussion of the basic compression techniques, existing video coding standards, and various parallelization approaches. Since video compression is multi-step in nature using various algorithms, parallel processing can be exploited at an individual algorithm or at a complete system level. The paper covers a broad spectrum of such approaches, outlining the basic philosophy of each approach and providing examples. We contrast these approaches when possible, highlight their pros and cons, and suggest future research directions. While the emphasis of this paper is on software-based methods, a significant discussion of hardware and VLSI is also included.

/pdf/video-compression-with-parallel-processing-2y3katbvpl.pdf

Video compression with parallel processing

We consider a cluster-based multimedia Web server that dynamically generates video units to satisfy the bit rate and bandwidth requirements of a variety of clients. The media server partitions the job into several tasks and schedules them on the backend computing nodes for processing. For stream-based applications, the main design criteria of the scheduling are to minimize the total processing time and maintain the order of media units for each outgoing stream. In this paper, we first design, implement, and evaluate three scheduling algorithms, first fit (FF), stream-based mapping (SM), and adaptive load sharing (ALS), for multimedia transcoding in a cluster environment. We determined that it is necessary to predict the CPU load for each multimedia task and schedule them accordingly due to the variability of the individual jobs/tasks. We, therefore, propose an online prediction algorithm that can dynamically predict the processing time per individual task (media unit). We then propose two new load scheduling algorithms, namely, prediction-based least load first (P-LLF) and prediction-based adaptive partitioning (P-AP), which can use prediction to improve the performance. The performance of the system is evaluated in terms of system throughput, out-of-order rate of outgoing media streams, and load balancing overhead through real measurements using a cluster of computers. The performance of the new load balancing algorithms is compared with all other load balancing schemes to show that P-AP greatly reduces the delay jitter and achieves high throughput for a variety of workloads in a heterogeneous cluster. It strikes a good balance between the throughput and output order of the processed media units

/pdf/load-balancing-in-a-cluster-based-web-server-for-multimedia-253fnsg4ou.pdf

Load Balancing in a Cluster-Based Web Server for Multimedia Applications

We describe QoSMap, an overlay construction mechanism which computes high quality overlay networks for applications having stringent constraints on hop-degrading QoS metrics and provides resilience against the Internet’s unpredictable network behavior. QoSMap implements three features to achieve the QoS and resiliency goals. In order to provide high QoS, it constructs overlay communication edges with short but efficient underlay paths. In case of QoS violations due to changing network conditions, QoSMap utilizes supplemental backup paths, which are specifically constructed in order to extend overlay lifetimes. QoSMap also avoids nodes that have experienced recent QoS failure, further improving the resilience of the overlays. PlanetLab experiments prove the ability of QoSMap to construct efficient and resilient overlays for applications with stringent QoS constraints of latency and packet loss for a variety of topologies. QoSMap-constructed overlays significantly outperform overlays constructed without the three QoSMap features. The results also reveal the usability and effectiveness of QoSMap to construct near-optimal solution in a short amount of time, compared to the optimal solution which has a very high time complexity.

QoSMap: Achieving Quality and Resilience through Overlay Construction

Providing interactive video on hand-held, mobile devices is extremely difficult. These devices are subject to processor, memory, and power constraints, and communicate over wireless links of rapidly varying quality. Furthermore, the size of encoded video is difficult to predict, complicating the encoding task. We present Fugue, a system that copes with these challenges through a division along time scales of adaptation. Fugue is structured as three sperate controllers: transmission, video and preference. This decomposition provides adaptation along different time scales: per-packet, per-frame, and per-video. The controllers are provided at modest time and space costs compared to the cost of video encoding. We present simulations confirming the efficacy of our transmission controller, and compare our video controller to several alternatives. We find that, in situations amenable to adaptive compression, our scheme provides video quality equal to or better than the alternatives at a comparable or substantially lower computational cost. We also find that distortion, the metric commonly used to compare mobile video, under-values the contribution smooth motion makes to perceived video quality.

/pdf/fugue-time-scales-of-adaptation-in-mobile-video-52b481ldet.pdf

Fugue: time scales of adaptation in mobile video

We describe a software-based MPEG4 video encoder which is implemented using parallel processing on a cluster of workstations collectively working as a virtual machine. The contributions of our work are as follows. First, a hierarchical Petri-nets-based modeling methodology is proposed to capture the spatiotemporal relationships among multiple objects at different levels of an MPEG-4 video sequence. Second, a scheduling algorithm is proposed to assign video objects to workstations for encoding in parallel. The algorithm determines the execution order of video objects, ensures that the synchronization requirements among them are enforced and that presentation deadlines are met. Third, a dynamic partitioning scheme is proposed which divides an object among multiple workstations to extract additional parallelism. The scheme achieves load balancing among the workstations with a low overhead. The striking feature of our encoder is that it adjusts the allocation and partitioning of objects automatically according to the dynamic variations in the video object behavior. We have made various additional software optimizations to further speed up the computation. The performance of the encoder can scale according to the number of workstations used. With 20 workstations, the encoder yields an encoding rate higher than real time, allowing the encoding of multiple sequences simultaneously.

A software-based MPEG-4 video encoder using parallel processing

MPEG-4 currently being finalized by the Moving Pictures Experts Group of the ISO is a multimedia standard, MPEG-4 aims to support content-based coding of audio, text, image, and video (synthetic and natural) data, multiplexing of coded data, as well as composition and representation of audiovisual scenes. One of the most critical components of an MPEG-4 environment is the system encoder. An MPEG-4 scene may contain several audio and video objects, images, and text, each of which must be encoded individually and then multiplexed to form the system bitstream. Due to its flexible features, object-based nature, and provision for user interaction, MPEG-4 encoder is highly suitable for a software-based implementation. A full-scale software-based MPEG-J system encoder with real-time encoding speed is a nontrivial task and requires massive computation. We have built such an encoder using a cluster of workstations collectively working as a virtual parallel machine. Parallel processing of MPEG-4 encoder needs to be carried out carefully as objects may appear or disappear dynamically in a scene. In addition, objects may be synchronized with each other. User interactions may also prohibit a straightforward parallelization. We propose a modeling methodology that captures the spatio-temporal relationship between various objects and user interaction. We then propose a number of scheduling algorithms that periodically allocate MPEG-4 objects to multiple workstations ensuring load balancing and synchronization requirements among multiple objects. Each scheduling algorithm has its own performance and complexity characteristics. The experimental results, while showing real-time encoding rates, exhibit tradeoffs between load balancing, scheduling overhead cost, and global performance.

/pdf/real-time-interactive-mpeg-4-system-encoder-using-a-cluster-3w4kewcaqv.pdf

Real-time interactive MPEG-4 system encoder using a cluster of workstations

MPEG-4 which is currently being developed by MPEG (Moving Pictures Experts Group), is poised to become a standard for supporting current and emerging interactive multimedia applications. The objective of MPEG-4 is to support content-based compression, communication, access and manipulation of digital objects which can be natural or synthetic. Since MPEG-4 based video consists of objects and provides full interactivity between the client and the server, a software-based implementation seems to be the only viable approach for building an MPEG-4 encoder. Parallel processing solves the problem of large computational requirements for building a real-time encoder. In this paper, we describe a parallel implementation of MPEG-4 video encoder using a cluster of workstations collectively working as a virtual machine. Parallelization of the MPEG-4 encoder poses an interesting problem since not only can objects be added or deleted from a video scene but their sizes and shapes may vary with time. Moreover, some of the computationally intensive parts of the encoder are non-uniform algorithms, which means their execution times are data dependent and cannot be predicted in advance. In order to guarantee the spatio-temporal relationship between various objects in a video, we propose a real-time scheduling algorithm for exploiting parallelism in the temporal domain. The algorithm divides the workstations into a number of groups and assigns one video object to one group of workstations for encoding. A dynamic shape-adaptive data partitioning strategy is proposed to exploit parallelism in the spatial domain. The partitioning strategy divides the data of an object among the workstations within a group. The scheduling scheme ensures the synchronization requirements among multiple objects while the dynamic data parallel approach adapts to the object shape variations to balance the load for all the workstations. The performance of the encoder can scale according to the number of workstations used. With 20 workstations, the encoder yields an encoding rate higher than real-time, allowing to encode multiple sequences simultaneously.

MPEG-4 based interactive video using parallel processing

In this paper, we first present an Object Composition Petri Nets (OCPN) based model methodology for describing the dynamic behaviour of the multiple video objects and user interactions during the entire MPEG-4 video session; then, a Group of Video Object Plane (GOV) based periodical scheduling algorithm is proposed to assign the encoder tasks to a cluster of workstations with load balancing guarantee The scheduling scheme can allocate the tasks efficiently according to the timing constraints as well as user interactions The performance of the encoder can scale according to the number of workstations used The experiment results indicate that a real-time encoding rate can be achieved for the sequences with multiple video objects

/pdf/modeling-and-scheduling-for-mpeg-4-based-video-encoder-using-17i38kaa7p.pdf

Modeling and Scheduling for MPEG-4 Based Video Encoder Using a Cluster of Workstations

With the rapid convergence in telecommunications, computer, and TV/film industries, rigorous efforts are in progress for designing the emerging multimedia standard-MPEG-4. The main features of MPEG-4 are its content-based interactivity, efficient coding, and universal access. For realizing MPEG-4 based application oriented systems, various issues must be carefully addressed. This paper describes a full scale MPEG-4 based interactive multimedia system project underway at the Video Technology Center of the Hong Kong University of Science and Technology. This project aims to provide complete MPEG-4 based interactive services tailored to various applications. In order to do so, a number of key enabling technologies need to be investigated. These include encoding and decoding, multimedia server design, a user interface, and MPEG-4 stream transmission over high speed networks. The unique features of the project are: (1) to explore the feasibility of parallel and distributed technology using a network-of-workstations (NOWs) to hinder the intensive computation required for real-time encoding; (2) to provide flexibility of software-based encoding/decoding; (3) to design an intelligent transport service mechanism which can satisfy the QoS requirement of MPEG-4.

Yong He

Papers

A software-based MPEG-4 video encoder using parallel processing

Real-time interactive MPEG-4 system encoder using a cluster of workstations

MPEG-4 based interactive video using parallel processing

Modeling and Scheduling for MPEG-4 Based Video Encoder Using a Cluster of Workstations

An MPEG-4 based interactive multimedia system