Degree of parallelism

About: Degree of parallelism is a research topic. Over the lifetime, 1515 publications have been published within this topic receiving 25546 citations.

Communications architecture for a high throughput storage processor

Abstract: A storage processor particularly suited to RAID systems provides high throughput for applications such as streaming video data An embodiment is configured as an ASIC with a high degree of parallelism in its interconnections The preferred embodiment provides a store and forward architecture configured around a switch with prioritization on data pathways critical to high throughput

Enabling intra-partition parallelism for partition-based operations

Abstract: Techniques are provided for increasing the degree of parallelism without incurring overhead costs associated with inter-nodal communication for performing parallel operations. One aspect of the invention is to distribute-phase partition-pairs of a parallel partition-wise operation on a pair of objects among the nodes of a database system. The -phase partition-pairs that are distributed to each node are further partitioned to form a new set of-phase partition-pairs. One -phase partition-pair from the set of new-phase partition-pairs is assigned to each slave process that is on a given node. In addition, a target object may be partitioned by applying an appropriate hash function to the tuples of the target object. The parallel operation is performed by broadcasting each tuple from a source table only to the group of slave processes that is working on the static partition to which the tuple is mapped.

Scheduling Mixed Real-Time and Non-real-Time Applications in MapReduce Environment

Abstract: MapReduce scheduling is becoming a hot topic as MapReduce attracts more and more attention from both industry and academia. In this paper, we focus on the scheduling of mixed real-time and non-real-time applications in MapReduce environment, which is a challenging problem but receives only limited attention. To solve this problem, we present a two-level MapReduce scheduler built on previous techniques and make two key contributions. First, to meet the performance goal of real-time applications, we propose a deadline scheduler which adopts (1) a sampling based approach-Tasks Forward Scheduling (TFS) to predict map/reduce task execution time(unlike prior work that requires users to input an estimated value). (2) a resource allocation model-Approximately Uniform Minimum Degree of parallelism (AUMD) to dynamically control each realtime job to execute with minimum tasks assignment in any time so as to maximize the number of concurrent real-time jobs. Second, through integrating this deadline scheduler into existing MapReduce scheduler, we develop a two-level scheduler with resource preemption supported, and it could schedule mixed real-time and non-real-time jobs according to their respective performance demands. We implement our scheduler in Hadoop system and experiments running on a real, small-scale cluster demonstrate that it could schedule mixed real-time and nonreal-time jobs to meet their different quality-of-service (QoS) demands.

A scalable scheduling scheme for functional parallelism on distributed memory multiprocessor systems

Abstract: We attempt a new variant of the scheduling problem by investigating the scalability of the schedule length with the required number of processors, by performing scheduling partially at compile time and partially at run time. Assuming infinite number of processors, the compile time schedule is found using a new concept of the threshold of a task that quantifies a trade-off between the schedule-length and the degree of parallelism. The schedule is found to minimize either the schedule length or the number of required processors and it satisfies: A feasibility condition which guarantees that the schedule delay of a task from its earliest start time is below the threshold, and an optimality condition which uses a merit function to decide the best task-processor match for a set of tasks competing for a given processor. At run time, the tasks are merged producing a schedule for a smaller number of available processors. This allows the program to be scaled down to the processors actually available at run time. Usefulness of this scheduling heuristic has been demonstrated by incorporating the scheduler in the compiler backend for targeting Sisal (Streams and Iterations in a Single Assignment Language) on iPSC/860. >

Collaborative multi-swarm PSO for task matching using graphics processing units

Abstract: We investigate the performance of a highly parallel Particle Swarm Optimization (PSO) algorithm implemented on the GPU. In order to achieve this high degree of parallelism we implement a collaborative multi-swarm PSO algorithm on the GPU which relies on the use of many swarms rather than just one. We choose to apply our PSO algorithm against a real-world application: the task matching problem in a heterogeneous distributed computing environment. Due to the potential for large problem sizes with high dimensionality, the task matching problem proves to be very thorough in testing the GPUs capabilities for handling PSO. Our results show that the GPU offers a high degree of performance and achieves a maximum of 37 times speedup over a sequential implementation when the problem size in terms of tasks is large and many swarms are used.