Showing papers on "Task (computing) published in 2015"

Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval

Abstract: Methods of deep neural networks (DNNs) have recently demonstrated superior performance on a number of natural language processing tasks. However, in most previous work, the models are learned based on either unsupervised objectives, which does not directly optimize the desired task, or singletask supervised objectives, which often suffer from insufficient training data. We develop a multi-task DNN for learning representations across multiple tasks, not only leveraging large amounts of cross-task data, but also benefiting from a regularization effect that leads to more general representations to help tasks in new domains. Our multi-task DNN approach combines tasks of multiple-domain classification (for query classification) and information retrieval (ranking for web search), and demonstrates significant gains over strong baselines in a comprehensive set of domain adaptation.

Curriculum learning of multiple tasks

Abstract: Sharing information between multiple tasks enables algorithms to achieve good generalization performance even from small amounts of training data. However, in a realistic scenario of multi-task learning not all tasks are equally related to each other, hence it could be advantageous to transfer information only between the most related tasks. In this work we propose an approach that processes multiple tasks in a sequence with sharing between subsequent tasks instead of solving all tasks jointly. Subsequently, we address the question of curriculum learning of tasks, i.e. finding the best order of tasks to be learned. Our approach is based on a generalization bound criterion for choosing the task order that optimizes the average expected classification performance over all tasks. Our experimental results show that learning multiple related tasks sequentially can be more effective than learning them jointly, the order in which tasks are being solved affects the overall performance, and that our model is able to automatically discover a favourable order of tasks.

A Server-Assigned Spatial Crowdsourcing Framework

Abstract: With the popularity of mobile devices, spatial crowdsourcing is rising as a new framework that enables human workers to solve tasks in the physical world. With spatial crowdsourcing, the goal is to crowdsource a set of spatiotemporal tasks (i.e., tasks related to time and location) to a set of workers, which requires the workers to physically travel to those locations in order to perform the tasks. In this article, we focus on one class of spatial crowdsourcing, in which the workers send their locations to the server and thereafter the server assigns to every worker tasks in proximity to the worker’s location with the aim of maximizing the overall number of assigned tasks. We formally define this maximum task assignment (MTA) problem in spatial crowdsourcing, and identify its challenges. We propose alternative solutions to address these challenges by exploiting the spatial properties of the problem space, including the spatial distribution and the travel cost of the workers. MTA is based on the assumptions that all tasks are of the same type and all workers are equally qualified in performing the tasks. Meanwhile, different types of tasks may require workers with various skill sets or expertise. Subsequently, we extend MTA by taking the expertise of the workers into consideration. We refer to this problem as the maximum score assignment (MSA) problem and show its practicality and generality. Extensive experiments with various synthetic and two real-world datasets show the applicability of our proposed framework.

Development and Evaluation of Search Tasks for IIR Experiments using a Cognitive Complexity Framework

Abstract: One of the most challenging aspects of designing interactive information retrieval (IIR) experiments with users is the development of search tasks. We describe an evaluation of 20 search tasks that were designed for use in IIR experiments and developed using a cognitive complexity framework from educational theory. The search tasks represent five levels of cognitive complexity and four topical domains. The tasks were evaluated in the context of a laboratory IIR experiment with 48 participants. Behavioral and self-report data were used to characterize and understand differences among tasks. Results showed more cognitively complex tasks required significantly more search activity from participants (e.g., more queries, clicks, and time to complete). However, participants did not evaluate more cognitively complex tasks as more difficult and were equally satisfied with their performances across tasks. Our work makes four contributions: (1) it adds to what is known about the relationship among task, search behaviors and user experience; (2) it presents a framework for task creation and evaluation; (3) it provides tasks and questionnaires that can be reused by others and (4) it raises questions about findings and assumptions of many recent studies that only use behavioral signals from search logs as evidence for task difficulty and searcher satisfaction, as many of our results directly contradict these findings.

Integrating Containers into Workflows: A Case Study Using Makeflow, Work Queue, and Docker

Abstract: Workflows are a widely used abstraction for representing large scientific applications and executing them on distributed systems such as clusters, clouds, and grids. However, workflow systems have been largely silent on the question of precisely what environment each task in the workflow is expected to run in. As a result, a workflow may run correctly in the environment in which it was designed, but when moved to another machine, is highly likely to fail due to differences in the operating system, installed applications, available data, and so forth. Lightweight container technology has recently arisen as a potential solution to this problem, by providing a well-defined execution environments at the operating system level. In this paper, we consider how to best integrate container technology into an existing workflow system, using Makeflow, Work Queue, and Docker as examples of current technology. A brief performance study of Docker shows very little overhead in CPU and I/O performance, but significant costs in creating and deleting containers. Taking this into account, we describe four different methods of connecting containers to different points of the infrastructure, and explain several methods of managing the container images that must be distributed to executing tasks. We explore the performance of a large bioinformatics workload on a Docker-enabled cluster, and observe the best configuration to be locally-managed containers that are shared between multiple tasks.

SCREAM: sketch resource allocation for software-defined measurement

Abstract: Software-defined networks can enable a variety of concurrent, dynamically instantiated, measurement tasks, that provide fine-grain visibility into network traffic. Recently, there have been many proposals for using sketches for network measurement. However, sketches in hardware switches use constrained resources such as SRAM memory, and the accuracy of measurement tasks is a function of the resources devoted to them on each switch. This paper presents SCREAM, a system for allocating resources to sketch-based measurement tasks that ensures a user-specified minimum accuracy. SCREAM estimates the instantaneous accuracy of tasks so as to dynamically adapt the allocated resources for each task. Thus, by finding the right amount of resources for each task on each switch and correctly merging sketches at the controller, SCREAM can multiplex resources among network-wide measurement tasks. Simulations with three measurement tasks (heavy hitter, hierarchical heavy hitter, and super source/destination detection) show that SCREAM can support more measurement tasks with higher accuracy than existing approaches.

LIBRA: Lightweight Data Skew Mitigation in MapReduce

Abstract: MapReduce is an effective tool for parallel data processing. One significant issue in practical MapReduce applications is data skew: the imbalance in the amount of data assigned to each task. This causes some tasks to take much longer to finish than others and can significantly impact performance. This paper presents LIBRA, a lightweight strategy to address the data skew problem among the reducers of MapReduce applications. Unlike previous work, LIBRA does not require any pre-run sampling of the input data or prevent the overlap between the map and the reduce stages. It uses an innovative sampling method which can achieve a highly accurate approximation to the distribution of the intermediate data by sampling only a small fraction of the intermediate data during the normal map processing. It allows the reduce tasks to start copying as soon as the chosen sample map tasks (only a small fraction of map tasks which are issued first) complete. It supports the split of large keys when application semantics permit and the total order of the output data. It considers the heterogeneity of the computing resources when balancing the load among the reduce tasks appropriately. LIBRA is applicable to a wide range of applications and is transparent to the users. We implement LIBRA in Hadoop and our experiments show that LIBRA has negligible overhead and can speed up the execution of some popular applications by up to a factor of 4.

Multi-robot auctions for allocation of tasks with temporal constraints

Abstract: We propose an auction algorithm to allocate tasks that have temporal constraints to cooperative robots. Temporal constraints are expressed as time windows, within which a task must be executed. There are no restrictions on the time windows, which are allowed to overlap. Robots model their temporal constraints using a simple temporal network, enabling them to maintain consistent schedules. When bidding on a task, a robot takes into account its own current commitments and an optimization objective, which is to minimize the time of completion of the last task alone or in combination with minimizing the distance traveled. The algorithm works both when all the tasks are known upfront and when tasks arrive dynamically. We show the performance of the algorithm in simulation with different numbers of tasks and robots, and compare it with a baseline greedy algorithm and a state-of-the-art auction algorithm. Our algorithm is computationally frugal and consistently allocates more tasks than the competing algorithms.

A scalable architecture for ordered parallelism

Abstract: We present Swarm, a novel architecture that exploits ordered irregular parallelism, which is abundant but hard to mine with current software and hardware techniques. In this architecture, programs consist of short tasks with programmer-specified timestamps. Swarm executes tasks speculatively and out of order, and efficiently speculates thousands of tasks ahead of the earliest active task to uncover ordered parallelism. Swarm builds on prior TLS and HTM schemes, and contributes several new techniques that allow it to scale to large core counts and speculation windows, including a new execution model, speculation-aware hardware task management, selective aborts, and scalable ordered commits. We evaluate Swarm on graph analytics, simulation, and database benchmarks. At 64 cores, Swarm achieves 51–122 × speedups over a single-core system, and outperforms software-only parallel algorithms by 3–18 ×.

Managed services coordinator

Abstract: A method is provided for applying tasks. The method can include receiving a plurality of tasks for a plurality of applications, an application of the plurality of applications is hosted by a computing environment utilizing a monitoring agent, and a task is performed for an operation of the application during a first period of time. The method can also include determining a first resource type and a first resource amount from the computing environment for performing the task. The method can also include grouping the plurality of tasks into a set of tasks based on the first resource type, at least two tasks from the set of tasks being performable in parallel during the first period of time. The method can also include determining whether the first resource amount of the first resource type is present in the computing environment during the first period of time.

Containerized Computational Task Execution Management Using a Secure Distributed Transaction Ledger

Abstract: A peer-to-peer network includes a cryptocurrency system with a block chain, a client system coupled to the cryptocurrency system, and a host system coupled to the cryptocurrency system The client system publishes a container execution request that includes information associating the client system with a containerized computational task, the cryptocurrency system incorporates the container execution request into a block chain of a cryptocurrency system, and the host system receives the container execution request via the block chain, retrieves the containerized computational task, and executes the containerized computational task

Representation Learning Using Multi-Task Deep Neural Networks

Abstract: A system may comprise one or more processors and memory storing instructions that, when executed by one or more processors, configure one or more processors to perform a number of operations or tasks, such as receiving a query or a document, and mapping the query or the document into a lower dimensional representation by performing at least one operational layer that shares at least two disparate tasks.

Control system for an automotive vehicle

Abstract: A control system for a vehicle having a plurality of subsystems in which each subsystem includes a sensor and/or actuator. A general purpose processor is programmed with a plurality of software applications in which each software application is associated with one of the vehicle subsystems. The subsystems and processor communicate with each other through an electrical bus and each subsystem includes a communication interface to receive commands from the processor as well as to send task signals to the processor. A task arbitrator prioritizes the order of execution of multiple task signals received by the processor during a preset time period. The general purpose processor then initiates execution of the software application associated with the subsystem of the sensor signal received from the task arbitrator and thereafter transmits a control signal to one or more of the subsystems via the bus as a result of execution of the associated software application.

Utility Functions and Resource Management in an Oversubscribed Heterogeneous Computing Environment

Abstract: We model an oversubscribed heterogeneous computing system where tasks arrive dynamically and a scheduler maps the tasks to machines for execution. The environment and workloads are based on those being investigated by the Extreme Scale Systems Center at Oak Ridge National Laboratory. Utility functions that are designed based on specifications from the system owner and users are used to create a metric for the performance of resource allocation heuristics. Each task has a time-varying utility (importance) that the enterprise will earn based on when the task successfully completes execution. We design multiple heuristics, which include a technique to drop low utility-earning tasks, to maximize the total utility that can be earned by completing tasks. The heuristics are evaluated using simulation experiments with two levels of oversubscription. The results show the benefit of having fast heuristics that account for the importance of a task and the heterogeneity of the environment when making allocation decisions in an oversubscribed environment. The ability to drop low utility-earning tasks allow the heuristics to tolerate the high oversubscription as well as earn significant utility.

Workload Estimation for Improving Resource Management Decisions in the Cloud

Abstract: In cloud computing, good resource management can benefit both cloud users as well as cloud providers. Workload prediction is a crucial step towards achieving good resource management. While it is possible to estimate the workloads of long-running tasks based on the periodicity in their historical workloads, it is difficult to do so for tasks which do not have such recurring workload patterns. In this paper, we present an innovative clustering based resource estimation approach which groups tasks that have similar characteristics into the same cluster. The historical workload data for tasks in a cluster are used to estimate the resources needed by new tasks based on the cluster(s) to which they belong. In particular, for a new task T, we measure T's initial workload and predict to which cluster(s) it may belong. Then, the workload information of the cluster(s) is used to estimate the workload of T. The approach is experimentally evaluated using Google dataset, including resource usage data of over half a million tasks. We develop a workload model based on the dataset which is then used to estimate the workload patterns of several randomly selected tasks from the trace log. The results confirm the effectiveness of this cluster-based method for estimating the resources required by each task.

Automatic error recovery in robot assembly operations using reverse execution

Abstract: Robotic assembly tasks are in general difficult to program and require a high degree of precision. As the complexity of the task increases it becomes increasingly unlikely that tasks can always be executed without errors. Preventing errors beyond a certain point is economically infeasible, in particular for small-batch productions. As an alternative, we propose a system for automatically handling certain classes of errors instead of preventing them. Specifically, we show that many operations can be automatically reversed. Errors can be handled through automatic reverse execution of the control program to a safe point, from which forward execution can be resumed. This paper describes the principles behind automatic reversal of robotic assembly operations, and experimentally demonstrates the use of a domain-specific language that supports automatic error handling through reverse execution. Our contribution represents the first experimental demonstration of reversible computing principles applied to industrial robotics.

Timing Analysis of Fixed Priority Self-Suspending Sporadic Tasks

Abstract: Many real-time systems include tasks that need to suspend their execution in order to externalize some of their operations or to wait for data, events or shared resources. Although commonly encountered in real-world systems, study of their timing analysis is still limited due to the problem complexity. In this paper, we invalidate a claim made in one of the earlier works [1], that led to the common belief that the timing analysis of one self-suspending task interacting with non-self suspending sporadic tasks is much easier than in the periodic case. This work highlights the complexity of the problem and presents a method to compute the exact worst-case response time (WCRT) of a self-suspending task with one suspension region. However, as the complexity of the analysis might rapidly grow with the number of tasks, we also define an optimization formulation to compute an upper-bound on the WCRT for tasks with multiple suspendion regions. In the experiments, our optimization framework outperforms all previous analysis techniques and often finds the exact WCRT.

Scalable Spatial Crowdsourcing: A Study of Distributed Algorithms

Abstract: Recently spatial crowd sourcing was introduced as a natural extension to traditional crowd sourcing allowing for tasks to have a geospatial component, i.e., A task can only be performed if a worker is physically present at the location of the task. The problem of assigning spatial tasks to workers in a spatial crowd sourcing system can be formulated as a weighted bipartite b-matching graph problem that can be solved optimally by existing methods for the minimum cost maximum flow problem. However, these methods are still too complex to run repeatedly for an online system, especially when the number of incoming workers and tasks increases. Hence, we propose a class of approaches that utilizes an online partitioning method to reduce the problem space across a set of cloud servers to construct independent bipartite graphs and solve the assignment problem in parallel. Our approaches solve the spatial task assignment approximately but competitive to the exact solution. We experimentally verify that our approximate approaches outperform the centralized and Map Reduce version of the exact approach with acceptable accuracy and thus suitable for online spatial crowd sourcing at scale.

Dynamically hierarchical resource-allocation algorithm in cloud computing environment

Abstract: The development of big data challenges the computing power and communication capability of cloud architecture, but traditional resource-allocation algorithms perform poorly due to the large-scale communication among cloud nodes. In this paper, a dynamically hierarchical, resource-allocation algorithm is proposed for multiple cloud nodes collaborating in big data environment. Using fuzzy pattern recognition, the algorithm dynamically divides tasks and nodes into different levels based on computing power and storage factors. Thus a dynamically adjusted mapping is generated between tasks and nodes. When a new task arrives, only the nodes corresponding to the task level join in the bid. The algorithm takes advantages of dynamical hierarchy to reduce the communication traffic during resource allocation. Both theoretical and experimental results illustrate that the proposed algorithm outperforms the MinMin algorithm in terms of communication traffic and makespan.

Cross-domain colony computing resource gathering and distributing method

Abstract: The invention relates to a cross-domain colony computing resource gathering and distributing method. The cross-domain colony computing resource gathering and distributing method comprises the following steps of establishing a parallel computing management platform; constructing a cross-domain distributed multistage colony resource pool environment; registering and updating colony resources; distributing colony task resources; and submitting colony tasks and recycling a result. The method reflects an overall situation type colony resource managing, controlling and sharing technology. In actual running, the number of computing nodes of a distributed type colony is limited, when a task load is large, queue-up computing is required, the circumstance that a local colony is busy while other colonies are unoccupied occurs, if a plurality of distributed type colonies are joined up and resources are shared, tasks of the busy colonies can be transferred to the unoccupied colonies, and the processing speed of batch tasks can be greatly increased.

Bayesian Methods for Intelligent Task Assignment in Crowdsourcing Systems

Abstract: In many decision-making scenarios, it is necessary to aggregate information from a number of different agents, be they people, sensors or computer systems. Each agent may have complementary analysis skills or access to different information, and their reliability may vary greatly. An example is using crowdsourcing to employ multiple human workers to perform analytical tasks. This chapter presents an information-theoretic approach to selecting informative decision-making agents, assigning them to specific tasks and combining their responses using a Bayesian method. For settings in which the agents are paid to undertake tasks, we introduce an automated algorithm for selecting a cohort of agents (workers) to complete informative tasks, hiring new members of the cohort and identifying those members whose services are no longer needed. We demonstrate empirically how our intelligent task assignment approach improves the accuracy of combined decisions while requiring fewer responses from the crowd.

Memory-processor co-scheduling in fixed priority systems

Abstract: A major obstacle towards the adoption of multi-core platforms for real-time systems is given by the difficulties in characterizing the interference due to memory contention. The simple fact that multiple cores may simultaneously access shared memory and communication resources introduces a significant pessimism in the timing and schedulability analysis. To counter this problem, predictable execution models have been proposed splitting task executions into two consecutive phases: a memory phase in which the required instruction and data are pre-fetched to local memory (M-phase), and an execution phase in which the task is executed with no memory contention (C-phase). Decoupling memory and execution phases not only simplifies the timing analysis, but it also allows a more efficient (and predictable) pipelining of memory and execution phases through proper co-scheduling algorithms.In this paper, we take a further step towards the design of smart co-scheduling algorithms for sporadic real-time tasks complying with the M/C (memory-computation) model. We provide a theoretical framework that aims at tightly characterizing the schedulability improvement obtainable with the adopted M/C task model on a single-core systems. We identify a tight critical instant for M/C tasks scheduled with fixed priority, providing an exact response-time analysis with pseudo-polynomial complexity. We show in our experiments that a significant schedulability improvement may be obtained with respect to classic execution models, placing an important building block towards the design of more efficient partitioned multi-core systems.

Multi-task model and feature joint learning

Abstract: Given several tasks, multi-task learning (MTL) learns multiple tasks jointly by exploring the interdependence between them. The basic assumption in MTL is that those tasks are indeed related. Existing MTL methods model the task relatedness/ interdependence in two different ways, either common parameter-sharing or common featuresharing across tasks. In this paper, we propose a novel multi-task learning method to jointly learn shared parameters and shared feature representation. Our objective is to learn a set of common features with which the tasks are related as closely as possible, therefore common parameters shared across tasks can be optimally learned. We present a detailed deviation of our multi-task learning method and propose an alternating algorithm to solve the non-convex optimization problem. We further present a theoretical bound which directly demonstrates that the proposed multi-task learning method can successfully model the relatedness via joint common parameter- and common feature-learning. Extensive experiments are conducted on several real world multi-task learning datasets. All results demonstrate the effectiveness of our multitask model and feature joint learning method.

Crowd-Selection Query Processing in Crowdsourcing Databases: A Task-Driven Approach.

Abstract: Crowd-selection is essential to crowdsourcing applications, since choosing the right workers with particular expertise to carry out specific crowdsourced tasks is extremely important. The central problem is simple but tricky: given a crowdsourced task, who is the right worker to ask? Currently, most existing work has mainly studied the problem of crowd-selection for simple crowdsourced tasks such as decision making and sentiment analysis. Their crowd- selection procedures are based on the trustworthiness of workers. However, for some complex tasks such as document review and question answering, selecting workers based on the latent category of tasks is a better solution. In this paper, we formulate a new problem of task-driven crowd- selection for complex tasks. We first develop a bayesian gener- ative model to exploit "who knows what" for the workers in the crowdsourcing environment. The model provides a principle and natural framework for capturing the latent skills of workers as well as the latent categories of crowdsourced tasks. The inference of the latent skills of workers is based on past resolved crowdsourced tasks with feedback scores. We assume that the feedback scores can illustrate the performance of the workers for the tasks. We then devise a variational algorithm that transforms the latent skill infer- ence with the proposed model into a standard optimization prob- lem, which can be solved efficiently. We verify the performance of our method through extensive experiments on the data collected from three well-known crowdsourcing platforms for question an- swering tasks such as Quora, Yahoo ! Answer and Stack Overflow.

Fast and High Quality Topology-Aware Task Mapping

Abstract: Considering the large number of processors and the size of the interconnection networks on exactable-capable supercomputers, mapping concurrently executable and communicating tasks of an application is complex problem that needs to be dealt with care. For parallel applications, the communication overhead can be a significant bottleneck on scalability. Topology-aware task-mapping methods that map the tasks tithe processors~(i.e., cores) by exploiting the underlying network information are very effective to avoid, or at worst bend, this limitation. We propose novel, efficient, and effective task mapping algorithms employing a graph model. The experiments show that the methods are faster than the existing approaches proposed for the same task, and on 4096 processors, the algorithms improve the communication hops and link contentions by 16% and 32%, respectively, on the average. In addition, they improve the average execution time of a parallel Spiv kernel and a communication-only application by 9% and 14%, respectively.

MEDUSA: A Predictable and High-Performance DRAM Controller for Multicore Based Embedded Systems

Abstract: Commercial-Off-The-Shelf (COTS) DRAM controllers are optimized for high memory throughput, but they do not provide predictable timing among memory requests from different cores in multicore systems. Therefore, memory requests from a critical real-time task on one core can be substantially delayed by memory requests from on-real-time tasks on the other cores. In this work, we propose a DRAM controller design, called MEDUSA, to provide predictable memory performance in multicore based real-time systems. MEDUSA can provide high time predictability when needed for real-time tasks but also strive to provide high average performance for non-real-time tasks through a close collaboration between the OSand the DRAM controller. In our approach, the OS partially partitions DRAM banks into two groups: reserved banks and shared banks. The reserved banks are exclusive to each core to provide predictable timing while the shared banks are shared by all cores to efficiently utilize the resources. MEDUSA has two separate queues for read and write requests, and it prioritizes reads over writes. In processing read requests, MEDUSA employs a two-level scheduling algorithm that prioritizes the memory requests to the reserved banks in a Round Robin fashion to provide strong timing predictability. In processing write requests, MEDUSA largely relies on the FR-FCFS for high throughput but makes an immediate switch to read upon arrival of read requests to the reserved banks. We implemented MEDUSA in a Gem5 full-system simulator and a Linux kernel and performed experiments using a set of synthetic and SPEC2006 benchmarks to analyze the performance impact of MEDUSA on both real-time and non-real-time tasks. The results show that MEDUSA achieves up to 95% better worst-case performance for real-time tasks while achieving up to 31% throughput improvement for non-real-time tasks.

Bicyclists' speed adaptation strategies when conducting self-paced vs. system-paced smartphone tasks in traffic

Abstract: The increasing prevalence of mobile phone usage while cycling has raised concerns, even though the number of cyclists involved in accidents does not increase at a comparable rate. A reason for this may be how cyclists adapt travelling speed and task execution to the current traffic situation. The aim of this study is to investigate speed adaptation among cyclists when conducting self-paced (initiated by the cyclist) vs. system-paced (initiated by somebody else) smartphone tasks in real traffic. Twenty-two cyclists completed a track in real traffic while listening to music, receiving and making calls, receiving and sending text messages, and searching for information on the internet. The route and the types of tasks were controlled, but the cyclists could choose rather freely when and where along the route to carry out the tasks, thus providing semi-naturalistic data on compensatory behaviour. The results clearly show that cyclists use conscious strategies to adapt their speed to accommodate the execution of secondary phone tasks. Regarding tactical behaviour, it was found that cyclists kept on cycling in 80% of the system-paced cases and in 70% of the self-paced cases. In the remaining cases, the cyclists chose to execute the phone task while standing still or when walking. Compared to the baseline (17.6 ± 3.5 km/h), the mean speed was slightly increased when the cyclists listened to music (18.2 ± 3.7 km/h) and clearly decreased when they interacted with the phone (13.0 ± 5.0 km/h). The speed reduction profile differed between self-paced and system-paced tasks with a preparatory speed reduction before task initiation for self-paced tasks. In conclusion, when the cyclists had the chance they either stopped or adapted their speed proactively to accommodate the execution of the phone task. For self-paced tasks, the speed reduction was finalised before task initialisation, and for system-paced tasks the speed adaptation occurred in reaction to the incoming task. It is recommended to investigate whether the observed compensatory behaviour is enough to offset the possible negative effects of smartphone use.

Determining storage tiers for placement of data sets during execution of tasks in a workflow

Abstract: Provided are a computer program product, system, and method for determining storage tiers for placement of data sets during execution of tasks in a workflow. A representation of a workflow execution pattern of tasks for a job indicates a dependency of the tasks and data sets operated on by the tasks. A determination is made of an assignment of the data sets for the tasks to a plurality of the storage tiers based on the dependency of the tasks indicated in the workflow execution pattern. A moving is scheduled of a subject data set of the data sets operated on by a subject task of the tasks that is subject to an event to an assigned storage tier indicated in the assignment for the subject task subject. The moving of the data set is scheduled to be performed in response to the event with respect to the subject task.

Pipeline execution of multiple map-reduce jobs

Abstract: Some examples include a pipeline manager that creates pipeline queues across data nodes in a cluster. The pipeline manager may assign a pipeline queue connection to contiguous map-reduce jobs so that a reduce task of a first map-reduce job sends data to the pipeline queue, and a map task of a second map-reduce job receives the data from the pipeline queue. Thus, the map task of the second job may begin using the data from the reduce task of the first job prior to completion of the first job. Furthermore, in some examples, the data nodes in the cluster may monitor access success to individual pipeline queues. If the access attempts to access successes exceeds a threshold, a data node may request an additional pipeline queue connection for a task. Additionally, if a failure occurs, information maintained at a data node may be used by the pipeline manager for recovery.