scispace - formally typeset
Proceedings ArticleDOI

A distributed shared buffer space for data-intensive applications

TLDR
This work proposes separating control and data transfer traffic by accessing data through a DSM-like cluster-wide shared buffer space and only including buffer references in the control messages, using a generic API for accessing buffers.
Abstract
Efficient memory allocation and data transfer for cluster-based data-intensive applications is a difficult task. Both changes in cluster interconnects and application workloads usually require timing of the application and network code. We propose separating control and data transfer traffic by accessing data through a DSM-like cluster-wide shared buffer space and only including buffer references in the control messages. Using a generic API for accessing buffers allows for tuning data transfer without changing the application code. A prototype, implemented in the context of a distributed storage system, has been validated with several networking technologies, showing that such a framework can combine performance and flexibility.

read more

Citations
More filters
Patent

Virtual disk storage techniques

TL;DR: In this paper, the authors describe techniques for storing virtual disk payload data in an exemplary configuration, each virtual disk extent can be associated with state information that indicates whether the virtual disk file is described by a virtual disk.
Patent

Virtualization and offload reads and writes

TL;DR: In this paper, an offload read allows a requestor to obtain a token that represents data while an offloading write allows the requestors to request that the data (or a part thereof) represented by a token be logically written.
Patent

Offload Read and Write Offload Provider

TL;DR: In this article, an offload provider may provide a token that represents data, and the offload providers may be expected to ensure that the data the token represents will not change while the token is valid.
Patent

Compatibly extending offload token size

TL;DR: In this paper, the authors describe a mechanism that allows an offload provider to use larger tokens, such as physical or virtual tokens, in response to offload read and write commands.
References
More filters
Proceedings ArticleDOI

Cooperative caching: using remote client memory to improve file system performance

TL;DR: In this article, the authors examine four cooperative caching algorithms using a trace-driven simulation study and conclude that cooperative caching can significantly improve file system read response time and that relatively simple cooperative caching methods are sufficient to realize most of the potential performance gain.
Proceedings ArticleDOI

Implementing global memory management in a workstation cluster

TL;DR: The objective is to use a single, unified, but distributed memory management algorithm at the lowest level of the operating system to manage memory globally at this level so that all system- and higher-level software, including VM, file systems, transaction systems, and user applications, can benefit from available cluster memory.
Journal ArticleDOI

Disk-directed I/O for MIMD multiprocessors

TL;DR: In this article, the authors proposed a disk-directed I/O technique, which allows the disk servers to determine the flow of data for maximum performance, and demonstrated that this technique provided consistent high performance that was largely independent of data distribution.
Book

The virtual interface architecture

TL;DR: In this paper, a protected, zero-copy, user-level network interface architecture is proposed to reduce the system overhead for sending and receiving messages between high-performance CPU/memory subsystems and networks to less than 10 microseconds.
Proceedings ArticleDOI

IO-lite: a unified I/O buffering and caching system

TL;DR: IO-Lite eliminates all copying and multiple buffering of I/O data, and enables various cross-subsystem optimizations, and shows performance improvements between 40 and 80% on real workloads as a result of IO -Lite.