Peer-to-peer redundant file server system and methods include clients that determine a target storage provider to contact for a particular storage transaction based on a pathname provided by the filesystem and a predetermined scheme such as a hash function applied to a portion of the pathname. Servers use the same scheme to determine where to store relevant file information so that the clients can locate the file information. The target storage provider may store the file itself and/or may store metadata that identifies one or more other storage providers where the file is stored. A file may be replicated in multiple storage providers, and the metadata may include a list of storage providers from which the clients can select (e.g., randomly) in order to access the file.

Peer-to-peer redundant file server system and methods

Described in detail herein are systems and methods for single instancing blocks of data in a data storage system. For example, the data storage system may include multiple computing devices (e.g., client computing devices) that store primary data. The data storage system may also include a secondary storage computing device, a single instance database, and one or more storage devices that store copies of the primary data (e.g., secondary copies, tertiary copies, etc.). The secondary storage computing device receives blocks of data from the computing devices and accesses the single instance database to determine whether the blocks of data are unique (meaning that no instances of the blocks of data are stored on the storage devices). If a block of data is unique, the single instance database stores it on a storage device. If not, the secondary storage computing device can avoid storing the block of data on the storage devices.

Block-level single instancing

The present disclosure relates to a distributed information storage system which functions as virtual cloud storage overlay on top of physical cloud storage systems. The disclosure discloses transparently solving all the data management related security, virtualization, reliability and enables transparent cloud storage migration, cloud storage virtualization, information dispersal and integration across disparate cloud storage devices operated by different providers or on-premise storage. The cloud storage is owned or hosted by same or different third-party providers who own the information contained in the storage which eliminates cloud dependencies. This present disclosure functions as a distributed cloud storage delivery platform enabling various functionalities like cloud storage virtualization, cloud storage integration, cloud storage management and cloud level RAID.

Distributed virtual storage cloud architecture and a method thereof

Operating profiles for consumers of computing resources may be automatically determined based on an analysis of actual resource usage measurements and other operating metrics. Measurements may be taken while a consumer, such as a virtual machine instance, uses computing resources, such as those provided by a host. A profile may be dynamically determined based on those measurements. Profiles may be generalized such that groups of consumers with similar usage profiles are associated with a single profile. Assignment decisions may be made based on the profiles, and computing resources may be reallocated or oversubscribed if the profiles indicate that the consumers are unlikely to fully utilize the resources reserved for them. Oversubscribed resources may be monitored, and consumers may be transferred to different resource providers if contention for resources is too high.

Automated profiling of resource usage

A system for exposing data stored in a cloud computing system to a content delivery network provider includes a database configured to receive and store metadata about the data, the database being implemented in the cloud computing system to store configuration metadata for the data related to the content delivery network, and an origin server configured to receive requests for the data from the content delivery network provider, and configured to provide the data to the content delivery network provider based on the metadata.

System and Method for Exposing Cloud Stored Data to a Content Delivery Network

A method and system is disclosed for providing a cloud storage system supporting existing APIs and protocols. The method of storing cloud storage system (CSS) object metadata separates object metadata that describes each CSS object as a collection of named chunks with chunk locations specified as a separate part of the metadata. Chunks are identified using globally unique permanent identifiers that are never re-used to identify different chunk payload. While avoiding the bottleneck of a single metadata server, the disclosed system provides ordering guarantees to clients such as guaranteeing access to the most recent version of an object. The disclosed system also provides end-to-end data integrity protection, inline data deduplication, configurable replication, hierarchical storage management and location-aware optimization of chunk storage.

Cloud storage system with distributed metadata

A method and system for providing unified local storage support for file and cloud access is disclosed. The method comprises writing a chunk on a storage server, and replicating the chunk to other selected storage servers when necessary. The method and system further comprise writing a version manifest on the storage server; replicating the version manifest to other selected storage servers when necessary. Object puts or appends are implemented by first chunking the object, determining if the chunks are new, transferring the chunks if required, followed by creation of a new version manifest referencing the chunks. Finally, the method and system include providing concurrent file-oriented read and write access consistent with the stored version manifests and chunks.

Unified local storage supporting file and cloud object access

Certain aspects of a method and system for deferred pinning of host memory for stateful network interfaces are disclosed. Aspects of a method may include dynamically pinning by a convergence network interface controller (CNIC), at least one of a plurality of memory pages. At least one of: a hypervisor and a guest operating system (GOS) may be prevented from swapping at least one of the dynamically pinned plurality of memory pages.

Method and system for deferred pinning of host memory for stateful network interfaces

A method for communication includes inputting from a host processor to a network interface device a sequence of work requests indicative of operations to be carried out by the network interface device with respect to a plurality of the connections. The device looks ahead through the sequence in order to identify at least first and second operations that are to be carried out with respect to one of the connections in response to first and second work requests, respectively, wherein the second work request does not immediately follow the first work request in the sequence. The device loads the context data for the one of the connections from a host memory into a context cache, and performs at least the first and second operations sequentially while the context data are held in the cache.

Efficient handling of work requests in a network interface device

Certain aspects of a method and system for transparent TCP offload with a user space library are disclosed Aspects of a method may include collecting TCP segments in a network interface card (NIC) without transferring state information to a host system When an event occurs that terminates the collection of TCP segments, a single aggregated TCP segment based on the collected TCP segments may be generated The aggregated TCP segment may be posted directly to a user space library, bypassing kernel processing of the aggregated TCP segment

Caitlin Bestler

Papers

Cloud storage system with distributed metadata

Unified local storage supporting file and cloud object access

Method and system for deferred pinning of host memory for stateful network interfaces

Efficient handling of work requests in a network interface device

Method and system for transparent TCP offload (TTO) with a user space library