The invention provides a method and system for creating an innovative file system that separates its directory presentation from its data store. The method and system include processing, division, distribution, managing, synchronizing, and reassembling of file system objects that does not delay the presentation of the content to the user, but also uses a reduced amount of storage space. The invention includes the ability to manage and control the integrity of the files distributed across the network, and the ability to serve and reconstruct files in real time using a Virtual File Control System.

Method and system for managing distributed content and related metadata

An improved system for accessing data within a distributed data storage network (“DDSN”) is disclosed. In a system implementing the disclosed invention, traffic is routed to individual slice servers within the DDSN in accordance with objective criteria as well as user-defined policies. In accordance with one aspect of the disclosed invention, when a data segment is written to a DDSN, the segment is divided into multiple data slices, which are simultaneously transmitted to different slice servers. In accordance with another aspect of the disclosed invention, when a data segment is read from a DDSN, a list of slice servers, each containing a data slice that could be used to reconstruct the requested data segment, is assembled, and sorted in accordance with a preference rating assigned to each of the slice servers. Sufficient data slices to reconstruct the data segment are then read in accordance with the preference ranking of the slice servers.

Smart access to a dispersed data storage network

A billing process is disclosed for an information dispersal system or digital data storage system. The original data to be stored is separated into a number of data 'slices' or shares (22, 24, 26, 28, 30, and 32). These data subsets are stored on separate digital data storage devices (34, 36, 38, 40, 42, and 44) as a way of increasing privacy and security. A set of metadata tables are created, separate from the dispersed file share storage, to maintain information about the original data size of each block, file or set of file shares dispersed on the grid.

Billing system for information dispersal system

A system, method, and apparatus for implementing a plurality of dispersed data storage networks using a set of slice servers are disclosed. A plurality of information records are maintained, with each information record corresponding to a dispersed data storage network. The information record maintains what slice servers are used to implement the dispersed data storage network, as well as other information needed to administer a DDSN, such as the information dispersal algorithm used, how data is stored, and whether data is compressed or encrypted.

Virtualized data storage vaults on a dispersed data storage network

An efficient method for breaking source data into smaller data subsets and storing those subsets along with coded information about some of the other data subsets on different storage nodes such that the original data can be recreated from a portion of those data subsets in an efficient manner.

Digital data storage system

The present invention provides a system and apparatus for efficient and reliable, control and distribution of data files or portions of files, applications, or other data objects in large-scale distributed networks. A unique content-management front-end provides efficient controls for triggering distribution of digitized data content to selected groups of a large number of remote computer servers. Transport-layer protocols interact with distribution controllers to automatically determine an optimized tree-like distribution sequence to group leaders selected by network devices at each remote site. Reliable store-and-forward transfer to clusters is accomplished using a unicast protocol in the ordered tree sequence. Once command messages and content arrive at all participating group leaders, local hybrid multicast protocols efficiently and reliably distribute them to the back-end nodes for interpretation and execution. Positive acknowledgement is then sent back to the content manager from each group leader, and the updated content in each remote device autonomously goes 'live' when the content change is locally completed.

Method and apparatus for reliable and scalable distribution of data files in distributed networks

A system and apparatus for control and distribution of data files in large-scale distributed networks. Members of a group of servers in a multicast network elect a group leader whenever a new group leader is required, as detected by absence of a periodic message published by the leader. Election is carried out by a system of voting by each candidate whereby each candidate has a priority calculated from its configuration, and the server with the highest priority is configured to claim the leadership faster than the other candidates. As part of the claim, each candidate multicasts its priority. Each candidate that receives a multicast claim for leadership from another candidate compares its own priority against the claimant and only votes for itself if its own priority is higher. After a preconfigured period of hearing no other claimants with higher priority, the candidate with the highest priority becomes the new leader.

Method and apparatus for election of group leaders in a distributed network

The present invention provides a system and apparatus for efficient and reliable, control and distribution of data files or portions of files, applications, or other data objects in large-scale distributed networks. A unique content-management front-end provides efficient controls for triggering distribution of digitized data content to selected groups of a large number of remote computer servers. Network distribution messages are dispatched according to a sorted list of servers, based on factors such as nearness, server processor speed, reliability, and CPU Usage. For large numbers of servers, a store-and-forward approach becomes much more efficient. A first selected server receives the message from a content control manager (CCM). The first server requests instructions for the next server listed on an ordered list in the CCM and forwards a copy of that message to the next identified server. Each server reports its completion and requests further instructions from the CCM. This mechanism permits highly efficient and robust distribution of assignments and data content from the CCM to each required GL using a store-and-forward tree structure.

Method and apparatus for scalable distribution of information in a distributed network

A method for distributing information to a plurality of servers is described. The information is transmitted from a control manager server to a first group leader server. The first group leader server transmits the information to a second group leader server. The second group leader server requests instructions from the control manager server. The second group leader server then transmits, in response to the instructions, the information to a further group leader server. The first group leader server transmits the information to back end servers belonging to a group of the first group leader server. The second group leader server transmits the information to back end servers belonging to a group of the second group leader server. The further group leader server transmits the information to back end servers belonging to a group of the further group leader server.

Vitaly S. Revsin

Papers

Method and apparatus for reliable and scalable distribution of data files in distributed networks

Method and apparatus for election of group leaders in a distributed network

Method and apparatus for scalable distribution of information in a distributed network

Method and apparatus for dynamic resource discovery and information distribution in a data network

System for creating and distributing prioritized list of computer nodes selected as participants in a distribution job