A computer system employs a globally addressable storage environment that allows a plurality of networked computers to access data by addressing even when the data is stored on a persistent storage device such as a computer hard disk and other traditionally non-addressable data storage devices. The computers can be located on a single computer network or on a plurality of interconnected computer networks such as two local area networks (LANs) coupled by a wide area network (WAN). The globally addressable storage environment allows data to be accessed and shared by and among the various computers on the plurality of networks.

Remote access and geographically distributed computers in a globally addressable storage environment

A data processing system includes one or more nodes, each node including a memory sub-system. The sub-system includes a fine-grained, memory, and a less-fine-grained (e.g., page-based) memory. The fine-grained memory optionally serves as a cache and/or as a write buffer for the page-based memory. Software executing on the system uses a node address space which enables access to the page-based memories of all nodes. Each node optionally provides ACID memory properties for at least a portion of the space. In at least a portion of the space, memory elements are mapped to locations in the page-based memory. In various embodiments, some of the elements are compressed, the compressed elements are packed into pages, the pages are written into available locations in the page-based memory, and a map maintains an association between the some of the elements and the locations.

System including a fine-grained memory and a less-fine-grained memory

A multi-user file storage service and system enable each user of a pre-subscribed user group to operate an arbitrary client node at an arbitrary geographic location, to communicate with a remote file server node via a wide area network and to access the files of the file group via the respective client node in communication with the remote file server node via the wide area network. More than one user of the pre-subscribed user group is permitted to access the file group at the remote file server node simultaneously. Illustratively, the integrity of the files at the remote file server node are maintained by controlling each access to each file at the remote file server node so that each access to files at the remote file server is performed, if at all, on a respective portion of each file as most recently updated at the remote file server node. Thus, all native operating system application programming interfaces operate as if all multi-user applications accessing the files function as if the remote server and client nodes were on the same local area network. Illustratively, an interface is provided for adapting file access one of the client nodes. The interface designates at the client node each accessible file of the group as stored on a virtual storage device. The interface enables access to the designated files in a fashion which is indistinguishable, by users of, and applications executing at, the client node, with access to one or more files stored on a physical storage device that is locally present at the client node. Illustratively, an encrypted key is transferred from the remote file server node to one of the client nodes via a secure channel. The key is encrypted using an encryption function not known locally at the remote file server node. The transferred key is decrypted at the client node. The key is used at the client node to decrypt information of the files downloaded from the remote file server node or to encrypt information of the files prior to uploading for storage at the remote file server node. Access control to a particular one of the files of the group can be delegated to an access control node.

Internet-based shared file service with native pc client access and semantics and distributed version control

A network of computer node interface to globally addressable memory system that provides persistent storage of data exchange periodic connectivity information. The exchanged connectivity information provides information regarding node failure to other nodes in the system, and the surviving nodes use the information to determine which node, if any, has ceased functioning. Various processes are used to recover the portion of the global address space for which the failed node was responsible, including RAM directory, disk directory, or file system information. Additionally, nodes may be subdivided into groups and connectivity information is exchanged between nodes belonging to a group. Each group then exchanges group-wise connectivity information and failures may be recovered.

System and method for providing highly available data storage using globally addressable memory

In a network of computer nodes, a structured storage system interfaces to a globally addressable memory system that provides persistent storage of data. The globally addressable memory system may be a distributed shared memory (DSM) system. A control program resident on each network node can direct the memory system to map file and directory data into the shared memory space. The memory system can include functionality to share data, coherently replicate data, and create log-based transaction data to allow for recovery. In one embodiment, the memory system provides memory device services to the data control program. These services can include read, write, allocate, flush, or any other similar or additional service suitable for providing low level control of a memory storage device. The data control program employs these memory system services to allocate and access portions of the shared memory space for creating and manipulating a structured store of data such as a file system, a database system, or a Web page system for storing, retrieving, and delivering objects such as files, database records or information, and Web pages.

Structured data storage using globally addressable memory

A computer cluster architecture including a plurality of CPUs at each of a plurality of nodes. Each CPU has the property of coherency and includes a primary cache. A local bus at each node couples: all the local caches, a local main memory having physical space assignable as-shared space and non-shared space and a local external coherency unit (ECU). An inter-node communication bus couples all the ECUs. Each ECU includes a monitoring section for monitoring the local and inter-node busses and a coherency section for a) responding to a non-shared cache-line request appearing on the local bus by directing the request to the non-shared space of the local memory and b) responding to a shared cache-line request appearing on the local bus by examining its coherence state to further determine if inter-node action is required to service the request and, if such action is required, transmitting a unique identifier and a coherency command to all the other ECUs. Each unit of information present in the shared space of the local memory is assigned, by the local ECU, a coherency state which may be: exclusive (the local copy of the requested information is unique in the cluster); 2) modified (the local copy has been updated by a CPU in the same node); 3) invalid (a local copy either does not exist or is known to be out-of-date); or 4) shared (the local copy is one of a plurality of current copies present in a plurality of nodes).

Multi-node cluster computer system incorporating an external coherency unit at each node to insure integrity of information stored in a shared, distributed memory

Cluster architectures are improved by introducing a Generalized Shared Memory , which is maintained in a consistent state by a hardware-based coherency mechanism that operates on shared objects, wherever they happen to be located. Each cluster node comprises one or more processors, a local main memory storing both private and shared objects and an external coherency unit connected to inter-node communication means. The external coherency unit comprises monitoring means adapted to monitor (snoop) both local and inter-node communications and coherency means adapted to act upon cache-line requests to shared objects in response to the coherence state of the cache-line. This increases both the performance and the versatility of the architectures by permitting the composition of private vs. shared memory to be of arbitrary size and dynamically variable on different computer nodes in the cluster.

John A. Wertz

Papers

Multi-node cluster computer system incorporating an external coherency unit at each node to insure integrity of information stored in a shared, distributed memory

Generalized shared memory in a cluster architecture for a computer system.

Verallgemeinerter gemeinsamer Speicher in einer Verbandarchitektur eines Rechnersystems