A disk drive system and method capable of dynamically allocating data is provided. The disk drive system may include a RAID subsystem having a pool of storage, for example a page pool of storage that maintains a free list of RAIDs, or a matrix of disk storage blocks that maintain a null list of RAIDs, and a disk manager having at least one disk storage system controller. The RAID subsystem and disk manager dynamically allocate data across the pool of storage and a plurality of disk drives based on RAID-to-disk mapping. The RAID subsystem and disk manager determine whether additional disk drives are required, and a notification is sent if the additional disk drives are required. Dynamic data allocation and data progression allow a user to acquire a disk drive later in time when it is needed. Dynamic data allocation also allows efficient data storage of snapshots/point-in-time copies of virtual volume pool of storage, instant data replay and data instant fusion for data backup, recovery etc., remote data storage, and data progression, etc.

Virtual disk drive system and method

Systems and methods are disclosed for performing data storage operations, including content-indexing, containerized deduplication, and policy-driven storage, within a cloud environment The systems support a variety of clients and cloud storage sites that may connect to the system in a cloud environment that requires data transfer over wide area networks, such as the Internet, which may have appreciable latency and/or packet loss, using various network protocols, including HTTP and FTP Methods are disclosed for content indexing data stored within a cloud environment to facilitate later searching, including collaborative searching Methods are also disclosed for performing containerized deduplication to reduce the strain on a system namespace, effectuate cost savings, etc Methods are disclosed for identifying suitable storage locations, including suitable cloud storage sites, for data files subject to a storage policy Further, systems and methods for providing a cloud gateway and a scalable data object store within a cloud environment are disclosed, along with other features

Performing data storage operations with a cloud storage environment, including automatically selecting among multiple cloud storage sites

In a network environment (200), multiple clients (210) and multiple servers (230) are connected via a local area network (LAN) (220) to a tape backup apparatus (240). Each client (210) and each server is provided with backup agent software (215), which schedules backup operations on the basis of time since the last backup, the amount of information generated since the last backup, or the like. An agent (215a) also sends a request to the tape backup apparatus (240), prior to an actual backup, including information representative of the files that it intends to back up. The tape backup apparatus (240) is provided with a mechanism to receive backup requests from the client agents (215) and accept or reject backup requests on the basis of backup server loading, network loading, or the like. The tape backup apparatus (240) is further provided with mechanisms to enact redundant file elimination (RFE), whereby the server indicates to the client agents, prior to files being backed up, that certain of the files to be backed up are already stored by the backup server. Thus, the clients do not need to send the redundant files to be backed up. The tape backup apparatus (240) comprises a two-tier data storage system in which all backed up data is stored in on-line media, such as a hard disk (244). A copy of the data on the on-line media (244) is periodically backed up to off-line media, such as tape. The backup server is provided with mechanisms whereby a client can restore any of its 'lost' data by copying it directly from the on-line storage, without the need for backup administrator assistance.

Data backup and recovery systems

The invention provides structure and method for controlling access to a shared storage device, such as a disk drive storage array, in computer systems and networks having a plurality of host computers. A method for controlling access to a hardware device in a computer system having a plurality of computers and at least one hardware device connected to the plurality of computers. The method includes the steps of associating a locally unique identifier with each the plurality of computers, defining a data structure in a memory identifying which particular ones of the computers based on the locally unique identifier may be granted access to the device; and querying the data structure to determine if a requesting one of the computers should be granted access to the hardware device. In one embodiment, the procedure for defining the data structure in memory includes defining a host computer ID map data structure in the memory; defining a port mapping table data structure comprising a plurality of port mapping table entries in the memory; defining a host identifier list data structure in the memory; defining a volume permission table data structure in the memory; and defining a volume number table data structure in the memory. In one particular embodiment, the memory is a memory of a memory controller controlling the hardware device, and the hardware device is a logical volume of a storage subsystem. The invention also provides an inventive controller structure, and a computer program product implementing the inventive method.

Method and system for controlling access share storage devices in a network environment by configuring host-to-volume mapping data structures in the controller memory for granting and denying access to the devices

Systems and methods for data classification to facilitate and improve data management within an enterprise are described. The disclosed systems and methods evaluate and define data management operations based on data characteristics rather than data location, among other things. Also provided are methods for generating a data structure of metadata that describes system data and storage operations. This data structure may be consulted to determine changes in system data rather than scanning the data files themselves.

Systems and methods for classifying and transferring information in a storage network

A storage controller has at least one physical data port for a data network including host processors. The storage controller is programmed to provide a plurality of virtual ports for access to storage, and a virtual switch for routing storage access requests from the physical port to the virtual ports. The virtual ports and the virtual switch are defined by software. The virtual ports appear to the hosts as physical ports in the data network. For example, in a Fiber-Channel network, the virtual ports have World Wide Names (WWNs) and are assigned temporary addresses (S_Ds), and the virtual switch provides a name server identifying the WWNs and S_IDs of the virtual ports. For convenient partitioning of storage among host processors, one or more virtual ports are assigned to each host, and a set of storage volumes are made accessible from each virtual port. A host can access storage at a virtual port only if the virtual port has been assigned to the host. Preferably, storage can be accessed through each virtual port by no more than one assigned host, although a shared volume may be accessible from more than one virtual port. The storage controller may provide a service for reporting to a host the virtual ports through which the host can access storage, and the storage volumes that are accessible to the host through each of the virtual ports.

Virtual ports for data transferring of a data storage system

A method of controlling distribution of processing system including a plurality of host data processors (12) connected to a data storage system (14) which includes a digital storage that is partitioned into a plurality of volumes (20). The method includes assigning ownership of the volumes (80) to the host processors (12) such that each of the host processors owns a different subset of the plurality of volumes, wherein a host processor is prohibited from sending I/O requests to any of the volumes which it does not own. The method further includes monitoring the I/O requests that are sent each of the volumes by each of the host processors; from information obtained through monitoring, generating workload statistics indicative of the distribution of workload among the host processors; detecting a workload imbalance in the workload statistics; and in response to detecting a workload imbalance, reassigning ownership of the volumes to the host processors (12) so as to change the distribution of workload among the host processors.

Dynamic load balancing

A storage controller for controling access to data storage has a memory and at least one data port for a data network including host processors. The memory is programmed to define a respective specification for each host processor of a respective subset of the data storage to which access by the host processor is restricted, and each specification is associated with a host identifier stored in the memory. When the storage controller receives a data access request from a host processor, it decodes a host identifier from the data access request, and searches the memory for a host identifier matching the host identifier decoded from the request. Upon finding a match, the respective specification of the respective subset for the host processor is accessed to determine whether or not storage specified by the storage access request is contained in the respective subset. If so, then storage access can continue, and otherwise, storage access is denied. Preferably the host identifier decoded from the request is a temporary address assigned by the network, and also stored in the memory in association with each respective specification is a relatively permanent identifier for the host processor.

Storage mapping and partitioning among multiple host processors in the presence of login state changes and host controller replacement

In a multi-version database, copies of prior committed versions (snapshots) are kept for access by the read-only transactions. The read-write transactions are selectively aborted to enforce an order of commitment of read-write transactions that is the same as an order of conflicts among the read-write transactions. In a preferred embodiment, the read-write transactions are serialized by maintaining and referencing a graph of conflicts among read-write transactions, and the read-only transactions are serialized by a timestamp mechanism for selection of the snapshots to be read. Each time that a read-write transaction is committed, the read-write transaction is assigned a unique timestamp that is used to timestamp all resources committed by the read-write transaction. Upon starting, each read-only transaction is also assigned a timestamp. Each read-only transaction reads only the latest committed versions of all resources, that are timestamped earlier than the timestamp of the read-only transaction. In a multiprocessing system, the timestamps are issued to global coordinators and distributed locally with atomic commit messages and global queries. Moreover, read-write transactions may selectively access a hierarchy of uncommitted versions to prepare for various possible commitment orders. The hierarchy defines a path for record access and for cascading aborts. A plurality of mutually-conflicting uncommitted versions may be prepared for each transaction to prepare for all possible commitment orders.

Distributed multi-version commitment ordering protocols for guaranteeing serializability during transaction processing

In a data storage subsystem providing data storage to host processors, a process of configuration defines a subset of the data storage that each host may access. A vector specification is a convenient mechanism for specifying a set of storage volumes that a host may access. For example, for each host processor, there is stored in memory of the data storage subsystem a list of contiguous ranges or vectors of the storage volumes that the host may access. To determine whether or not a specified logical volume number is included in the vector, a mudulus of the stride of the vector is computed from the difference between the address of the specified logical volume and the beginning address of the vector, and the modulus is compared to zero. To provide a mapping between logical unit numbers specified by the host and the logical volumes, a contiguous range of logical unit numbers may also be specified for each contiguous range or vector of storage volumes. The logical volume number is computed from a specified logical unit number by computing a difference between the specified logical unit number and the beginning logical unit number, multiplying the difference by the stride of the vector to produce a product, and adding the product to the beginning address of the vector.

Yoav Raz

Papers

Virtual ports for data transferring of a data storage system

Dynamic load balancing

Storage mapping and partitioning among multiple host processors in the presence of login state changes and host controller replacement

Distributed multi-version commitment ordering protocols for guaranteeing serializability during transaction processing

Configuring vectors of logical storage units for data storage partitioning and sharing