A data storage system which automatically provides and maintains identical secondary data on a preferably geographically remote secondary data storage device. The system includes a primary host computer located in the first geographic location which is coupled to a primary data storage system also located in the first geographic location. The primary data storage system includes at least one primary data storage device on which primary data is to be stored, and a primary data storage system controller which receives data from the primary host computer and controls the storing of the primary data on the primary data storage device. The primary data storage controller is coupled by high speed communication link to a secondary data storage system controller of a secondary data storage system, and coordinates the copying of the primary data to the secondary data storing system, and is responsive to an acknowledgement from the secondary data storage system controller of at least successful receipt of the primary data for updating the maintained list of the primary data to indicate that the copied primary data has been received by the secondary data storage system to be copied to the secondary data storage device.

Remote data mirroring

A high availability, high reliability storage system (215) that leverages rapid advances in commodity computing devices and the robust nature of internetwork technology such as the Internet (213). A system of parity distribution (505) in accordance with the present invention allows for greater fault tolerance and levels of storage efficiency than possible with conventional RAID (levels 0-5) paradigms. Data can be recovered or made available even in the case of loss of N, N+1, or more devices or storage elements (215) over which stripes of the data set have been distributed or partitioned. The present invention provides a parity distribution that can be used to distribute data stored in a single storage device or across multiple connected or otherwise networked devices.

System and method for data protection with multidimensional parity

The invention provides systems and methods for continuous back up of data stored on a computer network. To this end the systems of the invention include a synchronization process that replicates selected source data files data stored on the network and to create a corresponding set of replicated data files, called the target data files, that are stored on a back up server. This synchronization process builds a baseline data structure of target data files. In parallel to this synchronization process, the system includes a dynamic replication process that includes a plurality of agents, each of which monitors a portion of the source data files to detect and capture, at the byte-level, changes to the source data files. Each agent may record the changes to a respective journal file, and as the dynamic replication process detects that the journal files contain data, the journal files are transferred or copied to the back up server so that the captured changes can be written to the appropriate ones of the target data files.

Systems and methods for backing up data files

Systems and methods are disclosed for performing data storage operations, including content-indexing, containerized deduplication, and policy-driven storage, within a cloud environment The systems support a variety of clients and cloud storage sites that may connect to the system in a cloud environment that requires data transfer over wide area networks, such as the Internet, which may have appreciable latency and/or packet loss, using various network protocols, including HTTP and FTP Methods are disclosed for content indexing data stored within a cloud environment to facilitate later searching, including collaborative searching Methods are also disclosed for performing containerized deduplication to reduce the strain on a system namespace, effectuate cost savings, etc Methods are disclosed for identifying suitable storage locations, including suitable cloud storage sites, for data files subject to a storage policy Further, systems and methods for providing a cloud gateway and a scalable data object store within a cloud environment are disclosed, along with other features

Performing data storage operations with a cloud storage environment, including automatically selecting among multiple cloud storage sites

In a client/server environment, a method and means for reducing the storage requirement in the backup subsystem and further reducing the load on the transmission bandwidth where base files are maintained on the server in a segmented compressed format. When a file is modified on the client, the file is transmitted to the server and compared with the segmented compressed base version of the file utilizing a differencing function but without decompressing the entire base file. A delta file which is the difference between the compressed base file and the modified version of the file is created and stored on a storage medium which is part of the backup subsystem. Alternatively, a copy of frequently accessed base files are maintained on the client in a compressed format. Whenever the client detects that a frequently accessed file has been modified, the modified version of the file is differenced against the base version of that file without decompressing the entire base file and a delta file is generated. The delta file is then transmitted to the server to be stored at the server for storage medium to be utilized either immediately or at a later time to update the base version of the modified file on the server.

System and method for reducing storage requirement in backup subsystems utilizing segmented compression and differencing

A method in a data processing system for automatically terminating or resuming backup copy sessions after an abnormal interrupt or reset notification occurrence during a backup copy process. A status indication is entered into activity tables associated with a plurality of storage subsystems and devices within a data processing system in response to initiation of a backup copy session. Status indications are then entered upon successful completion of a backup copy session within the data processing system. The indications within the activity tables are reviewed to determine the status of a backup copy session upon restarting a resource manager, abnormal termination of a backup copy program, or an operating system initial program load. If a backup copy session has been initiated but not completed, the backup copy session is then terminated. The indications within the activity tables are also reviewed to determine the status of a backup copy session if a reset notification is raised by a storage subsystem. The tracks extents which are active for a volume associated with a particular session identification are determined. A comparison is then made between the tracks extents which are active and the volume and extent information associated with a physical session identification. If a match exists between the tracks extents which are active and the volume and extent information associated with a physical session identification, the backup copy session resumes. If a match does not exist, the backup copy session is terminated.

Method and system for automated termination and resumption in a time zero backup copy process

Backup copying of designated datasets representing a first selected point in time consistency may be performed in a data processing system on an attached storage subsystem concurrent with data processing system application execution by first suspending application execution only long enough to form a logical-to-physical address concordance, and thereafter physically backing up the datasets on the storage subsystem on a scheduled or opportunistic basis. An indication of each update to a selected portion of the designated datasets which occurs after the first selected point in time is stored and application initiated updates to uncopied designated datasets are first buffered. Thereafter, sidefiles are made of the affected datasets, or portions thereof, the updates are then written through to the storage subsystem, and the sidefiles written to an alternate storage location in backup copy order, as controlled by the address concordance. At a subsequent point in time only those portions of the designated datasets which have been updated after the first selected period and time are copied, utilizing an identical technique.

Method and system for incremental time zero backup copying of data

A method and apparatus to synchronize recovery logs transmitted to a remote site for recovering related databases having different logical structuring. In one embodiment, the related databases are a hierarchial structured database such as IMS and a relational structured database such as DB2.

Synchronizing recovery log having time stamp to a remote site for disaster recovery of a primary database having related hierarchial and relational databases

A remote data shadowing system provides storage based, real time disaster recovery capability. Record updates at a primary site cause write I/O operations in a storage subsystem therein. The write I/O operations are time stamped and the time, sequence, and physical locations of the record updates are collected in a primary data mover. The primary data mover groups sets of the record updates and associated control information based upon a predetermined time interval, the primary data mover appending a prefix header to the record(updates thereby forming self describing record sets. The self describing record sets are transmitted to a remote secondary site wherein consistency groups are formed such that the record updates are ordered so that the record updates can be shadowed in an order consistent with the order the record updates cause write I/O operations at the primary site.

Forming consistency groups using self-describing record sets for remote data duplexing

A disaster recovery system providing remote data shadowing between a primary and a secondary site uses a method and apparatus for swapping, or switching, host directed I/O operations from a primary data storage device to a secondary data storage device in a remote copy duplex pair. Application programs running in a host processor at the primary site first quiesce all I/O operations and record updates targeted to the primary data storage device. The remote copy duplex pair is checked to verify that the secondary data storage device is a fully synchronized copy of the primary data storage device. The secondary data storage device is then swapped with the primary data storage device by terminating the remote copy duplex pair, establishing an opposite direction remote copy duplex pair such that the secondary data storage device is a primary device of the duplex pair and the primary data storage device is a shadowing device, and updating the application programs running in the primary host processor with a device address of the secondary data storage device substituted as a device address of the primary data storage device. Finally, the primary host processor resumes its application programs such that subsequent I/O operations and record updates targeted for the primary data storage device are directed through a secondary storage controller to the secondary data storage device, and copied to the primary data storage device.

Robert Frederic Kern

Papers

Method and system for automated termination and resumption in a time zero backup copy process

Method and system for incremental time zero backup copying of data

Synchronizing recovery log having time stamp to a remote site for disaster recovery of a primary database having related hierarchial and relational databases

Forming consistency groups using self-describing record sets for remote data duplexing

Concurrent switch to shadowed device for storage controller and device errors