Systems and methods for data classification to facilitate and improve data management within an enterprise are described. The disclosed systems and methods evaluate and define data management operations based on data characteristics rather than data location, among other things. Also provided are methods for generating a data structure of metadata that describes system data and storage operations. This data structure may be consulted to determine changes in system data rather than scanning the data files themselves.

Systems and methods for classifying and transferring information in a storage network

Described in detail herein is a method of copying data of one or more virtual machines being hosted by one or more non-virtual machines. The method includes receiving an indication that specifies how to perform a copy of data of one or more virtual machines hosted by one or more virtual machine hosts. The method may include determining whether the one or more virtual machines are managed by a virtual machine manager that manages or facilitates management of the virtual machines. If so, the virtual machine manager is dynamically queried to automatically determine the virtual machines that it manages or that it facilitates management of. If not, a virtual machine host is dynamically queried to automatically determine the virtual machines that it hosts. The data of each virtual machine is then copied according to the specifications of the received indication.

Systems and methods for management of virtualization data

Data storage operations, including content-indexing, containerized deduplication, and policy-driven storage, are performed within a cloud environment. The systems support a variety of clients and cloud storage sites that may connect to the system in a cloud environment that requires data transfer over wide area networks, such as the Internet, which may have appreciable latency and/or packet loss, using various network protocols, including HTTP and FTP. Methods are disclosed for content indexing data stored within a cloud environment to facilitate later searching, including collaborative searching. Methods are also disclosed for performing containerized deduplication to reduce the strain on a system namespace, effectuate cost savings, etc. Methods are disclosed for identifying suitable storage locations, including suitable cloud storage sites, for data files subject to a storage policy. Further, systems and methods for providing a cloud gateway and a scalable data object store within a cloud environment are disclosed, along with other features.

Data object store and server for a cloud storage environment, including data deduplication and data management across multiple cloud storage sites

There is disclosed in a communications system having an origination storage device and a destination storage device, a data transfer pipeline apparatus for transferring data in a sequence of N stages, where N is a positive integer grater than 1, from the origination to the destination device. The data transfer apparatus comprises dedicated memory means having a predetermined number of buffers dedicated for carrying data associated with the transfer of data from the origination storage device to the destination device; and master control means for registering and controlling processes associated with the data transfer apparatus for participation in the N stage data transfer sequence. The processes include at least a first stage process for initiating the data transfer and a last Nth stage process for completing data transfer. The first stage process is operative to allocate a buffer from the predetermined number of buffers available within the dedicated memory means for collection, processing, and sending of the data from the origination device to a next stage process. The Nth stage process is operative to receive a buffer allocated to the first stage process from the (N−1)th stage process in the data transfer sequence and to free the buffer upon processing completion and storage in the destination device to permit reallocation of the buffer. The master control means further includes monitor means for monitoring the number of buffers from the pool of buffers allocated or assigned to particular processes in the pipeline, in order to prevent allocation of further buffers to a particular process when the number of buffers currently allocated exceeds a predetermined threshold.

Pipelined high speed data transfer mechanism

Performing data management operations on replicated data in a computer network. Log entries are generated for data management operations of an application executing on a source system. Consistency point entries are used to indicate a time of a known good, or recoverable, state of the application. A destination system is configured to process a copy of the log and consistency point entries to replicate data in a replication volume, the replicated data being a copy of the application data on the source system. When the replicated data represents a known good state of the application, as determined by the consistency point entries, the destination system(s) may perform a storage operation (e.g., snapshot, backup) to copy the replicated data and to logically associate the copied data with a time information (e.g., time stamp) indicative of the source system time when the application was in the known good state.

Systems and methods for performing data replication

A file backup facility is provided for a community of personal computers (PC1-3) linked to a central station (10) that includes a mass storage device (14). The backup facility enables a user to select files for backup to the mass storage device (14) and then subsequently to restore selected ones of the backed-up files. Concurrent backup from several personal computers is permitted with the files subject to backup being divided into blocks and the blocks interleaved for storage on the mass storage device (14). The selection of files for backup or restore is effected by presenting to the user a directory listing all relevant subdirectories and files and permitting the user to select/deselect files individual or by directory.

File backup facility for a community of personal computers

A method and apparatus for storing user data organized into a plurality of data records on tape is provided. User data is first accepted from a host computer. The user data is then organized into data groups independent of the record structure. Records are then assigned to data entities where each data entity has at least one data record. A group index is formed to which information pertaining to the records and/or entities contained within the group is written. Data is then stored on the tape in groups along with each associated group index.

Storing plural data records on tape in an entity with an index entry common to those records

A method of appending data to compressed data stored on tape (10) in the form of records (CR n ) wherein compressed data is stored in groups (G n ) independently of the record structure of the data and each group has an associated data structure (4), containing information relating to the group contents in terms of entities (E n ), where an entity can contain more than one record, and means (H) for storing information on the number of records in each entity (E n ) characterised by locating the entity containing the last record to be retained and changing said stored information to indicate the number of wanted records in that entity and writing the data being appended to a subsequent new entity.

Method and apparatus for appending data to compressed records previously stored on a sequentially-accessible storage medium

A tape storage device is used in conjunction with a host computer to archive data. The storage device has interface logic that buffers blocks of data from the host computer in a memory buffer. The interface logic processes data as the data is received to create a processed data block that is smaller in size than the original, unprocessed data block. While receiving elements of the data block, the interface logic repeatedly determines the amount of free memory in the memory buffer, and temporarily suspends the block transfer whenever the amount of free memory falls below a predetermined threshold. The block transfer is resumed when the amount of free memory becomes equal to or greater than the threshold. This utilizes nearly all available memory, while preventing buffer overflow, without any need to perform complex calculations to predict the size of the data block after processing.

Tape drive data buffering

Burst-mode data transfers between a SCSI host bus adapter and at least one SCSI bus device interface adapter are implemented by hardware. For a first embodiment of the invention, the device interface adapter is equipped with a first, second and third data registers, a comparator, a subtractor, and control logic in the form of an application specific integrated circuit. When a burst-mode transfer is requested, the first register is programmed with a value corresponding to the length of the transfer in bytes, and the second register is programmed with the maximum possible number of bytes in a burst. The comparator then compares the value in stored in the first register with the value stored in the second register and determines which is the smaller. The smaller of the two values is written to the third register. The subtractor then subtracts said third value from said first value to obtain a remainder value. The first value is then replaced with a new first value equal to said remainder value. The control logic orchestrates the steps of comparing said first and second values, storing the smaller of said first and second values in said third register, subtracting said third value from said first value to give a remainder value, replacing said first value with a new first value equal to said remainder value, and causing the steps of comparing, storing, subtracting and replacing to be repeated until said third value is equal to zero. Initial loading of the first and second registers is performed by either the control logic or by a microprocessor. For an alternate embodiment of the invention, the control logic, the comparator and the subtractor are replaced by a microprocessor.

Mark J. Simms

Papers

File backup facility for a community of personal computers

Storing plural data records on tape in an entity with an index entry common to those records

Method and apparatus for appending data to compressed records previously stored on a sequentially-accessible storage medium

Tape drive data buffering

System for implementing hardware automated control of burst mode data transfer over a communication link between devices operating in a block mode