Systems, methods and/or devices are used for efficient implementation of optimized host-based garbage collection strategies using xcopy and arrays of flash devices. In one aspect, a method of managing a storage system having one or more storage devices includes a host-based garbage collection operation that includes identifying two or more logical stripes in accordance with data storage information stored at the host system, and enabling a process of coalescing valid data in the two or more logical stripes. Further, the use of an internal copy operation (e.g., xcopy), allows the host-based garbage collection operation to occur without transferring data back to the host, thus minimizing the number of I/O operations between the host and storage devices. Additionally, use of the host-based garbage collection operation allows more sophisticated garbage collection algorithms (e.g., matching the current workload) to be used, and ensures that multiple logical stripes are available to write data.

Efficient implementation of optimized host-based garbage collection strategies using xcopy and multiple logical stripes

One embodiment facilitates a write operation in a shingled magnetic recording device. During operation, the system receives, by the storage device, data to be written to the storage device and access-frequency information associated with the data, wherein the storage device includes a plurality of concentric tracks. The system distributes a plurality of spare sector pools among the plurality of concentric tracks. The system selects a track onto which to write the data based on the access-frequency information, wherein data with a highest access-frequency is written to an outer track. The system appends the data at a current write pointer location of the selected track, thereby facilitating an enhanced data placement for subsequent access in the storage device.

Method and system for data placement in a hard disk drive based on access frequency for improved IOPS and utilization efficiency

Methods, systems, and computer programs are presented for deduplicating data in a storage device. One method includes an operation for identifying multiple deduplication domains for a storage system. A fingerprint index is created for each deduplication domain, where each data block stored in the storage system is associated with one of the plurality of deduplication domains. The method also includes operations for receiving a first data block the storage system, and for identifying a first deduplication domain from the plurality at of deduplication domains corresponding to the first data block. The first data block is deduplicated within the first deduplication domain utilizing a first fingerprint index associated with the first deduplication domain.

Multiple deduplication domains in network storage system

Some embodiments of the present invention include a method for identifying duplicate records from a group of records in a database system. The method includes generating a cluster of records from a group of records based on one or more keys; splitting the cluster of records into multiple subsets of records with each subset of records having fewer number of records than the cluster of records, wherein the splitting the cluster of records into multiple subsets of records is based on a number of records in the cluster of records exceeding a threshold; causing duplicate sets of records in each of the subsets of records to be identified, wherein a duplicate set of records includes one or more records, and wherein when a duplicate set of records includes two or more records, the two or more records are duplicates of one another; merging all of the duplicate sets of records identified from the multiple subsets of records forming a first group of duplicate sets of records; and forming a representative set of records based on selecting a representative record from each of the duplicate sets in the first group of duplicate sets of records.

Optimized subset processing for de-duplication

One embodiment facilitates a reduced write amplification. During operation, the system receives, by a computing device, a request to write data to a non-volatile memory. The system writes a first page of the data to a block of the non-volatile memory based on a first physical block address of a destination page of the block, wherein the destination page is a first available page of the block. The system maps, in a data structure by a flash translation layer module of the computing device, a first logical block address of the first page of the data to the first physical block address.

Method and system for enhancing flash translation layer mapping flexibility for performance and lifespan improvements

Region-based storage for volume data and metadata

Various embodiments for identifying data in a data deduplication system, by a processor device, are provided. In one embodiment, a method comprises efficiently identifying duplicate data in the data deduplication system by identifying fingerprint matches using a direct inter-region fingerprint lookup to search for the fingerprint matches in at least one of a plurality of metadata regions, the direct inter-region fingerprint lookup supplementing a central fingerprint index.

Direct lookup for identifying duplicate data in a data deduplication system

B-Tree data is serialized to existing data for all types of workloads. The serialized B-Tree data, that has been split, sorted and classified into identified data ranges, is then compressed.

Compression of serialized B-tree data

B-Tree data is serialized to existing data for all types of workloads by converting a B-Tree data structure into a format capable of being stored and resurrected while containing all data stored in the B-Tree data structure and information relating to the B-Tree data structure.

Efficient b-tree data serialization

For efficient serialization of journal data using a processor device, serialization is combined with journaling of data into one data storage system on a write operation with subsequent consolidation for serialization of the journal data.

Yifat Kuttner

Papers

Region-based storage for volume data and metadata

Direct lookup for identifying duplicate data in a data deduplication system

Compression of serialized B-tree data

Efficient b-tree data serialization

Efficient serialization of journal data