Database management systems will continue to manage large data volumes. Thus, efficient algorithms for accessing and manipulating large sets and sequences will be required to provide acceptable performance. The advent of object-oriented and extensible database systems will not solve this problem. On the contrary, modern data models exacerbate the problem: In order to manipulate large sets of complex objects as efficiently as today's database systems manipulate simple records, query-processing algorithms and software will become more complex, and a solid understanding of algorithm and architectural issues is essential for the designer of database management software. This survey provides a foundation for the design and implementation of query execution facilities in new database management systems. It describes a wide array of practical query evaluation techniques for both relational and postrelational database systems, including iterative execution of complex query evaluation plans, the duality of sort- and hash-based set-matching algorithms, types of parallel query execution and their implementation, and special operators for emerging database application domains.

/pdf/query-evaluation-techniques-for-large-databases-45td1anbls.pdf

Query evaluation techniques for large databases

DB2TM, IMS, and TandemTM systems. ARIES is applicable not only to database management systems but also to persistent object-oriented languages, recoverable file systems and transaction-based operating systems. ARIES has been implemented, to varying degrees, in IBM's OS/2TM Extended Edition Database Manager, DB2, Workstation Data Save Facility/VM, Starburst and QuickSilver, and in the University of Wisconsin's EXODUS and Gamma database machine.

/pdf/aries-a-transaction-recovery-method-supporting-fine-27kfd8jref.pdf

ARIES: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging

This paper presents the design of a read-optimized relational DBMS that contrasts sharply with most current systems, which are write-optimized. Among the many differences in its design are: storage of data by column rather than by row, careful coding and packing of objects into storage including main memory during query processing, storing an overlapping collection of column-oriented projections, rather than the current fare of tables and indexes, a non-traditional implementation of transactions which includes high availability and snapshot isolation for read-only transactions, and the extensive use of bitmap indexes to complement B-tree structures.We present preliminary performance data on a subset of TPC-H and show that the system we are building, C-Store, is substantially faster than popular commercial products. Hence, the architecture looks very encouraging.

/pdf/c-store-a-column-oriented-dbms-3oddwwyeo0.pdf

C-store: a column-oriented DBMS

/pdf/c-store-a-column-oriented-dbms-4q4ob7krcm.pdf

Modern computer systems have been built around the assumption that persistent storage is accessed via a slow, block-based interface. However, new byte-addressable, persistent memory technologies such as phase change memory (PCM) offer fast, fine-grained access to persistent storage.In this paper, we present a file system and a hardware architecture that are designed around the properties of persistent, byteaddressable memory. Our file system, BPFS, uses a new technique called short-circuit shadow paging to provide atomic, fine-grained updates to persistent storage. As a result, BPFS provides strong reliability guarantees and offers better performance than traditional file systems, even when both are run on top of byte-addressable, persistent memory. Our hardware architecture enforces atomicity and ordering guarantees required by BPFS while still providing the performance benefits of the L1 and L2 caches.Since these memory technologies are not yet widely available, we evaluate BPFS on DRAM against NTFS on both a RAM disk and a traditional disk. Then, we use microarchitectural simulations to estimate the performance of BPFS on PCM. Despite providing strong safety and consistency guarantees, BPFS on DRAM is typically twice as fast as NTFS on a RAM disk and 4-10 times faster than NTFS on disk. We also show that BPFS on PCM should be significantly faster than a traditional disk-based file system.

/pdf/better-i-o-through-byte-addressable-persistent-memory-2wrl2k9p36.pdf

Better I/O through byte-addressable, persistent memory

Many data base management systems' query optimizers choose at most one index for accessing the records of a table in a given query, even though many indexes may exist on the table. In spite of the fact that there are some systems which use multiple indexes, very little has been published about the concurrency control or query optimization implications (e.g., deciding how many indexes to use) of using multiple indexes. This paper addresses these issues and presents solutions to the associated problems. Techniques are presented for the efficient handling of record ID lists, elimination of some locking, and determination of how many and which indexes to use. The techniques are adaptive in the sense that the execution strategies may be modified at run-time (e.g., not use some indexes which were to have been used), if the assumptions made at optimization-time (e.g., about selectivities) turn out to be wrong. Opportunities for exploiting parallelism are also identified. A subset of our ideas have been implemented in IBM's DB2 V2R2 relational data base management system.

Single table access using multiple indexes: optimization, execution, and concurrency control techniques

We present several methods which relate to space management in a transaction system supporting fine-granularity (eg, record) locking These methods enable varying length records to be supported efficiently by permitting garbage collection to be performed within a page without the moved records having to be locked or the movements having to be logged We present methods to do the following: (1) When a transaction releases space, efficiently prevent that space from being consumed by other transactions until that transaction terminates, while allowing the same transaction to reuse the space it freed (2) Under the correct circumstances, avoid reading a totally empty deallocated page from disk during page reallocation (3) Updating and logging of free space inventory pages' (FSIPs') changes for correct recovery (4) Reduce locking during a table scan by a transaction using the isolation level of cursor stability Our methods improve concurrency and space utilization, and provide I/O and CPU savings Our space reservation and FSIP logging methods have been implemented in DB2 V3 in preparation for DB2's support of record locking

Don Haderle

Papers

ARIES: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging

Single table access using multiple indexes: optimization, execution, and concurrency control techniques

Algorithms for flexible space management in transaction systems supporting fine-granularity locking