This third edition of a classic textbook can be used to teach at the senior undergraduate and graduate levels. The material concentrates on fundamental theories as well as techniques and algorithms. The advent of the Internet and the World Wide Web, and, more recently, the emergence of cloud computing and streaming data applications, has forced a renewal of interest in distributed and parallel data management, while, at the same time, requiring a rethinking of some of the traditional techniques. This book covers the breadth and depth of this re-emerging field. The coverage consists of two parts. The first part discusses the fundamental principles of distributed data management and includes distribution design, data integration, distributed query processing and optimization, distributed transaction management, and replication. The second part focuses on more advanced topics and includes discussion of parallel database systems, distributed object management, peer-to-peer data management, web data management, data stream systems, and cloud computing. New in this Edition: New chapters, covering database replication, database integration, multidatabase query processing, peer-to-peer data management, and web data management. Coverage of emerging topics such as data streams and cloud computing Extensive revisions and updates based on years of class testing and feedback Ancillary teaching materials are available.

Principles of Distributed Database Systems

The development of the Internet in recent years has made it possible and useful to access many different information systems anywhere in the world to obtain information. While there is much research on the integration of heterogeneous information systems, most commercial systems stop short of the actual integration of available data. Data fusion is the process of fusing multiple records representing the same real-world object into a single, consistent, and clean representation.This article places data fusion into the greater context of data integration, precisely defines the goals of data fusion, namely, complete, concise, and consistent data, and highlights the challenges of data fusion, namely, uncertain and conflicting data values. We give an overview and classification of different ways of fusing data and present several techniques based on standard and advanced operators of the relational algebra and SQL. Finally, the article features a comprehensive survey of data integration systems from academia and industry, showing if and how data fusion is performed in each.

Data fusion

The join operation is one of the fundamental relational database query operations. It facilitates the retrieval of information from two different relations based on a Cartesian product of the two relations. The join is one of the most diffidult operations to implement efficiently, as no predefined links between relations are required to exist (as they are with network and hierarchical systems). The join is the only relational algebra operation that allows the combining of related tuples from relations on different attribute schemes. Since it is executed frequently and is expensive, much research effort has been applied to the optimization of join processing. In this paper, the different kinds of joins and the various implementation techniques are surveyed. These different methods are classified based on how they partition tuples from different relations. Some require that all tuples from one be compared to all tuples from another; other algorithms only compare some tuples from each. In addition, some techniques perform an explicit partitioning, whereas others are implicit.

/pdf/join-processing-in-relational-databases-4dnr7kie1q.pdf

Join processing in relational databases

The objective of a multidatabase system is to provide a single uniform interface to accessing multiple independent databases being managed by multiple independent, and possibly heterogeneous, database systems. One crucial element in the design of a multidatabase system is the design of a data definition language for specifying a schema that represents the integration of the schemas of multiple independent databases. The design of such a language in turn requires a comprehensive classification of the conflicts (i.e., discrepancies) among the schemas of the independent databases and development of techniques for resolving (i.e., homogenizing) all of the conflicts in the classification. An earlier paper provided a comprehensive classification of schematic conflicts that may arise when integrating multiple independent relational database (RDB) schemas into a single multidatabase (MDB) schema. In this paper, we provide a comprehensive classification of techniques for resolving the schematic conflicts that may arise when integrating multiple RDB schemas, or RDB schemas and object-oriented database (OODB) schemas, or multiple OODB schemas. The classification of conflict resolution techniques includes not only those necessary for resolving schematic conflicts identified in the earlier paper, but also additional conflicts that arise when OODBs become part of the databases to be integrated. Most of the conflict resolution techniques discussed in the paper have already been incorporated into SQL/M, a multidatabase language implemented in UniSQL/M, a commercially available multidatabase system from UniSQL, Inc. which integrated SQL-based relational database systems and the UniSQL/X unified relational and object-oriented database system.

On resolving schematic heterogeneity in multidatabase systems

This paper studies heterogeneous database systems from the multiple (poly) source @rrt) perspective. It aims at addressing issues such as “where is the data from” and “which intermediate data sources were used to arrive at that data” issues which are critical to many users in utilizing information composed from multiple sources. Specifically, it presents a polygen model for resolving the Data Source Tagging and Intermediate Source Tagging problems. Secondly, it presents a data-driven query translation mechanism for mapping a polygen query into a set of local queries dynamically. A concrete example is also provided to exemplify polygen query processing.

/pdf/a-polygen-model-for-heterogeneous-database-systems-the-3y6h6x9q9a.pdf

A Polygen Model for Heterogeneous Database Systems: The Source Tagging Perspective

This paper describes the query optimizer of the Mermaid system which provides a user with a unified view of multiple preexisting databases which may be stored under different DBMS's. The algorithm is designed for databases which may contain replicated or fragmented relations and for users who are primarily making interactive, ad hoc queries. Although the implementation of the algorithm is a front-end system, not an integrated distributed DBMS, it should be applicable to a distributed DBMS also.

Query Processing in a Fragmented Relational Distributed System: Mermaid

AIDA is a distributed data management (DDM) front-end system which runs on top of existing databases stored in different data management systems (DBMSs). It appears to the user to be a distributed DBMS, although many of the DBMS functions are actually provided by the underlying DBMS. It currently runs above three relational DBMSs, the IDM database machine connected to a VAX, Ingres on two Suns, and Mistress on a third Sun. One of the key components of the AIDA system is Mermaid which provides query optimization, networking, and system control. Mermaid required research into query optimization where the costs of operations on the different processors and DBMSs is variable, the network cost is variable, and capabilities of the different DBMSs is variable. We extended existing distributed query optimization algorithms to support replicated and fragmented relations. In this paper, we discuss our experiences with the development and testing of the Mermaid system and with the integration of Mermaid with the AIDA translator. We discuss the types of commands, the system controller, and our experiences with distributed debugging. Then we give the results of our initial performance tests.

Mermaid — Experiences with network operation

The Mermaid testbed system has been developed as part of an ongoing research program at SDC to explore issues in distributed data management. The Mermaid testbed provides a uniform front end that makes the complexity of manipulating data in distributed heterogeneous databases under various data management systems (DBMSs) transparent to the user. It is being used to test query optimization algorithms as well as user interface methodology.

Distributed query processing strategies in Mermaid, a frontend to data management systems

We propose new adaptive techniques for distributed query optimization. These techniques are divided into two groups: the ones that improve efficiency of query execution (directly) and the ones that improve cost estimations for query execution strategies. Some of the proposed techniques utilize semantic information and knowledge acquisition to adapt to the environment. The latter, in contrast to the former, is not a well-established idea. This is a disturbing fact since knowledge acquisition can give significant improvements in performance of a query optimization algorithm. Performing analysis manually is extrernely time consuming and tedious. Therefore, some learning capacity should be added to the system. Some knowledge acquisition techniques that result in adaptive (dynamic) adjustment to run-time changes are proposed.

David Brill

Papers

Query Processing in a Fragmented Relational Distributed System: Mermaid

Mermaid — Experiences with network operation

Distributed query processing strategies in Mermaid, a frontend to data management systems

Adaptive techniques for distributed query optimization

Partitioning Relation for Parallel Processing in Fast Local Networks.