Showing papers on "Distributed database published in 2000"

The state of the art in distributed query processing

Abstract: Distributed data processing is becoming a reality. Businesses want to do it for many reasons, and they often must do it in order to stay competitive. While much of the infrastructure for distributed data processing is already there (e.g., modern network technology), a number of issues make distributed data processing still a complex undertaking: (1) distributed systems can become very large, involving thousands of heterogeneous sites including PCs and mainframe server machines; (2) the state of a distributed system changes rapidly because the load of sites varies over time and new sites are added to the system; (3) legacy systems need to be integrated—such legacy systems usually have not been designed for distributed data processing and now need to interact with other (modern) systems in a distributed environment. This paper presents the state of the art of query processing for distributed database and information systems. The paper presents the “textbook” architecture for distributed query processing and a series of techniques that are particularly useful for distributed database systems. These techniques include special join techniques, techniques to exploit intraquery paralleli sm, techniques to reduce communication costs, and techniques to exploit caching and replication of data. Furthermore, the paper discusses different kinds of distributed systems such as client-server, middleware (multitier), and heterogeneous database systems, and shows how query processing works in these systems.

Querying the physical world

Abstract: In the next decade, millions of sensors and small-scale mobile devices will integrate processors, memory, and communication capabilities. Networks of devices will be widely deployed for monitoring applications. In these new applications, users need to query very large collections of devices in an ad hoc manner. Most existing systems rely on a centralized system for collecting device data. These systems lack flexibility because data is extracted in a predefined way. Also, they do not scale to a large number of devices because large volumes of raw data are transferred. In our new concept of a device database system, distributed query execution techniques are applied to leverage the computing capabilities of devices, and to reduce communication. We define an abstraction that allows us to represent a device network as a database and we describe how distributed query processing techniques are applied in this new context.

Understanding replication in databases and distributed systems

Abstract: Replication is an area of interest to both distributed systems and databases. The solutions developed from these two perspectives are conceptually similar but differ in many aspects: model, assumptions, mechanisms, guarantees provided, and implementation. In this paper, we provide an abstract and "neutral" framework to compare replication techniques from both communities. The framework has been designed to emphasize the role played by different mechanisms and to facilitate comparisons. The paper describes the replication techniques used in both communities, compares them, and points out ways in which they can be integrated to arrive to better, more robust replication protocols.

Accessing and updating a configuration database from distributed physical locations within a process control system

Abstract: A configuration database includes multiple databases distributed at a plurality of physical locations within a process control system. Each of the databases may store a subset of the configuration data and this subset of configuration data may be accessed by users at any of the sites within the process control system. A database server having a shared cache accesses a database in a manner that enables multiple subscribers to read configuration data from the database with only a minimal number of reads to the database. To prevent the configuration data being viewed by subscribers within the process control system from becoming stale, the database server automatically detects changes to an item within the configuration database and sends notifications of changes made to the item to each of the subscribers of that item so that a user always views the state of the configuration as it actually exists within the configuration database.

Survivable information storage systems

Abstract: As society increasingly relies on digitally stored and accessed information, supporting the availability, integrity and confidentiality of this information is crucial. We need systems in which users can securely store critical information, ensuring that it persists, is continuously accessible, cannot be destroyed and is kept confidential. A survivable storage system would provide these guarantees over time and despite malicious compromises of storage node subsets. The PASIS architecture flexibly and efficiently combines proven technologies (decentralized storage system technologies, data redundancy and encoding, and dynamic self-maintenance) for constructing information storage systems whose availability, confidentiality and integrity policies can survive component failures and malicious attacks.

A new approach to developing and implementing eager database replication protocols

Abstract: Database replication is traditionally seen as a way to increase the availability and performance of distributed databases. Although a large number of protocols providing data consistency and fault-tolerance have been proposed, few of these ideas have ever been used in commercial products due to their complexity and performance implications. Instead, current products allow inconsistencies and often resort to centralized approaches which eliminates some of the advantages of replication. As an alternative, we propose a suite of replication protocols that addresses the main problems related to database replication. On the one hand, our protocols maintain data consistency and the same transactional semantics found in centralized systems. On the other hand, they provide flexibility and reasonable performance. To do so, our protocols take advantage of the rich semantics of group communication primitives and the relaxed isolation guarantees provided by most databases. This allows us to eliminate the possibility of deadlocks, reduce the message overhead and increase performance. A detailed simulation study shows the feasibility of the approach and the flexibility with which different types of bottlenecks can be circumvented.

Database load balancing for multi-tier computer systems

Abstract: A load balancing method and system for a transaction computer system having multiple database servers for at least one database, wherein database servers cooperate to provide a unified view of the data in the database. The method includes the steps of establishing connections to said multiple database servers for communicating with said database servers; and assigning transactions to respective ones of said multiple database servers to balance respective loads of said multiple database servers. Assigning each new transaction includes the steps of determining possible assignments of that new transaction to one or more of said multiple database servers, each said possible assignment to one of said multiple database servers being based on a load balancing scheme to balance respective loads of said multiple database servers; and assigning that new transaction to one of said multiple database servers as a function of said possible assignments of that new transaction, to balance respective loads of said multiple database servers.

Internet database system

Abstract: An incrementally-scalable database system and method. The system architecture enables database servers to be scaled by adding resources, such as additional servers, without requiring that the system be taken offline. Such scaling includes both adding one or more computer servers to a given server cluster, which enables an increase in database read transaction throughput, and adding one or more server clusters to the system configuration, which provides for increased read and write transaction throughput. The system also provides for load balancing read transactions across each server cluster, and load balancing write transactions across a plurality of server clusters. The system architecture includes an application server layer including one or more computers on which an application program(s) is running, a database server layer comprising two or more server clusters that each include two or more computer servers with replicated data, and an intermediate “virtual transaction” layer that includes at least two computers that facilitate database transactions with one or more databases operating in the database server layer. Data in the database(s) are evenly distributed across the server clusters in fragmented mutually exclusive subsets of data based on a hashing function. An application program interface is provided so as to enable application programs to perform a full range of database transactions without regard for where data is stored, or what database(s) is operating in the database server layer.

Virtual backbone generation and maintenance in ad hoc network mobility management

Abstract: In this paper, we present the implementation issues of a virtual backbone that supports the operations of the uniform quorum system (UQS) and the randomized database group (RDG) mobility management schemes in an ad hoc network. The virtual backbone comprises nodes that are dynamically selected to contain databases that store the location information of the network nodes. Together with the UQS and RDG schemes, the virtual backbone allows both dynamic database residence and dynamic database access, which provide high degree of location data availability and reliability. We introduce a distributed database coverage heuristic (DDCH), which is equivalent to the centralized greedy algorithm for virtual backbone generation, but only requires local information exchange and local computation. We show how DDCH can be employed to dynamically maintain the structure of the virtual backbone, along with database merging, as the network topology changes. We also provide a means to maintain connectivity among the virtual backbone nodes. We discuss optimization issues of DDCH through simulations. Simulation results suggest that the cost of ad hoc mobility management with a virtual backbone can be far below that of the conventional link-state routing.

Methods and systems for synchronization of mobile devices with a remote database

Abstract: A computer system for synchronizing a mobile device with a remote enterprise database includes a replication database. The replication database is mapped to and maintained as a subset image of the remote enterprise database through a bi-directional replication process. A mobile device gateway server is coupled to the replication database. The mobile device gateway includes a concentrator, which is configured to selectively receive replication data from the replication database, transform the received replication data into a format native to the mobile device and to selectively send the received and transformed replication data to the mobile device. The concentrator is also configured to receive mobile device data, transform and to send the received mobile device data to the replication database. The mobile device gateway server acts as a middle tier between one or more enterprise databases and one or more mobile devices or any network-enabled thin client.

Performance modeling of distributed and replicated databases

Abstract: The paper surveys performance models for distributed and replicated database systems. Over the last 20 years (1980-2000), a variety of such performance models have been developed and they differ in: (1) which aspects of a real system are or are not captured in the model (e.g., replication, communication, nonuniform data access, etc.); and (2) how these aspects are modeled. We classify the different alternatives and modeling assumptions and discuss their interdependencies and expressiveness for the representation of distributed databases. This leads to a set of building blocks for analytical performance models. To illustrate the work that is surveyed, we select a combination of these proven modeling concepts and give an example of how to compose a balanced analytical model of a replicated database. We use this example to show how to derive meaningful performance values and to discuss the applicability and expressiveness of performance models for distributed and replicated databases. Finally, we compare the analytical results to measurements in a distributed database system.

Image retrieval by examples

Abstract: A currently important research field in information sciences is the management of nontraditional distributed multimedia databases. Two related key issues are to achieve an efficient content-based query by example retrieval and a fast response time. This paper presents the architecture of a distributed image retrieval system which provides novel solutions to these key issues. In particular, a method to quantify the effectiveness of low level visual descriptors in database query tasks is presented. The results are also used to improve the system response time, which is an important issue when querying very large databases. A new mechanism to adapt system query strategies to user behavior is also introduced in order to improve the effectiveness of relevance feedback and overall system response time. Finally, the issue of browsing multiple distributed databases is considered and a solution is proposed using multidimensional scaling techniques.

Scalable Replication in Database Clusters

Abstract: In this paper, we explore data replication protocols that provide both fault tolerance and good performance without compromising consistency. We do this by combining transactional concurrency control with group communication primitives. In our approach, transactions are executed at only one site so that not all nodes incur in the overhead of producing results. To further reduce latency, we use an optimistic multicast technique that overlaps transaction execution with total order message delivery. The protocols we present in the paper provide correct executions while minimizing overhead and providing higher scalability.

Mobile agents for World Wide Web distributed database access

Abstract: The popularity of the Web as a universal access mechanism for network information has created the need for developing Web-based DBMS client/server applications. However, the current commercial applet-based approaches for accessing database systems offer limited flexibility, scalability, and robustness. We propose a new framework for Web-based distributed access to database systems based on Java-based mobile agents. The framework supports lightweight, portable, and autonomous clients as well as operation on slow or expensive networks. The implementation of the framework using the aglet workbench shows that its performance is comparable to, and in some case outperforms, the current approach. In fact, in wireless and dial-up environments and for average size transactions, a client/agent/server adaptation of the framework provides a performance improvement of approximately a factor of ten. For the fixed network, the gains are about 40 percent and 30 percent, respectively. We expect our framework to perform even better when deployed using different implementation platforms as indicated by our preliminary results from an implementation based on Voyager.

System and method for managing the configuration of hierarchically networked data processing devices

Abstract: Described is an IT network (1) comprising distributed managers (2) and agents (3) which are arranged hierarchically in correspondence with an underlying likewise hierarchically arranged IT environment. Further a distributed database (5), or respective parts thereof, is provided for each manager (2) for storing information (6) about the configuration of the IT devices (4). For each IT device (4), an agent (3) is provided for retrieving the configuration information (6) for that IT device (4) from the database (5). The managers (2) and agents (3) use the computing power of the entire managed IT environment, whereby the managers (2) are implemented on IT servers (7) and the agents (3) are implemented on their corresponding IT devices (4). The agents (3) comprise intelligent behaviour insofar as they determine any necessary changes in the configuration of an IT device (4) being arranged at the same network hierarchy level or below that level independently on their own.

Scalable distributed data structures for database management

Abstract: t MainMain memory is continuously improving both in price and capacity. With thisthis comes new storage problems as well as new directions of usage. Just beforebefore the millennium, several main memory database systems are becoming commerciallycommercially available. The hot areas include boosting the performance of web-enabledweb-enabled systems, such as search-engines, and auctioning systems. WeWe present a novel data storage structure the fi-storage structure, aa high performance data structure, allowing automatically indexed storage ofof very large amounts of multi-attribute data. The experiments show excellentlent performance for point retrieval, and highly efficient pruning for pattern searches .. It provides the balanced storage previously achieved by random kdtrees,trees, but avoids their increased pattern match search times, by an effective

Database intergration using neural networks: implementation and experiences

Abstract: Applications in a wide variety of industries require access to multiple heterogeneous distributed databases. One step in heterogeneous database integration is semantic integration: identifying corresponding attributes in different databases that represent the same real world concept. The rules of semantic integration can not be ‘pre-programmed’ since the information to be accessed is heterogeneous and attribute correspondences could be fuzzy. Manually comparing all possible pairs of attributes is an unreasonably large task. We have applied artificial neural networks (ANNs) to this problem. Metadata describing attributes is automatically extracted from a database to represent their ‘signatures’. The metadata is used to train neural networks to find similar patterns of metadata describing corresponding attributes from other databases. In our system, the rules to determine corresponding attributes are discovered through machine learning. This paper describes how we applied neural network techniques in a database integration problem and how we represent an attribute with its metadata as discriminators. This paper focuses on our experiments on effectiveness of neural networks and each discriminator. We also discuss difficulties of using neural networks for this problem and our wish list for the Machine Learning community.

Update propagation strategies to improve freshness in lazy master replicated databases

Abstract: Many distributed database applications need to replicate data to improve data availability and query response time. The two-phase commit protocol guarantees mutual consistency of replicated data but does not provide good performance. Lazy replication has been used as an alternative solution in several types of applications such as on-line financial transactions and telecommunication systems. In this case, mutual consistency is relaxed and the concept of freshness is used to measure the deviation between replica copies. In this paper, we propose two update propagation strategies that improve freshness. Both of them use immediate propagation: updates to a primary copy are propagated towards a slave node as soon as they are detected at the master node without waiting for the commitment of the update transaction. Our performance study shows that our strategies can improve data freshness by up to five times compared with the deferred approach.

Method, system, and computer program product for managing database servers and service

Abstract: A system, method, and computer program product for managing database servers and services is disclosed, which expands the capabilities of existing systems by providing better scalability and security. A database management system includes databases consisting of database clients and servers that handle the storing of information, a multi-database client that manages database servers, service modules which allow other systems to access the database servers, and a database access manager that manages requests from the service modules. A database access manager facilitates and coordinates interaction between the service modules and the multi-database client. Service modules provide interfaces from which other external or internal systems can access the managed databases. Requests can be sent between systems and the database access manager via the service modules. Two present and representative implementations of service modules are: user management and call detail record (CDR) collection. The user management service module allows systems to access user account information stored in the databases. The CDR collection service module inserts details for each call made on the network.

View Security as the Basis for Data Warehouse Security

Abstract: Access . permissions in a data warehouse are currently managed in a separate world from the sources’ policies. The consequences are inconsistencies, slow response to change, and wasted administrative work. We present a different approach, which treats the sources’ exported tables and the warehouse as part of the same distributed database. Our main result is a way to control derived products by extending SQL grants rather than creating entirely new mechanisms. We provide a powerful, sound inference theory that derives permissions on warehouse tables (both materialized and virtual), making the system easier to administer and its applications more robust. We also propose a new permission construct suitable for views that filter data from mutually-suspicious parties.

Method and apparatus for database fault tolerance with instant transaction replication using off-the-shelf database servers and low bandwidth networks

Abstract: This invention concerns providing a fault tolerant data service using multiple redundant off-the-shelf-database servers connected by low bandwidth networks. The data service continues operation regardless partial processing hardware failures, network failures, operating system failures and incidental database malfunctions. A database gateway automatically replicates database communication packets and monitors the wellness of communication networks and supporting database servers. It can quickly identify a unstable connection or a database server, thus providing highly reliable data service to all clients. An algorithm is disclosed to accomplish these tasks using minimal original client data.

Modern Database Management

Abstract: From the Publisher: Reacting to the current business environment, Modern Database Management, 6/e addresses current issues in the market, such as Internet, data warehousing and object-orientation. While sufficient technical explanations are given, the book instructs from a business perspective, allowing readers to understand the role of database management within a business. Chapter topics cover the database environment and development process, modeling data in the organization, advanced data modeling, logical database design and the relational model, physical database design and performance, SQL, advanced SQL, the client/server environment, the Internet environment, data warehousing, data and database administration, distributed databases, object-oriented data modeling, and object-oriented database development.

Distributed maintenance of multiple product views

Abstract: We present three mechanisms for maintaining consistent product views in a distributed product information database. The mechanisms are used when one of the views makes a change to the product model and the other views must be updated to maintain consistency.

Method and system for database query

Abstract: A method and system is disclosed for searching a database resident list or set of lists for user-defined text patterns, identifying a match, reporting information related to the search result, and maintaining the lists to be searched in a mutually updating distributed database with automatic failover. One embodiment of the present invention provides a method of using a computer utility program to maintain a database of sanctioned entities in a plurality of sanction lists where such sanction lists arise, for example, from OFAC, foreign governments, or internally. The computer utility program allows an authorized user to search user-selected text patterns against the sanction lists to identify a sanctioned entity. Further, the computer utility program is network based so that sanction lists may be searched using different hardware configurations worldwide. In addition, because the database is distributed and mutually updating, sanction lists may be dynamically updated globally and are completely redundant.

Privacy-protected advertising system

Abstract: A database of consumer profiles is generated from multiple sources of information including demographic databases identifying demographic attributes of the consumers and transaction records for the consumers. The transaction records are processed to generate transaction attributes and interests of the consumer. The consumer profiles identify deterministic and probabilistic attributes about the consumer, but do not contain privacy violating information such as raw transaction records. The consumer profiles may be maintained in a plurality of distributed databases. Advertisers generate profiles that identify attributes of an intended target market of the advertisement. The advertisement profiles are in the form of operators that can be applied to the database of consumer profiles to determine applicability of advertisements to the subscribers. The operators may only be applied to or make measurements on certain “observables”. The operators will not be able to obtain private information from the database of consumer profiles.

Content-based retrieval of 3D models in distributed Web databases by visual shape information

Abstract: Digital documents assembled from different components, like text, images, video, audio and 3D models, are supported today by many distributed applications. This paper presents a new approach to retrieving 3D models as part of digital documents using content-based image retrieval functionality. To find a 3D model best-fitting the user's query, shape information is extracted from the retrieved models and a score is generated by comparing shape images with the graphical user input. A distributed database environment with a WWW interface is presented and used as a generic implementation platform for this technique. The new retrieval method for 3D geometry is then described by extending the generic architecture in two steps, with a server-side module and an interactive client display. I outline how this method can be used for partial geometry downloading of 3D models in digital libraries.

Servicing database requests using read-only database servers coupled to a master database server

Abstract: One embodiment of the present invention provides a system that speeds up an application by allowing database tables associated with the application to be partitioned across N read-only slave database servers. In this way, each slave database server can process 1/Nth of the workload of the application. Since this processing can take place in parallel, the system can potentially speed up the application by a factor of N. The system includes a master-slave database hierarchy, which includes a plurality of read-only database servers for servicing read operations and a master database server for servicing both read operations and write operations. This master database server contains an entire database and the plurality of read-only database servers contain local read-only copies of portions of the database. The system operates by receiving a database request from an application server at a read-only database server.

Supporting dynamic interactions among Web-based information sources

Abstract: The ubiquity of the World Wide Web offers an ideal opportunity for the deployment of highly distributed applications. Now that connectivity is no longer an issue, attention has turned to providing a middleware infrastructure that will sustain data sharing among Web-accessible databases. We present a dynamic architecture and system for describing, locating, and accessing data from Web-accessible databases. We propose the use of flexible organizational constructs service links and coalitions to facilitate data organization, discovery, and sharing among Internet-accessible databases. A language is also proposed to support the definition and manipulation of these constructs. The implementation combines Java, CORBA, database API (JDBC), agent, and database technologies to support a scalable and portable architecture interconnecting large networks of heterogeneous and autonomous databases. We report on an experiment to provide uniform access to a Web of healthcare-related databases.

Grid-based data management in distributed simulation

Abstract: The data distribution management (DDM) service provided by the High Level Architecture (HLA) run-time infrastructure (RTI) is aimed at making data communication more efficient by sending data only to those federates who need the data, as opposed to the broadcasting mechanism employed by distributed interactive simulation (DIS). Several DDM methods, including class-, region and grid-based, have appeared in the literature. This paper discusses grid-based DDM and presents a simulation platform developed and implemented for investigating the impact of grid-based filtering on performance. Our experimental results show that the choice of the grid cell size has a substantial impact on the performance of distributed simulation. The optimal cell size depends on the characteristics of the application and the underlying computer system. In many practical simulations, it is not easy to determine the optimal cell size. Our experience indicates that in such a case, one should use larger cells rather than very small ones.