From the Publisher:
Data warehouses have captured the attention of practitioners and researchers alike. But the design and optimization of data warehouses remains an art rather than a science. This book presents a comparative review of the state of the art and best current practice of data warehouses. It covers source and data integration, multidimensional aggregation, query optimization, update propagation, metadata management, quality assessment, and design optimization. Also, based on results of the European Data Warehouse Quality project, it offers a conceptual framework by which the architecture and quality of data warehouse efforts can be assessed and improved using enriched metadata management combined with advanced techniques from databases, business modeling, and artificial intelligence. For researchers and database professionals in academia and industry, the book offers an excellent introduction to the issues of quality and metadata usage in the context of data warehouses.

Fundamentals of Data Warehouses

There is a tremendous interest in big data by academia, industry and a large user base. Several commercial and open source providers unleashed a variety of products to support big data storage and processing. As these products mature, there is a need to evaluate and compare the performance of these systems.In this paper, we present BigBench, an end-to-end big data benchmark proposal. The underlying business model of BigBench is a product retailer. The proposal covers a data model and synthetic data generator that addresses the variety, velocity and volume aspects of big data systems containing structured, semi-structured and unstructured data. The structured part of the BigBench data model is adopted from the TPC-DS benchmark, which is enriched with semi-structured and unstructured data components. The semi-structured part captures registered and guest user clicks on the retailer's website. The unstructured data captures product reviews submitted online. The data generator designed for BigBench provides scalable volumes of raw data based on a scale factor. The BigBench workload is designed around a set of queries against the data model. From a business prospective, the queries cover the different categories of big data analytics proposed by McKinsey. From a technical prospective, the queries are designed to span three different dimensions based on data sources, query processing types and analytic techniques.We illustrate the feasibility of BigBench by implementing it on the Teradata Aster Database. The test includes generating and loading a 200 Gigabyte BigBench data set and testing the workload by executing the BigBench queries (written using Teradata Aster SQL-MR) and reporting their response times.

BigBench: towards an industry standard benchmark for big data analytics

The tradeoffs between consistency, performance, and availability are well understood. Traditionally, however, designers of replicated systems have been forced to choose from either strong consistency guarantees or none at all. This paper explores the semantic space between traditional strong and optimistic consistency models for replicated services. We argue that an important class of applications can tolerate relaxed consistency, but benefit from bounding the maximum rate of inconsistent access in an application-specific manner. Thus, we develop a conit-based continuous consistency model to capture the consistency spectrum using three application-independent metrics, numerical error, order error, and staleness. We then present the design and implementation of TACT, a middleware layer that enforces arbitrary consistency bounds among replicas using these metrics. We argue that the TACT consistency model can simultaneously achieve the often conflicting goals of generality and practicality by describing how a broad range of applications can express their consistency semantics using TACT and by demonstrating that application-independent algorithms can efficiently enforce target consistency levels. Finally, we show that three replicated applications running across the Internet demonstrate significant semantic and performance benefits from using our framework.

/pdf/design-and-evaluation-of-a-conit-based-continuous-pyunl5lyxe.pdf

Design and evaluation of a conit-based continuous consistency model for replicated services

 \ N\| N\wOEN^|< e\ eNNN¡  ^  N¢X\£<¤ N¥N ¦||§N| ̈\<©RN\dV\a | aN ¦|<§ « \\NX¡;aUN\ O¢X\ N¬  ^  \NW \0\|aUNN­@®u7 | a^NN¡e¤o  O¤ \ a \ aN\`¦||a |\ ̄ 0§N WJ§N|\§X a ¢N<N <£  | 7 §Xa  | \| O « aO¤ I J O¢X¦ \X§° a |± aNN­ 20\NO¢NN¡¤@\ O¤3 O¤¬ a\O3 ^\X \¦| \| §X ^9| §N\^\\NU| &&N¦  \a\w\£<\Na4­e ́ * WXjaμ§N \N*  N¤  \<§N¦|  ` N| |a  ¦|NaUN<\\| |·¶[\u3 _ ̧ 1 oe­R®u¦|\  O¢<μaN9N»<^N\|aUN<\¦[\N ¦|\  O¢ | @\ a a <£1⁄4j\ a& aN  0¦||a |\1⁄43 \|  O \N ¦||§N\| §NJ^4O¢μaN <£  μa \ a    μ\ \X§° a |± aNN­

/pdf/don-t-be-lazy-be-consistent-postgres-r-a-new-way-to-ygao4pfddr.pdf

Don't Be Lazy, Be Consistent: Postgres-R, A New Way to Implement Database Replication

This paper describes Jaql, a declarative scripting language for analyzing large semistructured datasets in parallel using Hadoop's MapReduce framework Jaql is currently used in IBM's InfoSphere BigInsights [5] and Cognos Consumer Insight [9] products Jaql's design features are: (1) a flexible data model, (2) reusability, (3) varying levels of abstraction, and (4) scalability Jaql's data model is inspired by JSON and can be used to represent datasets that vary from flat, relational tables to collections of semistructured documents A Jaql script can start without any schema and evolve over time from a partial to a rigid schema Reusability is provided through the use of higher-order functions and by packaging related functions into modules Most Jaql scripts work at a high level of abstraction for concise specification of logical operations (eg, join), but Jaql's notion of physical transparency also provides a lower level of abstraction if necessary This allows users to pin down the evaluation plan of a script for greater control or even add new operators The Jaql compiler automatically rewrites Jaql scripts so they can run in parallel on Hadoop In addition to describing Jaql's design, we present the results of scale-up experiments on Hadoop running Jaql scripts for intranet data analysis and log processing

/pdf/jaql-a-scripting-language-for-large-scale-semistructured-qrc1ux411l.pdf

Jaql: a scripting language for large scale semistructured data analysis

XML parsing is generally known to have poor performance characteristics relative to transactional database processing. Yet, its potentially fatal impact on overall database performance is being underestimated. We report real-word database applications where XML parsing performance is a key obstacle to a successful XML deployment. There is a considerable share of XML database applications which are prone to fail at an early and simple road block: XML parsing. We analyze XML parsing performance and quantify the extra overhead of DTD and schema validation. Comparison with relational database performance shows that the desired response times and transaction rates over XML data can not be achieved without major improvements in XML parsing technology. Thus, we identify research topics which are most promising for XML parser performance in database systems.

https://lists.w3.org/Archives/Public/www-ws/2004Oct/att-0032/MNicola_CIKM_2003_1_.pdf

XML parsing: a threat to database performance

The paper surveys performance models for distributed and replicated database systems. Over the last 20 years (1980-2000), a variety of such performance models have been developed and they differ in: (1) which aspects of a real system are or are not captured in the model (e.g., replication, communication, nonuniform data access, etc.); and (2) how these aspects are modeled. We classify the different alternatives and modeling assumptions and discuss their interdependencies and expressiveness for the representation of distributed databases. This leads to a set of building blocks for analytical performance models. To illustrate the work that is surveyed, we select a combination of these proven modeling concepts and give an example of how to compose a balanced analytical model of a replicated database. We use this example to show how to derive meaningful performance values and to discuss the applicability and expressiveness of performance models for distributed and replicated databases. Finally, we compare the analytical results to measurements in a distributed database system.

/pdf/performance-modeling-of-distributed-and-replicated-databases-1czpwcjl7p.pdf

Performance modeling of distributed and replicated databases

The major relational database systems have been providing XML support for several years, predominantly by mapping XML to existing concepts such as LOBs or (object-)relational tables. The limitations of these approaches are well known in research and industry. Thus, a forthcoming version of DB2 Universal Database® is enhanced with comprehensive native XML support. "Native" means that XML documents are stored on disk pages in tree structures matching the XML data model. This avoids the mapping between XML and relational structures, and the corresponding limitations. The native XML storage is complemented with XML indexes, full XQuery, SQL/XML, and XML Schema support, as well as utilities such as a parallel high-speed XML bulk loader. This makes DB2 a true hybrid database system which places equal weight on XML and relational data management.

/pdf/native-xml-support-in-db2-universal-database-43udclnwml.pdf

Native XML support in DB2 universal database

XML database functionality has been emerging in "XML-only" databases as well as in the major relational database products. Yet, there is no industry standard XML database benchmark to evaluate alternative implementations. The research community has proposed several benchmarks which are all useful in their respective scope, such as evaluating XQuery processors. However, they do not aim to evaluate a database system in its entirety and do not represent all relevant characteristics of a real-world XML application. Often they only define read-only single-user tests on a single XML document. We have developed an application-oriented and domain-specific benchmark called "Transaction Processing over XML" (TPoX). It exercises all aspects of XML databases, including storage, indexing, logging, transaction processing, and concurrency control. Based on our analysis of real XML applications, TPoX simulates a financial multi-user workload with XML data conforming to the FIXML standard. In this paper we describe TPoX and present early performance results. We also make its implementation publicly available.

/pdf/an-xml-transaction-processing-benchmark-1wlb1js77i.pdf

An XML transaction processing benchmark

Applications in finance and telecommunications (intelligent network, network management, mobile computing) cause renewed interest in distributed and replicated data management. Since synchronous update of replicated data is experienced to degrade distributed systems performance substantially, relaxing the requirement of coherency (mutual consistency) has become a favorable approach to achieve high performance in replicated databases. In this paper we present formal concepts for specifying relaxed coherency which allows to calculate an independent measure of relaxation, called + coherency index. We incorporate this metric into a detailed analytical queueing model which emphasizes on the quality of replication to evaluate the impact of relaxed coherency on the performance of replicated databases. The model considers response time, throughput, scalability and network traffic as performance criteria. As it turns out, performance improvements through relaxed coherency depend significantly on various system parameters. We closely examine the trade off between consistency and performance, and show that in many situations a slight relaxation of coherency can increase performance remarkably. Finally, we verify these results by benchmarking an implementation of relaxed coherency.

/pdf/improving-performance-in-replicated-databases-through-2l221zysut.pdf

Matthias Nicola

Papers

XML parsing: a threat to database performance

Performance modeling of distributed and replicated databases

Native XML support in DB2 universal database

An XML transaction processing benchmark

Improving Performance in Replicated Databases through Relaxed Coherency