The problem of answering queries using views is to find efficient methods of answering a query using a set of previously defined materialized views over the database, rather than accessing the database relations. The problem has recently received significant attention because of its relevance to a wide variety of data management problems. In query optimization, finding a rewriting of a query using a set of materialized views can yield a more efficient query execution plan. To support the separation of the logical and physical views of data, a storage schema can be described using views over the logical schema. As a result, finding a query execution plan that accesses the storage amounts to solving the problem of answering queries using views. Finally, the problem arises in data integration systems, where data sources can be described as precomputed views over a mediated schema. This article surveys the state of the art on the problem of answering queries using views, and synthesizes the disparate works into a coherent framework. We describe the different applications of the problem, the algorithms proposed to solve it and the relevant theoretical results.

/pdf/answering-queries-using-views-a-survey-4aug5icc6d.pdf

Answering queries using views: A survey

An object is to provide a semiconductor device of which a manufacturing process is not complicated and by which cost can be suppressed, by forming a thin film transistor using an oxide semiconductor film typified by zinc oxide, and a manufacturing method thereof. For the semiconductor device, a gate electrode is formed over a substrate; a gate insulating film is formed covering the gate electrode; an oxide semiconductor film is formed over the gate insulating film; and a first conductive film and a second conductive film are formed over the oxide semiconductor film. The oxide semiconductor film has at least a crystallized region in a channel region.

Semiconductor device, and manufacturing method thereof

Distributed data processing is becoming a reality. Businesses want to do it for many reasons, and they often must do it in order to stay competitive. While much of the infrastructure for distributed data processing is already there (e.g., modern network technology), a number of issues make distributed data processing still a complex undertaking: (1) distributed systems can become very large, involving thousands of heterogeneous sites including PCs and mainframe server machines; (2) the state of a distributed system changes rapidly because the load of sites varies over time and new sites are added to the system; (3) legacy systems need to be integrated—such legacy systems usually have not been designed for distributed data processing and now need to interact with other (modern) systems in a distributed environment. This paper presents the state of the art of query processing for distributed database and information systems. The paper presents the “textbook” architecture for distributed query processing and a series of techniques that are particularly useful for distributed database systems. These techniques include special join techniques, techniques to exploit intraquery paralleli sm, techniques to reduce communication costs, and techniques to exploit caching and replication of data. Furthermore, the paper discusses different kinds of distributed systems such as client-server, middleware (multitier), and heterogeneous database systems, and shows how query processing works in these systems.

/pdf/the-state-of-the-art-in-distributed-query-processing-2b23xet25f.pdf

The state of the art in distributed query processing

Process gas discharged from a bypass pipe to a gas exhaust system can be prevented from diffusing back to the inside of a process chamber without having to install a dedicated vacuum pump at the downstream side of the bypass pipe. The substrate processing apparatus includes a process chamber accommodating a substrate, a gas supply system supplying process gas from a process gas source to the process chamber for processing the substrate, a gas exhaust system configured to exhaust the process chamber, two or more vacuum pumps installed in series at the gas exhaust system, and a bypass pipe connected between the gas supply system and the gas exhaust system. The most upstream one of the vacuum pumps is a mechanical booster pump, and the bypass pipe is connected between the mechanical booster pump and the rest vacuum pumps located at a downstream side of the mechanical booster pump.

Substrate Processing Apparatus

Storm has long served as the main platform for real-time analytics at Twitter. However, as the scale of data being processed in real-time at Twitter has increased, along with an increase in the diversity and the number of use cases, many limitations of Storm have become apparent. We need a system that scales better, has better debug-ability, has better performance, and is easier to manage -- all while working in a shared cluster infrastructure. We considered various alternatives to meet these needs, and in the end concluded that we needed to build a new real-time stream data processing system. This paper presents the design and implementation of this new system, called Heron. Heron is now the de facto stream data processing engine inside Twitter, and in this paper we also share our experiences from running Heron in production. In this paper, we also provide empirical evidence demonstrating the efficiency and scalability of Heron.

/pdf/twitter-heron-stream-processing-at-scale-4xgl2u36tr.pdf

Twitter Heron: Stream Processing at Scale

A method and system is provided for processing queries. Specifically, techniques are provided for handling a query that does not reference a particular materialized view, where the query requires access to values from a particular column not contained in the materialized view. A technique is also provided for processing a query that does not reference a particular materialized aggregate view, where the materialized aggregate view specifies an outer join between a child table and a parent table and the query specifies a particular type of join between the child table and the parent table, where the particular type of join is one of an inner join, an anti-join and a semi-join. The query is rewritten to produce a rewritten query that accesses the materialized aggregate view to produce data required by the query. A technique is also provided for processing a query that does not reference a particular materialized view and that specifies that results are to be grouped by a first set of one or more columns, where the materialized view reflects data that is grouped by a second set of one or more columns. If each column in the first set of columns either exactly matches a column in the second set of columns, is functionally dependent on another column in the first set of columns, or is functionally dependent on a column in the second set of columns, then the query is rewritten to produce a rewritten query that references the materialized view.

Rewriting a query in terms of a summary based on functional dependencies and join backs, and based on join derivability

Oracle Materialized Views (MVs) are designed for data warehousing and replication. For data warehousing, MVs based on inner/outer equijoins with optional aggregation, can be refreshed on transaction boundaries, on demand, or periodically. Refreshes are optimized for bulk loads and can use a multi-MV scheduler. MVs based on subqueries on remote tables support bidirectional replication. Optimization with MVs includes transparent query rewrite based on costbased selection method. The ability to rewrite a large class of queries based on a small set of MVs is supported by using Dimensions (new Oracle object), losslessness of joins, functional dependency, column equivalence, join derivability, joinback and aggregate rollup.

/pdf/materialized-views-in-oracle-4mafinkdcn.pdf

Materialized Views in Oracle

An aggregate index is used for accessing an aggregate value associated with one or more rows of a database table. The aggregate values are stored in the index entries of the index, thus allowing determination of aggregate values without accessing the underlying database table. The aggregate index is created by determining an aggregate value associated with an index entry, and storing the aggregate value with the index entry. The aggregate value is determined by accessing information corresponding to the index entry and processing the information. In the case of an index entry in the form of a branch node, the information might be aggregate values from other nodes, or values of items in the database pointed to by the index entries that are leaf nodes. The aggregate values may be stored in index entries of any type of index.

Apparatus and method for aggregate indexes

One of the critical deficiencies of SQL is lack of support for n-dimensional array-based computations which are frequent in OLAP environments. Relational OLAP (ROLAP) applications have to emulate them using joins, recently introduced SQL Window Functions [18] and complex and inefficient CASE expressions. The designated place in SQL for specifying calculations is the SELECT clause, which is extremely limiting and forces the user to generate queries using nested views, subqueries and complex joins. Furthermore, SQL-query optimizer is pre-occupied with determining efficient join orders and choosing optimal access methods and largely disregards optimization of complex numerical formulas. Execution methods concentrated on efficient computation of a cube [11], [16] rather than on random access structures for inter-row calculations. This has created a gap that has been filled by spreadsheets and specialized MOLAP engines, which are good at formulas for mathematical modeling but lack the formalism of the relational model, are difficult to manage, and exhibit scalability problems. This paper presents SQL extensions involving array based calculations for complex modeling. In addition, we present optimizations, access structures and execution models for processing them efficiently.

/pdf/spreadsheets-in-rdbms-for-olap-4gvx9scqyt.pdf

Spreadsheets in RDBMS for OLAP

A method and apparatus are provided for performing incremental refreshes to materialized views defined by one-to-one lossless joins. Each base table of the materialized view is selected to be processed as the current "selected table". During the processing, if the selected table is the right table of an outer join, then selected columns of rows of the materialized view are set to NULL based on rows of the selected table that have been updated or deleted in the selected table after a prior refresh operation. Then, NULL values in selected columns of the materialized view are updated to reflect new values caused by updates and inserts into the selected table that occurred after the prior refresh operation. If the selected table is not the right table of an outer join, then rows from the materialized view are deleted based on rows of the selected table that have been updated or deleted in the selected table after the prior refresh operation. Then, rows are inserted into the materialized view based on updates and inserts into the selected table that occurred after the prior refresh operation. The incremental refresh techniques are "memoryless" in that they do not require a record of the sequence of changes that were made subsequent to the prior refresh operation. Techniques are described for performing the incremental refresh steps through the use of database commands and queries.

Andrew Witkowski

Papers

Rewriting a query in terms of a summary based on functional dependencies and join backs, and based on join derivability

Materialized Views in Oracle

Apparatus and method for aggregate indexes

Spreadsheets in RDBMS for OLAP

Incremental maintenance of materialized views containing one-to-one lossless joins