scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Dynamic incremental maintenance of materialized view based on attribute affinity

04 Dec 2014-pp 12-17
TL;DR: In this paper authors adopt an incremental view maintenance policy based on attribute affinity to update the materialized views at run time without using extra space and minimizing the data transfer between the secondary memory and primary memory (where the active materialization views reside).
Abstract: View materialization is being practiced over several years in large data centric applications like database, data warehouse, data mining etc. for faster query processing. Initially the materialized views are formed based on some methodologies, however the performance (hit-miss ratio) of the materialized views may degrade after certain time if the incoming query pattern changes. This situation could be handled efficiently by employing a view maintenance scheme which works dynamically during query execution at run time. As these materialized views involves huge amount of data, consideration of time and space complexity during the maintenance process plays an important role. In this paper authors adopt an incremental view maintenance policy based on attribute affinity to update the materialized views at run time without using extra space and minimizing the data transfer between the secondary memory and primary memory (where the active materialized views reside). This in turn reduces time complexity and supports incremental maintenance eliminating the requirement of full replacement of existing materialized views.
Citations
More filters
DissertationDOI
20 Apr 2017
TL;DR: This thesis work defines a process to determine the minimal set of workload queries and the set of views to materialize and proposes a dynamic process that allows users to upgrade the CoDe model with a context-aware editor, and build an optimized lattice structure able to minimize the effort to recalculate it.
Abstract: Data warehouse systems aim to support decision making by providing users with the appropriate information at the right time. This task is particularly challenging in business contexts where large amount of data is produced at a high speed. To this end, data warehouses have been equipped with Online Analytical Processing tools that help users to make fast and precise decisions through the execution of complex queries. Since the computation of these queries is time consuming, data warehouses precompute a set of materialized views answering to the workload queries. This thesis work defines a process to determine the minimal set of workload queries and the set of views to materialize. The set of queries is represented by an optimized lattice structure used to select the views to be materialized according to the processing time costs and the view storage space. The minimal set of required Online Analytical Processing queries is computed by analyzing the data model defined with the visual language CoDe (Complexity Design). The latter allows to conceptually organize the visualization of data reports and to generate visualizations of data obtained from data-mart queries. CoDe adopts a hybrid modeling process combining two main methodologies: user-driven and data-driven. The first aims to create a model according to the user knowledge, requirements, and analysis needs, whilst the latter has in charge to concretize data and their relationships in the model through Online Analytical Processing queries. Since the materialized views change over time, we also propose a dynamic process that allows users to (i) upgrade the CoDe model with a context-aware editor, (ii) build an optimized lattice structure able to minimize the effort to recalculate it, and (iii) propose the new set of views to materialize. Moreover, the process applies a Markov strategy

2 citations


Cites background from "Dynamic incremental maintenance of ..."

  • ...In the literature several works has been outlined such as [60, 38, 17, 15] that concerns cache updating algorithms and [50,75, 59,47,19] that concerns incremental views maintenance algorithms....

    [...]

  • ...in [19] exploit the linear regression on attributes to find the co-relations between such attributes....

    [...]

Posted Content
TL;DR: An experimental platform is developed using real data-sets to evaluate the effectiveness in terms of performance and precision of the proposed techniques to remove the mismatch between the MV collection and reporting requirements.
Abstract: With the need for flexible and on-demand decision support, Dynamic Data Warehouses (DDW) provide benefits over traditional data warehouses due to their dynamic characteristics in structuring and access mechanism. A DDW is a data framework that accommodates data source changes easily to allow seamless querying to users. Materialized Views (MV) are proven to be an effective methodology to enhance the process of retrieving data from a DDW as results are pre-computed and stored in it. However, due to the static nature of materialized views, the level of dynamicity that can be provided at the MV access layer is restricted. As a result, the collection of materialized views is not compatible with ever-changing reporting requirements. It is important that the MV collection is consistent with current and upcoming queries. The solution to the above problem must consider the following aspects: (a) MV must be matched against an OLAP query in order to recognize whether the MV can answer the query, (b) enable scalability in the MV collection, an intuitive mechanism to prune it and retrieve closely matching MVs must be incorporated, (c) MV collection must be able to evolve in correspondence to the regularly changing user query patterns. Therefore, the primary objective of this paper is to explore these aspects and provide a well-rounded solution for the MV access layer to remove the mismatch between the MV collection and reporting requirements. Our contribution to solve the problem includes a Query Matching Technique, a Domain Matching Technique and Maintenance of the MV collection. We developed an experimental platform using real data-sets to evaluate the effectiveness in terms of performance and precision of the proposed techniques.

Cites methods from "Dynamic incremental maintenance of ..."

  • ...The works in [13, 21] propose techniques to keep the MVs up-to-date in relation to the changes in the data sources....

    [...]

Proceedings ArticleDOI
20 Jun 2021
TL;DR: In this article, a fuzzy based materialized data-cube driven warehouse architecture for fast decision making is proposed, which achieves faster data analysis and better hit-miss ratio in the materialised data-cubes.
Abstract: Data warehouse is used across the organizations for analytical processing. It is organized in the form of lattice of cuboids for carrying out the different types of analysis. The structure of lattice of cuboids grows exponentially with the number of dimensions. Every cuboid contains huge amount of data. The data size is multiplied in the data warehouse as a new entry takes place in any dimension. In modern day business analysis, processing the data in real-time is one of the intrinsic requirements. However, with the gigantic structure of lattice of cuboids along with the highly loaded data in every cuboid it is difficult to achieve the real-time processing. View materialization is practiced over the decades for faster query processing in the database applications. This research proposed a fuzzy based materialized data-cube driven warehouse architecture for fast decision making. Experimental output shows the proposed methodology achieves faster data analysis and better hit-miss ratio in the materialized data-cubes.
References
More filters
Posted Content
TL;DR: In this paper, a framework for materialized view selection that exploits a data mining technique (clustering), in order to determine clusters of similar queries is proposed. But it is based on cost models that evaluate the cost of accessing data using views and the costs of storing these views.
Abstract: Materialized view selection is a non-trivial task. Hence, its complexity must be reduced. A judicious choice of views must be cost-driven and influenced by the workload experienced by the system. In this paper, we propose a framework for materialized view selection that exploits a data mining technique (clustering), in order to determine clusters of similar queries. We also propose a view merging algorithm that builds a set of candidate views, as well as a greedy process for selecting a set of views to materialize. This selection is based on cost models that evaluate the cost of accessing data using views and the cost of storing these views. To validate our strategy, we executed a workload of decision-support queries on a test data warehouse, with and without using our strategy. Our experimental results demonstrate its efficiency, even when storage space is limited.

116 citations

Proceedings ArticleDOI
23 Feb 1998
TL;DR: This work defines simple views and materialized views for such graph structured data, analyzing options for representing record identity and references in the view and develops incremental maintenance algorithms for these views.
Abstract: Studies the problem of maintaining materialized views of graph structured data. The base data consists of records containing identifiers of other records. The data could represent traditional objects (with methods, attributes and a class hierarchy), but it could also represent a lower-level data structure. We define simple views and materialized views for such graph structured data, analyzing options for representing record identity and references in the view. We develop incremental maintenance algorithms for these views.

116 citations


Additional excerpts

  • ...B. Algorithm Materialized_View_Maintenance...

    [...]

Proceedings ArticleDOI
27 May 1997
TL;DR: This work addresses some issues related to determining this set of shared views to be materialized in order to achieve the best combination of good performance and low maintenance, and provides an algorithm for achieving this goal.
Abstract: Data warehouses are accessed by different queries with different frequencies. The portions of data accessed by a query can be treated as a view. When these views are related to each other and defined over overlapping portions of the base data, then it may be more efficient not to materialize all the views, but rather to materialize certain "shared views" from which the query results can be generated. We address some issues related to determining this set of shared views to be materialized in order to achieve the best combination of good performance and low maintenance, and provide an algorithm for achieving this goal.

74 citations


"Dynamic incremental maintenance of ..." refers methods in this paper

  • ...The Sizeof operator is used to compute the total amount of data held by the views or the attributes....

    [...]

Proceedings ArticleDOI
02 Aug 1999
TL;DR: The approach is to regard the complex changes done to a view definition after synchronization as an atomic unit; another is to exploit knowledge of how the view definition was synchronized, especially the containment information between the old and new views.
Abstract: While current view technology assumes that information systems (ISs) do not change their schemas, our Evolvable View Environment (EVE) project addresses this problem by evolving the view definitions affected by IS schema changes, which we call view synchronization. In EVE, the view synchronizer rewrites the view definitions by replacing view components with suitable components from other ISs. However, after such a view redefinition process, the view extents, if materialized, must also be brought up to date. In this paper, we propose strategies to address this incremental adaptation of the view extent after view synchronization. One key idea of our approach is to regard the complex changes done to a view definition after synchronization as an atomic unit; another is to exploit knowledge of how the view definition was synchronized, especially the containment information between the old and new views. Our techniques would successfully adapt views under the unavailability of base relations, while currently known maintenance strategies from the literature would fail.

37 citations


Additional excerpts

  • ...B. Algorithm Materialized_View_Maintenance...

    [...]

Book ChapterDOI
03 Sep 2006
TL;DR: A framework for materialized view selection is proposed that exploits a data mining technique (clustering) in order to determine clusters of similar queries and a view merging algorithm that builds a set of candidate views, as well as a greedy process for selecting aSet of views to materialize.
Abstract: Materialized view selection is a non-trivial task. Hence, its complexity must be reduced. A judicious choice of views must be cost-driven and influenced by the workload experienced by the system. In this paper, we propose a framework for materialized view selection that exploits a data mining technique (clustering), in order to determine clusters of similar queries. We also propose a view merging algorithm that builds a set of candidate views, as well as a greedy process for selecting a set of views to materialize. This selection is based on cost models that evaluate the cost of accessing data using views and the cost of storing these views. To validate our strategy, we executed a workload of decision-support queries on a test data warehouse, with and without using our strategy. Our experimental results demonstrate its efficiency, even when storage space is limited.

23 citations


"Dynamic incremental maintenance of ..." refers methods in this paper

  • ...The Sizeof operator is used to compute the total amount of data held by the views or the attributes....

    [...]