Efficient Traversal in Data Warehouse Based on Concept Hierarchy Using Galois Connections
19 Feb 2011-pp 335-339
TL;DR: A new algorithm has been proposed using a dynamic data structure that reduces over time resulting in better space utilization and also reduction of computation time and offers formalism in analysis using concept hierarchy in an abstract interpretation framework.
Abstract: This paper propose a new methodology for efficient implementation of OLAP operations using concept hierarchies of attributes in a data warehouse. The different granularity associated with a particular dimension and the hierarchy amongst those may be represented as a lattice. The focus is to move up (roll-up) and down (drill-down) within the lattice structure using an algorithm with optimal time complexity. In this paper, a new algorithm has been proposed using a dynamic data structure that reduces over time resulting in better space utilization and also reduction of computation time. A Galois Connection is identified on this lattice structure with well-defined abstraction and concretization functions based on the concept hierarchy. The contribution offers formalism in analysis using concept hierarchy in an abstract interpretation framework.
Citations
More filters
TL;DR: This research work dynamically finds the most cost effective path from the lattice structure of cuboids based on concept hierarchy to minimize the query access time.
13 citations
01 Jan 2021
TL;DR: This paper proposes an algorithm for cuboid materialization starting from a source cuboid to the target cuboid in an optimal way such that the intermediate cuboids consume less space and require lower time to generate by making sure those cuboids have the least number of rows compared to other valid cuboids available for selection.
Abstract: In the field of business intelligence, we require the analysis of multidimensional data with the need for it being fast and interactive. Data warehousing and OLAP approaches have been developed for this purpose in which the data is viewed in the form of a multidimensional data cube which allows interactive analysis of the data in various levels of abstraction presented in a graphical manner. In data cube, there may arise a need to materialize a particular cuboid given that some other cuboid is presently materialized, in this paper, we propose an algorithm for cuboid materialization starting from a source cuboid to the target cuboid in an optimal way such that the intermediate cuboids consume less space and require lower time to generate by making sure those cuboids have the least number of rows compared to other valid cuboids available for selection, by sorting them based on the product of cardinalities of dimensions present in each cuboid.
5 citations
TL;DR: This paper identifies a warehouse model to build an analytical framework and analyze different important parameters which directly impact the changes of share market and identifies different applications of this analytical model for forecasting information to help decision making.
Abstract: The objective of this paper is identifying a warehouse model to build an analytical framework and analyze different important parameters which directly impact the changes of share market. We identify parameters that represent different viewing windows and perspectives towards stock market performance and movement trends. We categorize and define many intrinsic as well as external factors that may affect stock market as a whole. Sensex and Nifty are used as the pulse of Indian stock market. In this paper, we focus on defining a suitable OLAP model which can cater all the parameters that affect share market. We also identify different applications of this analytical model for forecasting information to help decision making.
4 citations
Cites background from "Efficient Traversal in Data Warehou..."
...The following is a figure of 3-D data cube, consist of three dimensions A, B and C where each cuboid represents different degree of summarization [5],[6]....
[...]
Posted Content•
TL;DR: In this paper, the authors propose a framework illustrating the barriers and suggested solutions in the way of achieving real-time OLAP answers that are significantly used in decision support systems and data warehouses.
Abstract: The overwhelmingly increasing amount of stored data has spurred researchers seeking different methods in order to optimally take advantage of it which mostly have faced a response time problem as a result of this enormous size of data. Most of solutions have suggested materialization as a favourite solution. However, such a solution cannot attain Real- Time answers anyhow. In this paper we propose a framework illustrating the barriers and suggested solutions in the way of achieving Real-Time OLAP answers that are significantly used in decision support systems and data warehouses.
3 citations
TL;DR: An algorithm for formal analysis leading towards detection of an optimal path for any two given valid pair of cuboids at different levels is proposed based on branch and bound method for selection of optimal path.
Abstract: analysis requires the computation of many aggregate functions over a large volume of collected data. To provide the various viewpoints for the analysts, these data are organized as a multi-dimensional data model called data cubes. Each cell in a data cube represents a unique set of values for the different dimensions and contains the metrics of interest. The different abstraction and concretization associated with a dimension may be represented as lattice. The focus is to move up and drill down within the lattice using an algorithm with optimal space and computation. In the lattice of cuboids, there exist multiple paths for summarization from a lower to an upper level of cuboid. The alternate paths involve different amounts of storage space and different volume of computations. Thus objective of this paper is to design an algorithm for formal analysis leading towards detection of an optimal path for any two given valid pair of cuboids at different levels. Algorithm is proposed based on branch and bound method for selection of optimal path. Experimental results in the last show that the solution obtained by the algorithm gives the optimal solution in terms of space and time computation.
2 citations
Cites background or methods from "Efficient Traversal in Data Warehou..."
...Sen [21][22] is based on two operations roll-up and drill-down for finding the optimized path to traverse between two data cube of valid dimension in term of intermediate cuboid sizes....
[...]
...Various researchers proposed number of algorithms for the selection and computation of data cubes [3][4][12][13][21][22]....
[...]
References
More filters
Book•
08 Sep 2000TL;DR: This book presents dozens of algorithms and implementation examples, all in pseudo-code and suitable for use in real-world, large-scale data mining projects, and provides a comprehensive, practical look at the concepts and techniques you need to get the most out of real business data.
Abstract: The increasing volume of data in modern business and science calls for more complex and sophisticated tools. Although advances in data mining technology have made extensive data collection much easier, it's still always evolving and there is a constant need for new techniques and tools that can help us transform this data into useful information and knowledge. Since the previous edition's publication, great advances have been made in the field of data mining. Not only does the third of edition of Data Mining: Concepts and Techniques continue the tradition of equipping you with an understanding and application of the theory and practice of discovering patterns hidden in large data sets, it also focuses on new, important topics in the field: data warehouses and data cube technology, mining stream, mining social networks, and mining spatial, multimedia and other complex data. Each chapter is a stand-alone guide to a critical topic, presenting proven algorithms and sound implementations ready to be used directly or with strategic modification against live data. This is the resource you need if you want to apply today's most powerful data mining techniques to meet real business challenges. * Presents dozens of algorithms and implementation examples, all in pseudo-code and suitable for use in real-world, large-scale data mining projects. * Addresses advanced topics such as mining object-relational databases, spatial databases, multimedia databases, time-series databases, text databases, the World Wide Web, and applications in several fields. *Provides a comprehensive, practical look at the concepts and techniques you need to get the most out of real business data
23,600 citations
"Efficient Traversal in Data Warehou..." refers background in this paper
...Whenever a roll-up or drill-down is performed [ 4 ] [5], the size of individual record is not changed but the numbers of tuples that are returned are changed....
[...]
...In data warehousing, concept hierarchies are necessary for the most two common operations of OLAP namely drill-down and roll-up [ 4 ]....
[...]
...A concept hierarchy [2] [ 4 ] identifies a sequence of mappings on attributes from low-level general concept to high-level aggregated concepts....
[...]
01 Jan 1977
TL;DR: In this paper, the abstract interpretation of programs is used to describe computations in another universe of abstract objects, so that the results of abstract execution give some information on the actual computations.
Abstract: A program denotes computations in some universe of objects. Abstract interpretation of programs consists in using that denotation to describe computations in another universe of abstract objects, so that the results of abstract execution give some information on the actual computations. An intuitive example (which we borrow from Sintzoff [72]) is the rule of signs. The text -1515 * 17 may be understood to denote computations on the abstract universe {(+), (-), (±)} where the semantics of arithmetic operators is defined by the rule of signs. The abstract execution -1515 * 17 → -(+) * (+) → (-) * (+) → (-), proves that -1515 * 17 is a negative number. Abstract interpretation is concerned by a particular underlying structure of the usual universe of computations (the sign, in our example). It gives a summary of some facets of the actual executions of a program. In general this summary is simple to obtain but inaccurate (e.g. -1515 + 17 → -(+) + (+) → (-) + (+) → (±)). Despite its fundamentally incomplete results abstract interpretation allows the programmer or the compiler to answer questions which do not need full knowledge of program executions or which tolerate an imprecise answer, (e.g. partial correctness proofs of programs ignoring the termination problems, type checking, program optimizations which are not carried in the absence of certainty about their feasibility, …).
6,829 citations
"Efficient Traversal in Data Warehou..." refers background in this paper
...The concept of abstract interpretation [6] [7] [8] is incorporated here to perform static analysis [7]....
[...]
02 Dec 1995
TL;DR: Methods for mining knowledge at multiple concept levels can often be developed by extension of existing data mining techniques, and it is often necessary to adopt techniques such as step-by-step generalization/specialization or progressive deepening of a knowledge mining process.
Abstract: Most studies on data mining have been focused at mining rules at single concept levels, i.e,, either at the primitive level or at a rather high concept level. However, it is often desirable to discover knowledge at multiple concept levels. Mining knowledge at multiple levels may help database users find some interesting rules which are difficult to be discovered otherwise and view database contents at different abstraction levels and from different angles. Methods for mining knowledge at multiple concept levels can often be developed by extension of existing data mining techniques. Moreover, for eficient processing and interactive mining of multiple-level rules, it is often necessary to adopt techniques such as step-by-step generalization/specialization or progressive deepening of a knowledge mining process. Other issues, such as visual representation of knowledge at multiple levels, and “redundant” rule filtering, should also be studied in depth.
82 citations
"Efficient Traversal in Data Warehou..." refers background in this paper
...Data mining allows Knowledge Discovery in Database (KDD) at different conceptual levels [3]....
[...]
10 Nov 2008
TL;DR: It is proved that, for Galois insertions, widening is preserved by abstraction, and it is shown how widening operators can be combined for the cartesian and reduced product of abstract domains.
Abstract: Interpretation, one of the most applied techniques for semantics based static analysis of software, is based on two main key-concepts: the correspondence between concrete and abstract semantics through Galois connections/insertions, and the feasibility of a fixed point computation of the abstract semantics, through the fast convergence of widening operators. The latter point is crucial to ensure the scalability of the analysis to large software systems. In this paper, we investigate which properties are necessary to support a systematic design of widening operators, by discussing and comparing different definitions in the literature, and by proposing various ways to combine them. In particular, we prove that, for Galois insertions, widening is preserved by abstraction, and we show how widening operators can be combined for the cartesian and reduced product of abstract domains.
31 citations
"Efficient Traversal in Data Warehou..." refers background in this paper
...The concept of abstract interpretation [6] [7] [8] is incorporated here to perform static analysis [7]....
[...]
10 Oct 2005
TL;DR: Abstract Interpretation is a theory of approximation of mathematical structures, in particular those involved in the semantic models of computer systems, that can be applied to the systematic construction of methods and effective algorithms to approximate undecidable or very complex problems in computer science.
Abstract: Interpretation is a theory of approximation of mathematical structures, in particular those involved in the semantic models of computer systems [4,10,11]. interpretation can be applied to the systematic construction of methods and effective algorithms to approximate undecidable or very complex problems in computer science. The scope of application is rather large e.g. from type inference [5], model-checking [13], program transformation [14], watermarking [15] to context-free grammar parser generation [16].
25 citations
"Efficient Traversal in Data Warehou..." refers background in this paper
...The concept of abstract interpretation [6] [7] [8] is incorporated here to perform static analysis [7]....
[...]