scispace - formally typeset
Patent

OLAP Query Processing Method Oriented to Database and HADOOP Hybrid Platform

Yan-Song Zhang, +1 more
Reads0
Chats0
TLDR
In this paper, an OLAP query processing method oriented to a database and Hadoop hybrid platform is described, where the database technology and the hadoop technology are combined, and the storage performance of the database and the high expandability and high availability of the Hadoops are combined; the database query processing and the MapReduce query processing are integrated in a loosely-coupled mode.
Abstract: 
An OLAP query processing method oriented to a database and Hadoop hybrid platform is described. When OLAP query processing is performed, the processing is executed first on a main working copy, and a query processing result is recorded in an aggregate result table of a local database; when a working node is faulty, node information of a fault-tolerant copy corresponding to the main working copy is searched for through namenode, and a MapReduce task is invoked to complete the OLAP query processing task on the fault-tolerant copy. The database technology and the Hadoop technology are combined, and the storage performance of the database and the high expandability and high availability of the Hadoop are combined; the database query processing and the MapReduce query processing are integrated in a loosely-coupled mode, thereby ensuring the high query processing performance, and ensuring the high fault-tolerance performance.

read more

Citations
More filters
Patent

Data processing method and system

Liu Kunpeng, +2 more
TL;DR: In this paper, a data processing method and system are used for solving the problem that the outer-layer application access efficiency is low, and a communication module receives an access request packet sent by a service processing module; the communication module calls an Mapreduce plug-in to conduct splitting on the access request, and at least one access request and routing information corresponding to each access request are obtained.
Patent

Processing datasets with a dbms engine

Ronnie Chaiken, +2 more
TL;DR: In this article, the authors present a system and method to process a dataset with a database management system (DBMS) engine, which includes splitting bulk data into a plurality of chunks.
Patent

Manifest-based snapshots in distributed computing environments

Jonathan Ming-Cyn Hsieh, +1 more
TL;DR: In this article, the manifest-based snapshots in distributed computing environments have been proposed, where a master node identifies multiple slave nodes on which a data object is stored in the cloud-computing platform and creates a snapshot manifest representing the snapshot of the data object.
Patent

Object caching method based on disk

Cheng Xingrong, +4 more
TL;DR: In this article, an object caching method based on a disk is described. But the method is not suitable for the use of data of any size and any type, and the file storage structure of the secondary indexes is not considered.
Patent

Predicate pushdown with late materialization in database query processing

Ronald J. Barber, +3 more
TL;DR: The authors provides query processing for column stores by accumulating table record attributes during application of query plan operators on a table, when said attributes and associated attribute values are to be consumed for an operation in the query plan.
References
More filters
Patent

Scalable analysis platform for semi-structured data

Dimitris Tsirogiannis, +6 more
TL;DR: In this article, the authors propose a method of operating a query system that includes retrieving objects from a data source, wherein each of the retrieved objects includes data and metadata describing the data.
Patent

Computer systems and methods for the query and visualization of multidimensional databases

Chris Stolte, +2 more
TL;DR: In this article, a visual table, comprising a plurality of panes, is constructed by providing a specification that is in a language based on the hierarchical structure of the database, in some cases, this language can include fields that are in the database schema.
Patent

Method for visualizing information in a data warehousing environment

Li-Wen Chen
TL;DR: In this paper, a method for graphically analyzing relationships in data (103) from one or more data sources of an enterprise is presented. But the method is especially useful in conjunction with a meta-model based technique for modeling the enterprise data.
Patent

Systems and methods for processing data

Daniel J. Abadi, +3 more
TL;DR: In this paper, a system, method, and computer program product for processing data are disclosed, which includes a data processing framework, a plurality of database systems coupled to the data-processing framework, and a storage component in communication with the dataprocessing framework and the plurality database systems, configured to store information about each partition of the data processing task being processed.
Patent

High-concurrency query operator and method

George Candea, +1 more
TL;DR: In this paper, the authors present a method for concurrently executing a set of multiple queries through a processor to improve a resource usage within a data warehouse system. But the method does not consider the group of users of the system to simultaneously run queries.