Proceedings ArticleDOI
How to efficiently snapshot transactional data: hardware or software controlled?
Henrik Mühe,Alfons Kemper,Thomas Neumann +2 more
- pp 17-26
Reads0
Chats0
TLDR
An in-memory database system that separates transaction processing from OLAP query processing via periodically refreshed snapshots is designed, so that OLAP queries can be executed without any synchronization and OLTP transaction processing follows the lock-free, mostly serial processing paradigm of H-Store.Abstract:
The quest for real-time business intelligence requires executing mixed transaction and query processing workloads on the same current database state. However, as Harizopoulos et al. [6] showed for transactional processing, co-execution using classical concurrency control techniques will not yield the necessary performance -- even in re-emerging main memory database systems. Therefore, we designed an in-memory database system that separates transaction processing from OLAP query processing via periodically refreshed snapshots. Thus, OLAP queries can be executed without any synchronization and OLTP transaction processing follows the lock-free, mostly serial processing paradigm of H-Store [8]. In this paper, we analyze different snapshot mechanisms: Hardware-supported Page Shadowing, which lazily copies memory pages when changed by transactions, software controlled Tuple Shadowing, which generates a new version when a tuple is modified, software controlled Twin Tuple, which constantly maintains two versions of each tuple and HotCold Shadowing, which effectively combines Tuple Shadowing and hardware-supported Page Shadowing by clustering update-intensive objects. We evaluate their performance based on the mixed workload CH-BenCHmark which combines the TPC-C and the TPC-H benchmarks on the same database schema and state.read more
Citations
More filters
Journal ArticleDOI
In-Memory Big Data Management and Processing: A Survey
TL;DR: This survey aims to provide a thorough review of a wide range of in-memory data management and processing proposals and systems, including both data storage systems and data processing frameworks.
Posted Content
Compacting Transactional Data in Hybrid OLTP & OLAP Databases
TL;DR: This approach reorganizes and compresses transactional data online and yet hardly affects the mission-critical OLTP throughput by unburdening the OLTP threads from all additional processing and performing these tasks asynchronously.
Journal ArticleDOI
Compacting transactional data in hybrid OLTP&OLAP databases
TL;DR: In this article, the compaction of memory-resident databases is proposed, which separates the mutable working set from the immutable "frozen" data and compresses the immutable data and optimizes it for efficient, memoryconsumption-friendly snapshotting.
Journal Article
Processing in the Hybrid OLTP & OLAP Main-Memory Database System HyPer.
Alfons Kemper,Thomas Neumann,Jan Finis,Florian Funke,Viktor Leis,Henrik Mühe,Tobias Mühlbauer,Wolf Rödiger +7 more
TL;DR: The HyPerScript transaction programming language, the mainmemory indexing technique ART, which is decisive for high transaction processing performance, and HyPer’s transaction management that allows heterogeneous workloads consisting of short pre-canned transactions, OLAP-style queries, and long interactive transactions are surveyed.
Journal ArticleDOI
RUMA has it: rewired user-space memory access is possible!
TL;DR: RUMA: Rewired User-space Memory Access allows for physiological data management and allows developers to freely rewire the mappings from virtual to physical memory (in user space) while at the same time exploiting the virtual memory support offered by hardware and operating system.
References
More filters
Proceedings ArticleDOI
HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots
Alfons Kemper,Thomas Neumann +1 more
TL;DR: This work presents an efficient hybrid system, called HyPer, that can handle both OLTP and OLAP simultaneously by using hardware-assisted replication mechanisms to maintain consistent snapshots of the transactional data.
Journal ArticleDOI
Schism: a workload-driven approach to database replication and partitioning
TL;DR: Schism consistently outperforms simple partitioning schemes, and in some cases proves superior to the best known manual partitioning, reducing the cost of distributed transactions up to 30%.
Journal ArticleDOI
H-store: a high-performance, distributed main memory transaction processing system
Robert Kallman,Hideaki Kimura,Jonathan Natkins,Andrew Pavlo,Alexander Rasin,Stanley B. Zdonik,Evan P. C. Jones,Samuel Madden,Michael Stonebraker,Yang Zhang,John Hugg,Daniel J. Abadi +11 more
TL;DR: The demonstration presented here provides insight on the development of a distributed main memory OLTP database and allows for the further study of the challenges inherent in this operating environment.
Journal ArticleDOI
Main memory database systems: an overview
TL;DR: The authors survey the major memory residence optimizations and briefly discuss some of the MMDBs that have been designed or implemented.
Journal ArticleDOI
Efficiently compiling efficient query plans for modern hardware
TL;DR: This work presents a novel compilation strategy that translates a query into compact and efficient machine code using the LLVM compiler framework and integrates these techniques into the HyPer main memory database system and shows that this results in excellent query performance while requiring only modest compilation time.