htmlabstractWith the performance of modern computers improving at a rapid pace, database 
technology has problems with fully exploiting the benefits that each new hardware 
generation brings. This has caused a significant performance gap between 
general-purpose databases and specialized, application-optimized solutions for 
large-volume computation-intensive processing problems, as found in areas including 
information retrieval, scientific data management and decision support. 
This thesis attempts to enhance the state-of-the-art in architecture-conscious 
database research, both in the query execution layer as well as in the data storage 
layer, and in the way these work together. Thus, rather than focusing on 
an isolated problem or algorithm, the thesis presents a new database system 
architecture, realized in the MonetDB/X100 prototype, that combines a coherent 
set of new architecture-conscious techniques that are designed to work well 
together. 
The motivation for the new query execution layer comes from the analysis 
of the problems of two popular approaches to query processing: tuple-at-a-time 
operator pipelining, used in most existing systems, and column-at-a-time materializing 
operators, found in MonetDB. MonetDB/X100 proposes a new vectorized 
in-cache execution model, that exploits ideas from both approaches, and combines 
the scalability of the former with the high-performance bulk processing of 
the latter. This is achieved by modifying the traditional operator pipeline model 
to operate on cache-resident vectors of data using highly optimized primitive 
functions. Additionally, within this architecture, a set of hardware-conscious 
design and programming techniques is presented, enabling efficient execution 
of typical data processing tasks. The resulting query execution layer efficiently 
exploits modern super-scalar CPUs and cache-memory systems and achieves 
in-memory performance often one or two orders of magnitude higher than the 
existing approaches. 
In the storage area there are two hardware trends that significantly influence 
219 
220 Summary 
database performance. First, the imbalance between sequential disk bandwidth 
and random disk latency continuously increases. As a result, access methods that 
rely on random I/O become less attractive, making various forms of sequential 
access the preferred option. MonetDB/X100 follows this idea with ColumnBM 
– a bandwidth-optimized column store. Secondly, both disk bandwidth and latency 
improve significantly slower than the computing power of modern CPUs, 
especially with the advent of multi-core CPUs. ColumnBM introduces two techniques 
that address this issue. Lightweight in-cache compression allows trading 
some processor time for an increased perceived disk bandwidth. High decompression 
performance is achieved by applying the decompression on the RAMcache 
boundary, providing cache-resident data directly to the execution layer. 
Additionally, introduced family of compression methods provides performance 
an order of magnitude higher than previous solutions. Cooperative scans observe 
current system activity and dynamically schedule I/O operations to exploit overlapping 
demands of different queries. This allows to amortize the cost of disk 
access among multiple consumers, and also better utilize the available buffer 
space, providing much better performance with many concurrently executing 
queries. 
By combining the CPU-efficient processing with a bandwidth-optimized storage 
facility, MonetDB/X100 has been able to achieve its high in-memory raw 
query execution power also on huge disk-resident datasets. We evaluated its 
performance both on TPC-H decision support data sets as well as in the area 
of large-volume information retrieval (the Terabyte TREC task), where it successfully 
competed with the specialized solutions, both for in-memory and diskbased 
tasks.

/pdf/balancing-vectorized-query-execution-with-bandwidth-3bf5a3pf2b.pdf

Balancing vectorized query execution with bandwidth-optimized storage

Disclosed is a system, method, and program for accessing an application program, such as a transaction manager. A request for a file is received. The request includes a pathname and the pathname includes at least one directory name and the file name. The pathname is processed by a file system to determine whether the request is for a file on a storage device or an operation for the application program to perform. The file request is processed to generate a control message for the application program after determining that the request is not for a file on the storage device. The control message is then passed to the application program to cause the application program to perform an operation.

Method, system for using file name to access application program where a logical file system processes pathname to determine whether the request is a file on storage device or operation for application program

The data base concept derives from early military on-line systems, and was not originally associated with the specific technologies of modern data base management systems. While the idea of an integrated data base, or "bucket of facts," spread into corporate data processing and management circles during the early 1960s, it was seldom realized in practice. File-processing packages were among the very first distributed as supported products, but only in the late 1960s were they first called "data base management systems," in large part through the actions of the Data Base Task Group of the Committee on Data Systems Languages (CODASYL). As the DBMS concept spread, the data base itself was effectively redefined as the informational content of a packaged DBMS. Throughout the process, managerial descriptions of the data base as a flexible and integrated repository for all corporate data stood in sharp contrast with the useful but limited nature of actual systems.

/pdf/a-veritable-bucket-of-facts-origins-of-the-data-base-51m8jr4liz.pdf

"A veritable bucket of facts" origins of the data base management system

Generalized report generation and file maintenance programs were widely used in the 1950s, standardized by the Share user group with 9PAC and Surge. By the 1960s the first recognizable DBMS systems, such IMS and IDS, had evolved to address the challenges of disk drives and MIS projects. Finally, in the late 1960s Codasyl's Data Base Task Group formulated the DBMS concept itself.

https://www.tomandmaria.com/Tom/Writing/HowThe%20DataGotItsBase.pdf

How Data Got its Base: Information Storage Software in the 1950s and 1960s

A method, apparatus, and article of manufacture for accessing a hierarchical database using an object-oriented programming model. The database is modeled into an objects framework, wherein the objects framework corresponds to application views, data structures, and data defined and stored in the database. The database is then accessed through the objects framework. A class definition tool generates class definitions for objects that encapsulate or wrapper the data retrieved from the database.

Object-oriented programming model for accessing hierarchical databases

This technical note recognizes the thirtieth anniversary of the IBM Information Management System (IMSTM) and describes the initial requirements that led to the development of IMS. It also describes how IMS has evolved over the past thirty years to implement new technology while preserving customers' application programming investments.

K. R. Blackman

Papers

Technical note: IMS celebrates thirty years as an IBM product