htmlabstractWith the performance of modern computers improving at a rapid pace, database 
technology has problems with fully exploiting the benefits that each new hardware 
generation brings. This has caused a significant performance gap between 
general-purpose databases and specialized, application-optimized solutions for 
large-volume computation-intensive processing problems, as found in areas including 
information retrieval, scientific data management and decision support. 
This thesis attempts to enhance the state-of-the-art in architecture-conscious 
database research, both in the query execution layer as well as in the data storage 
layer, and in the way these work together. Thus, rather than focusing on 
an isolated problem or algorithm, the thesis presents a new database system 
architecture, realized in the MonetDB/X100 prototype, that combines a coherent 
set of new architecture-conscious techniques that are designed to work well 
together. 
The motivation for the new query execution layer comes from the analysis 
of the problems of two popular approaches to query processing: tuple-at-a-time 
operator pipelining, used in most existing systems, and column-at-a-time materializing 
operators, found in MonetDB. MonetDB/X100 proposes a new vectorized 
in-cache execution model, that exploits ideas from both approaches, and combines 
the scalability of the former with the high-performance bulk processing of 
the latter. This is achieved by modifying the traditional operator pipeline model 
to operate on cache-resident vectors of data using highly optimized primitive 
functions. Additionally, within this architecture, a set of hardware-conscious 
design and programming techniques is presented, enabling efficient execution 
of typical data processing tasks. The resulting query execution layer efficiently 
exploits modern super-scalar CPUs and cache-memory systems and achieves 
in-memory performance often one or two orders of magnitude higher than the 
existing approaches. 
In the storage area there are two hardware trends that significantly influence 
219 
220 Summary 
database performance. First, the imbalance between sequential disk bandwidth 
and random disk latency continuously increases. As a result, access methods that 
rely on random I/O become less attractive, making various forms of sequential 
access the preferred option. MonetDB/X100 follows this idea with ColumnBM 
– a bandwidth-optimized column store. Secondly, both disk bandwidth and latency 
improve significantly slower than the computing power of modern CPUs, 
especially with the advent of multi-core CPUs. ColumnBM introduces two techniques 
that address this issue. Lightweight in-cache compression allows trading 
some processor time for an increased perceived disk bandwidth. High decompression 
performance is achieved by applying the decompression on the RAMcache 
boundary, providing cache-resident data directly to the execution layer. 
Additionally, introduced family of compression methods provides performance 
an order of magnitude higher than previous solutions. Cooperative scans observe 
current system activity and dynamically schedule I/O operations to exploit overlapping 
demands of different queries. This allows to amortize the cost of disk 
access among multiple consumers, and also better utilize the available buffer 
space, providing much better performance with many concurrently executing 
queries. 
By combining the CPU-efficient processing with a bandwidth-optimized storage 
facility, MonetDB/X100 has been able to achieve its high in-memory raw 
query execution power also on huge disk-resident datasets. We evaluated its 
performance both on TPC-H decision support data sets as well as in the area 
of large-volume information retrieval (the Terabyte TREC task), where it successfully 
competed with the specialized solutions, both for in-memory and diskbased 
tasks.

/pdf/balancing-vectorized-query-execution-with-bandwidth-3bf5a3pf2b.pdf

Balancing vectorized query execution with bandwidth-optimized storage

The amount of digital data we produce every day far surpasses our ability to process this data, and finding useful information in this constant flow of data has become one of the major challenges of the 21st century. Search engines are one way of accessing large data collections. Their algorithms have evolved far beyond simply matching search queries to sets of documents. Today’s most sophisticated search engines combine hundreds of relevance signals to provide the best possible results for each searcher. Current approaches for tuning the parameters of search engines can be highly effective. However, they typically require considerable expertise and manual effort. They rely on supervised learning to rank, meaning that they learn from manually annotated examples of relevant documents for given queries. Obtaining large quantities of sufficiently accurate manual annotations is becoming increasingly difficult, especially for personalized search, access to sensitive data, or search in settings that change over time. In this thesis, I develop new online learning to rank techniques, based on insights from reinforcement learning. In contrast to supervised approaches, these methods allow search engines to learn directly from users’ interactions. User interactions can typically be observed easily and cheaply, and reflect the preferences of real users. Interpreting user interactions and learning from them is challenging, because they can be biased and noisy. The contributions of this thesis include a novel interleaved comparison method, called probabilistic interleave, that allows unbiased comparisons of search engine result rankings, and methods for learning quickly and effectively from the resulting relative feedback. The obtained analytical and experimental results show how search engines can effectively learn from user interactions. In the future, these and similar techniques can open up new ways for gaining useful information from ever larger amounts of data.

/pdf/fast-and-reliable-online-learning-to-rank-for-information-40rh9l5ybt.pdf

Fast and reliable online learning to rank for information retrieval

Users may download and print one copy of any publication from the public portal for the purpose of private study or research You may not further distribute the material or use it for any profit-making activity or commercial gain You may freely distribute the URL identifying the publication in the public portal Take down policy If you believe that this document breaches copyright, please contact us providing details, and we will remove access to the work immediately and investigate your claim.

Multinomial Language Learning: Investigations into the Geometry of Language

We study the problem of finding frequent itemsets in a continuous stream of transactions. The current frequency of an itemset in a stream is defined as its maximal frequency over all possible windows in the stream from any point in the past until the current state that satisfy a minimal length constraint. Properties of this new measure are studied and an incremental algorithm that allows, at any time, to immediately produce the current frequencies of all frequent itemsets is proposed. Experimental and theoretical analysis show that the space requirements for the algorithm are extremely small for many realistic data distributions.

Mining frequent itemsets in a stream

A method for designing an Enterprise information system, which is based on an implementation independent model of the organization, has been developed. This model, also called an ontological model, consists of both implementation independent business processes and implementation independent information processes. An Enterprise information system (EIS) has to be understood as an implementation scenario of the ontological model. This study contains a specification framework that provides rules for taking into account during engineering the EIS.

/pdf/a-method-for-enterprise-ontology-based-design-of-enterprise-3mlwsv52tx.pdf

A Method for Enterprise Ontology based Design of Enterprise Information Systems

/pdf/modelling-web-usage-in-a-changing-environment-4bqh858y3g.pdf

P.I. Hofgesang

Papers

Modelling Web Usage in a Changing Environment