Optimal aggregation algorithms for middleware

Distributed data processing is becoming a reality. Businesses want to do it for many reasons, and they often must do it in order to stay competitive. While much of the infrastructure for distributed data processing is already there (e.g., modern network technology), a number of issues make distributed data processing still a complex undertaking: (1) distributed systems can become very large, involving thousands of heterogeneous sites including PCs and mainframe server machines; (2) the state of a distributed system changes rapidly because the load of sites varies over time and new sites are added to the system; (3) legacy systems need to be integrated—such legacy systems usually have not been designed for distributed data processing and now need to interact with other (modern) systems in a distributed environment. This paper presents the state of the art of query processing for distributed database and information systems. The paper presents the “textbook” architecture for distributed query processing and a series of techniques that are particularly useful for distributed database systems. These techniques include special join techniques, techniques to exploit intraquery paralleli sm, techniques to reduce communication costs, and techniques to exploit caching and replication of data. Furthermore, the paper discusses different kinds of distributed systems such as client-server, middleware (multitier), and heterogeneous database systems, and shows how query processing works in these systems.

/pdf/the-state-of-the-art-in-distributed-query-processing-2b23xet25f.pdf

The state of the art in distributed query processing

Assume that each object in a database has m grades, or scores, one for each of m attributes. For example, an object can have a color grade, that tells how red it is, and a shape grade, that tells how round it is. For each attribute, there is a sorted list, which lists each object and its grade under that attribute, sorted by grade (highest grade first). There is some monotone aggregation function, or combining rule, such as min or average, that combines the individual grades to obtain an overall grade.To determine objects that have the best overall grades, the naive algorithm must access every object in the database, to find its grade under each attribute. Fagin has given an algorithm (“Fagin's Algorithm”, or FA) that is much more efficient. For some distributions on grades, and for some monotone aggregation functions, FA is optimal in a high-probability sense.We analyze an elegant and remarkably simple algorithm (“the threshold algorithm”, or TA) that is optimal in a much stronger sense than FA. We show that TA is essentially optimal, not just for some monotone aggregation functions, but for all of them, and not just in a high-probability sense, but over every database. Unlike FA, which requires large buffers (whose size may grow unboundedly as the database size grows), TA requires only a small, constant-size buffer.We distinguish two types of access: sorted access (where the middleware system obtains the grade of an object in some sorted list by proceeding through the list sequentially from the top), and random access (where the middleware system requests the grade of object in a list, and obtains it in one step). We consider the scenarios where random access is either impossible, or expensive relative to sorted access, and provide algorithms that are essentially optimal for these cases as well.

/pdf/optimal-aggregation-algorithms-for-middleware-51azba38t7.pdf

We study an on-line broadcast scheduling problem in which requests have deadlines, and the objective is to maximize the weighted throughput, i.e., the weighted total length of the satisfied requests. For the case where all requested pages have the same length, we present an online deterministic algorithm named BAR and prove that it is 4.56-competitive. This improves the previous algorithm of Kim and Chwa [11] which is shown to be 5-competitive by Chan et al. [4]. In the case that pages may have different lengths, we prove a lower bound of Ω(Δ/logΔ) on the competitive ratio where Δ is the ratio of maximum to minimum page lengths. This improves upon the previous $\sqrt{\Delta}$ lower bound in [11,4] and is much closer to the current upper bound of ($\Delta+2\sqrt{\Delta}+2$) in [7]. Furthermore, for small values of Δ we give better lower bounds.

Improved on-line broadcast scheduling with deadlines

Information Dissemination applications are gaining increasing popularity due to dramatic improvements in communications bandwidth and ubiquity. The sheer volume of data available necessitates the use of selective approaches to dissemination in order to avoid overwhelming users with unnecessaryinformation. Existing mechanisms for selective dissemination typically rely on simple keyword matching or “bag of words” information retrieval techniques. The advent of XML as a standard for information exchangeand the development of query languages for XML data enables the development of more sophisticated filtering mechanisms that take structure information into account. We have developed several index organizations and search algorithms for performing efficient filtering of XML documents for large-scale information dissemination systems. In this paper we describe these techniques and examine their performance across a range of document, workload, and scale scenarios.

/pdf/efficient-filtering-of-xml-documents-for-selective-42gzlowzv8.pdf

Efficient Filtering of XML Documents for Selective Dissemination of Information

Broadcast is becoming an increasingly attractive data-dissemination method for large client populations. In order to effectively utilize a broadcast medium for such a service, it is necessary to have efficient on-line scheduling algorithms that can balance individual and overall performance and can scale in terms of data set sizes, client populations, and broadcast bandwidth. We propose an algorithm, called R/spl times/W, that provides good performance across all of these criteria and can be tuned to trade off average and worst-case waiting time. Unlike previous work on low overhead scheduling, the algorithm does not use estimates of the access probabilities of items, but rather, it makes scheduling decisions based on the current queue state, allowing it to easily adapt to changes in the intensity and distribution of the workload. We demonstrate the performance advantages of the algorithm under a range of scenarios using a simulation model and present analytical results that describe the intrinsic behavior of the algorithm.

R/spl times/W: a scheduling approach for large-scale on-demand data broadcast

Advances in telecommunications have enabled the deployment of broadcast-based wide-area information services that provide on-demand data access to very large client populations. In order to effectively utilize a broadcast medium for such a service, it is necessary to have efficient, on-line scheduling algorithms that can balance individual and overall performance and can scale in terms of data set sizes, client populations, and broadcast bandwidth. In this study we introduce a parameterized algorithm that provides good performance across all of these criteria and can be tuned to emphasize either average or worst case waiting time. Unlike previous work on low overhead scheduling, the algorithm is not based on estimates of the access probabilities of items, but rather, it makes scheduling decisions based on the current queue state, allowing it to easily adapt to changes in the intensity and distribution of the workload. We examine the performance of the algorithm using a simulation model.

Scheduling for large-scale on-demand data broadcasting

Broadcast is becoming an increasingly attractive data dissemination method for large client populations. In order to eeectively utilize a broadcast medium for such a service, it is necessary to have eecient, on-line scheduling algorithms that can balance individual and overall performance, and can scale in terms of data set sizes, client populations, and broadcast bandwidth. We propose an algorithm, called RxW, that provides good performance across all of these criteria and that can be tuned to trade oo average and worst case waiting time. Unlike previous work on low overhead scheduling, the algorithm does not use estimates of the access probabilities of items, but rather, it makes scheduling decisions based on the current queue state, allowing it to easily adapt to changes in the intensity and distribution of the workload. We demonstrate the performance advantages of the algorithm under a range of scenarios using a simulation model and present analytical results that describe the intrinsic behavior of the algorithm.

On-Demand Broadcast Scheduling

Design and implementation issues in the fuzzy object-oriented data model

:A>@<b_DCGAdPBEG>R<:c\A8:Q@L]J.A8<0H(L[MOz:CTL0PBc:zbP[<bc4EGADCG ? >@_DA8EHGbP[HP[CTA>@<::ADCTAD<.HTQRJz:CTL0PBcb_DPBEwHw_DPBbP[zbQRA bP0EqPBc\A>Uc\ADPBCGAdP$c:P]HFP z:CGL.PBc:_8PBEwHP[<P]HGHGCFPB_ HG>@?BA4c:P]HFP c\A8QR>@?BA8CGJ P[QRHGA8CG<kP]HG>@?BA2MSLBCQ@PBCG0A&_Q@>RA8<.H  LB:a:QUP]HT>RL0<bE8`v:ADCTAbP0Evz ADAD<Ew>@B<:>Rk_8P[<.HvyL0CGZ HGL]P[CFc:EOc\A8?BA8QRL0:>R<b$LB<\QR>@<:Az:CTL0P0c:_DP0EKHET_F:Adc\a:QR >@<:P[Q@BLBCT>RHG:E MSLBCEGJ\EKHTADE :ADCTA!P[Q@Q c:P]HFP >RHGADEOPBCGAyCGAdPBc\>@QRJPY?]PB>RQUP[z:Q@A>@<¡HG:AEGADCT?BADCd¢ E6P[>@< ADLBCTJB` :A8EGA£EwHGabc:>RAdE(>@B<bLBCTA HG:Ac:P]HFP2P[<: P[0ADAD<.H>UETEwa:AdEOHTbP]HP[CT>@EGA:A8<HG:A c:P[HTP HTL£z A z:CTL0P0c:_DP0EKH£akEKH&z A L0z\HTPB>R<bA8c MSCGL0¤EGA8_DLB<bcbP[CTJ EwHGLBCFP[0A LBC,CTADL[HTA Q@L\_DP]HT>RL0<bE8`¥<¦HT:>@E4bPBkA8C8V  A§:CTLB L0EGA HG:CTADA_LB:Q@ADA8<0HFP[CTJ ̈EGLBQ@a\HG>@LB<kE HGL EGab_F ̈©Ba[«a ¬­«aD®B ̄±°.®_L0<b_A8CG<kED2§L0: LBCGHGa:<:>UEw HG>U_ ET_F:A8c:a:QR>@<:kVEGADCT?BA8C2_DP0_F:>R<bbVP[<bc bCGADMSAHT_F: >@<:b`&bA8EGAHGA8_Fb<:>@3.a:AdE$A ́\:Q@LB>RH:>@<.HTE$:CTL]? >@c:A8c z J,HG:AET_F:A8c\abQR>@<:,P[Q@BLBCT>RHG:,`$μ¶c:AHTPB>RQ@A8c kA8Cw MSLBCTP[<b_DA AD?]P[Q@abP]HT>RL0< akEw>@<:P[<·¥K ̧o1 a:QRHG>U_DPBEwHw zbP0EwAdc HGAdEKHTzkAdc EG:L]vE$HTbP]HHG:AdEwA4c:P]HFP EwHTPBB>@<: HGAd_F:<:>U3.a:A8Ev_8P[< c\CFP[P]HT>@_8P[Q@QRJA8<:bPB<b_A HT:A$kA8Cw MSLBCTP[<b_DA L[M£P Q@PBCG0AEG_8P[Q@A,LB<\c\ADPB<bcz:CTL0P0c  _DP0EKHvEGJ\EKHTAD,`

/pdf/data-staging-for-on-demand-broadcast-2xt7d9q1ue.pdf

Demet Aksoy

Papers

R/spl times/W: a scheduling approach for large-scale on-demand data broadcast

Scheduling for large-scale on-demand data broadcasting

On-Demand Broadcast Scheduling

Design and implementation issues in the fuzzy object-oriented data model

Data Staging for On-Demand Broadcast