A method and system for synchronizing a main database of a server and a local database of a handheld device. A user can use an application residing in the handheld device to make transactions in the local database. During a synchronization operation, the handheld device and server are coupled. The system then determines whether the application should be updated and, if so, causes the server to provide an update. The system also causes the handheld device to provide to the server information related to the transactions made by the user to the local database. The system causes the server to perform transactions on the main database based on the transaction information. The system also causes the server to extract data from the main database. The server can then provide at least some of the extracted data to the handheld device to update the local database.

Method and system for server synchronization with a computing device

A method and system for direct server synchronization (100) with a computing device is disclosed. In one embodiment, the method comprises synchronizing directly a handheld device (110) and an enterprise server (130), which comprises retrieving a record extraction sequence from the server; and extracting records stored on a database according to the record extraction sequence, wherein the extracted records are not already stored on the computing device.

Method and system for direct server synchronization with a computing device

RDF is a data model for schema-free structured information that is gaining momentum in the context of Semantic-Web data, life sciences, and also Web 2.0 platforms. The "pay-as-you-go" nature of RDF and the flexible pattern-matching capabilities of its query language SPARQL entail efficiency and scalability challenges for complex queries including long join paths. This paper presents the RDF-3X engine, an implementation of SPARQL that achieves excellent performance by pursuing a RISC-style architecture with streamlined indexing and query processing. The physical design is identical for all RDF-3X databases regardless of their workloads, and completely eliminates the need for index tuning by exhaustive indexes for all permutations of subject-property-object triples and their binary and unary projections. These indexes are highly compressed, and the query processor can aggressively leverage fast merge joins with excellent performance of processor caches. The query optimizer is able to choose optimal join orders even for complex queries, with a cost model that includes statistical synopses for entire join paths. Although RDF-3X is optimized for queries, it also provides good support for efficient online updates by means of a staging architecture: direct updates to the main database indexes are deferred, and instead applied to compact differential indexes which are later merged into the main indexes in a batched manner. Experimental studies with several large-scale datasets with more than 50 million RDF triples and benchmark queries that include pattern matching, manyway star-joins, and long path-joins demonstrate that RDF-3X can outperform the previously best alternatives by one or two orders of magnitude.

/pdf/the-rdf-3x-engine-for-scalable-management-of-rdf-data-4khwpjqm1b.pdf

The RDF-3X engine for scalable management of RDF data

Data management workloads are increasingly write-intensive and subject to strict latency SLAs. This presents a dilemma: Update in place systems have unmatched latency but poor write throughput. In contrast, existing log structured techniques improve write throughput but sacrifice read performance and exhibit unacceptable latency spikes.We begin by presenting a new performance metric: read fanout, and argue that, with read and write amplification, it better characterizes real-world indexes than approaches such as asymptotic analysis and price/performance.We then present bLSM, a Log Structured Merge (LSM) tree with the advantages of B-Trees and log structured approaches: (1) Unlike existing log structured trees, bLSM has near-optimal read and scan performance, and (2) its new "spring and gear" merge scheduler bounds write latency without impacting throughput or allowing merges to block writes for extended periods of time. It does this by ensuring merges at each level of the tree make steady progress without resorting to techniques that degrade read performance.We use Bloom filters to improve index performance, and find a number of subtleties arise. First, we ensure reads can stop after finding one version of a record. Otherwise, frequently written items would incur multiple B-Tree lookups. Second, many applications check for existing values at insert. Avoiding the seek performed by the check is crucial.

bLSM: a general purpose log structured merge tree

Partitioning within a B-tree, based on an artificial leading key column and combined with online reorganization, can be exploited during external merge sort for accurate deep read-ahead and dynamic resource allocation, during index creation for a reduced delay until the first query can search the new index, during data loading for streaming integration of new data into a fully indexed database, and for miscellaneous other operations. Despite improving multiple fundamental database operations using a single basic mechanism, the proposal offers these benefits without requiring data structures or algorithms not yet supported in modern relational database management systems. While some of the ideas discussed here have been touched upon elsewhere, the focus here is on re-thinking the relationship between sorting and B-trees more thoroughly, on exploiting this relationship to simplify and unify data structures and algorithms, and on gathering comprehensive lists of issues and benefits.

/pdf/sorting-and-indexing-with-partitioned-b-trees-1n0y32365x.pdf

Sorting And Indexing With Partitioned B-Trees

Broadcast is a scalable way of disseminating data because broadcasting an item satisfies all outstanding client requests for it. However, because the transmission medium is shared, individual requests may have high response times. In this paper, we show how to minimize the average response time given multiple broadcast channels by optimally partitioning data among them. We also offer an approximation algorithm that is less complex than the optimal and show that its performance is near-optimal for a wide range of parameters. Finally, we briefly discuss the extensibility of our work with two simple, yet seldom researched extensions, namely, handling varying sized items and generating single channel schedules.

Efficient data allocation over multiple channels at broadcast servers

The rate of increase in hard disk storage capacity continues to outpace the rate of decrease in hard disk seek time. This trend implies that the value of a seek is increasing exponentially relative to the value of storage.

With this trend in mind, we introduce the partitioned exponential file (PE file) which is a generic storage manager that can be customized for many different types of data (e.g., numerical, spatial, or temporal). The PE file is intended for use in environments with intense update loads and concurrent, analytic queries. Such an environment may be found, for example, in long-running scientific applications which can produce petabytes of data. For example, the proposed Large Synoptic Survey Telescope [36] will produce 50---100 petabytes of observational, scientific data over its multi-year lifetime. This database will never be taken off-line, so bursty update loads of tens of terabytes per day must be handled concurrently with data analysis. In the PE file, data are organized as a series of on-disk sorts with a careful, global organization. Because the PE file relies heavily on sequential I/O, only a fraction of a disk seek is required for a typical record insertion or retrieval.

In addition to describing the PE file, we also detail a set of benchmarking experiments for T1SM, which is a PE file customized for use with multi-attribute data records ordered on a single numerical attribute. In our benchmarking, we implement and test many competing data organizations that can be used to index and store such data, such as the B+-Tree, the LSM-Tree, the Buffer Tree, the Stepped Merge Method, and the Y-Tree. As expected, no organization is the best over all benchmarks, but our experiments show that T1SM is the best choice in many situations, suggesting that it is the best overall. Specifically, T1SM performs exceptionally well in the case of a heavy query workload that must be handled concurrently with an intense insertion stream. Our experiments show that T1SM (and its close cousin, the T2SM storage manager for spatial data) can handle very heavy mixed workloads of this type, and still maintain acceptably small query latencies.

The partitioned exponential file for database storage management

In this paper, we propose techniques for scheduling data broadcasts that are favorable in terms of both response and tuning time. In other words, these techniques ensure that a typical data request will be quickly satisfied and its reception will require a low client-side energy expenditure. By generating broadcast schedules based on Acharya et al.'s broadcast disk paradigm, we bridge the gap between these two mutually exclusive bodies of work-response time and energy expenditure. We prove the utility of our approach analytically and via experiments. Our analysis of optimal scheduling is presented under a variety of assumptions about size and popularity of data items, making our results generalizable to a range of applications.

Bridging the Gap between Response Time and Energy-Efficiency in Broadcast Schedule Design

To avoid the high cost of continuous connectivity, a class of mobile applications employs replicas of shared data that are periodically updated. Updates to these replicas are typically performed on a client-by-client basis--that is, the server individually computes and transmits updates to each client--limiting scalability. By basing updates on replica groups (instead of clients), however, update generation complexity is no longer bound by client population size. Clients then download updates of pertinent groups. Proper group design reduces redundancies in server processing, disk usage and bandwidth usage, and dimininishes the tie between the complexity of updating replicas and the size of the client population. In this paper, we expand on previous work done on group design, include a detailed I/O cost model for update generation, and propose a heuristic-based greedy algorithm for group computation. Experimental results with an adapted commercial replication system demonstrate a significant increase in overall scalability over the client-centric approach.

Scaling replica maintenance in intermittently synchronized mobile databases

We consider the class of mobile computing applications in which clients naturally operate on shared data without a connection to the server. When appropriate, a server connection is made in order to exchange updates. The update server computes and retransmits these updates on a clientby-client basis; consequently, the complexity of these operations is on the order of the number of clients, limiting scalability. We recently proposed exploiting overlap in client data subscriptions by organizing updates to these subscriptions into groups (with less overlap) instead of on a clientby-client basis. By grouping updates, update processing is performed only once per group, irrespective of the number of clients. Additionally, we may gain bandwidth scalability by employing broadcast delivery since, unlike in the case of the per-client approach, multiple clients may be interested in a group's updates. In this work, we model the operations of such database systems and introduce a framework for evaluation of group design. Since such fragmentation algorithms are computationally intractable, a heuristic graphbased group generation algorithm is speci ed. Performance results executed on a prototype developed using commercially available software are presented.

Wai Gen Yee

Papers

Efficient data allocation over multiple channels at broadcast servers

The partitioned exponential file for database storage management

Bridging the Gap between Response Time and Energy-Efficiency in Broadcast Schedule Design

Scaling replica maintenance in intermittently synchronized mobile databases

A framework for designing update objects to improve server scalability in intermittently synchronized databases