Different notions of provenance for database queries have been proposed and studied in the past few years. In this article, we detail three main notions of database provenance, some of their applications, and compare and contrast amongst them. Specifically, we review why, how, and where provenance, describe the relationships among these notions of provenance, and describe some of their applications in confidence computation, view maintenance and update, debugging, and annotation propagation.

/pdf/provenance-in-databases-why-how-and-where-2i4pg6ky6g.pdf

Provenance in Databases: Why, How, and Where

Provenance in the context of workflows, both for the data they derive and for their specification, is an essential component to allow for result reproducibility, sharing, and knowledge re-use in the scientific community. Several workshops have been held on the topic, and it has been the focus of many research projects and prototype systems. This tutorial provides an overview of research issues in provenance for scientific workflows, with a focus on recent literature and technology in this area. It is aimed at a general database research audience and at people who work with scientific data and workflows. We will (1) provide a general overview of scientific workflows, (2) describe research on provenance for scientific workflows and show in detail how provenance is supported in existing systems; (3) discuss emerging applications that are enabled by provenance; and (4) outline open problems and new directions for database-related research.

/pdf/provenance-and-scientific-workflows-challenges-and-27dsk8tmf9.pdf

Provenance and scientific workflows: challenges and opportunities

Communication Complexity: Basics

We propose a novel index structure, termed XTrie, that supports the efficient filtering of XML documents based on XPath expressions. Our XTrie index structure offers several novel features that make it especially attractive for large scale publish/subscribe systems. First, XTrie is designed to support effective filtering based on complex XPath expressions (as opposed to simple, single-path specifications). Second, our XTrie structure and algorithms are designed to support both ordered and unordered matching of XML data. Third, by indexing on sequences of element names organized in a trie structure and using a sophisticated matching algorithm, XTrie is able to both reduce the number of unnecessary index probes as well as avoid redundant matchings, thereby providing extremely efficient filtering. Our experimental results over a wide range of XML document and XPath expression workloads demonstrate that our XTrie index structure outperforms earlier approaches by wide margins.

/pdf/efficient-filtering-of-xml-documents-with-xpath-expressions-4ks207aqfu.pdf

Efficient filtering of XML documents with XPath expressions

The prevalent use of XML highlights the need for a generic, flexible access-control mechanism for XML documents that supports efficient and secure query access, without revealing sensitive information unauthorized users. This paper introduces a novel paradigm for specifying XML security constraints and investigates the enforcement of such constraints during XML query evaluation. Our approach is based on the novel concept of security views, which provide for each user group (a) an XML view consisting of all and only the information that the users are authorized to access, and (b) a view DTD that the XML view conforms to. Security views effectively protect sensitive data from access and potential inferences by unauthorized user, and provide authorized users with necessary schema information to facilitate effective query formulation and optimization. We propose an efficient algorithm for deriving security view definitions from security policies (defined on the original document DTD) for different user groups. We also develop novel algorithms for XPath query rewriting and optimization such that queries over security views can be efficiently answered without materializing the views. Our algorithms transform a query over a security view to an equivalent query over the original document, and effectively prune query nodes by exploiting the structural properties of the document DTD in conjunction with approximate XPath containment tests. Our work is the first to study a flexible, DTD-based access-control model for XML and its implications on the XML query-execution engine. Furthermore, it is among the first efforts for query rewriting and optimization in the presence of general DTDs for a rich a class of XPath queries. An empirical study based on real-life DTDs verifies the effectiveness of our approach.

/pdf/secure-xml-querying-with-security-views-1yibt58568.pdf

Secure XML querying with security views

As the world is increasingly networked and digitized, the data we store has more and more frequently been chopped, baked, diced and stewed. In consequence, there is an increasing need to store and manage provenance for each data item stored in a database, describing exactly where it came from, and what manipulations have been applied to it. Storage of the complete provenance of each data item can become prohibitively expensive. In this paper, we identify important properties of provenance that can be used to considerably reduce the amount of storage required.We identify three different techniques: a family of factorization processes and two methods based on inheritance, to decrease the amount of storage required for provenance. We have used the techniques described in this work to significantly reduce the provenance storage costs associated with constructing MiMI [22], a warehouse of data regarding protein interactions, as well as two provenance stores, Karma [31] and PReServ [20], produced through workflow execution. In these real provenance sets, we were able to reduce the size of the provenance by up to a factor of 20. Additionally, we show that this reduced store can be queried efficiently and further that incremental changes can be made inexpensively.

/pdf/efficient-provenance-storage-oifhr996pi.pdf

Efficient provenance storage

We consider the problem of minimizing tree pattern queries (TPQ) that arise in XML and in LDAP-style network directories. In [Minimization of Tree Pattern Queries, Proc. ACM SIGMOD Intl. Conf. Management of Data, 2001, pp. 497-508], Amer-Yahia, Cho, Lakshmanan and Srivastava presented an O(n4) algorithm for minimizing TPQs in the absence of integrity constraints (Case 1); n is the number of nodes in the query. Then they considered the problem of minimizing TPQs in the presence of three kinds of integrity constraints: required-child, required-descendant and subtype (Case 2). They presented an O(n6) algorithm for minimizing TPQs in the presence of only required-child and required-descendant constraints (i.e., no subtypes allowed; Case 3). We present O(n2), O(n4) and O(n2) algorithms for minimizing TPQs in these three cases, respectively, based on the concept of graph simulation. We believe that our O(n2) algorithms for Cases 1 and 3 are runtime optimal.

Efficient algorithms for minimizing tree pattern queries

Tree Pattern Queries (TPQ), Branching Path Queries (BPQ), and Core XPath (CXPath) are subclasses of the XML query language XPath, TPQ ⊂ BPQ ⊂ CX Path ⊂ X Path. Let TPQ = TPQ+ ⊂ BPQ+ ⊂ CX Path+ ⊂ X Path+ denote the corresponding subclasses, consisting of queries that do not involve the boolean negation operator not in their predicates. Simulation and bisimulation are two different binary relations on graph vertices that have previously been studied in connection with some of these classes. For instance, TPQ queries can be minimized using simulation. Most relevantly, for an XML document, its bisimulation quotient is the smallest index that covers (i.e., can be used to answer) all BPQ queries. Our results are as follows: • A CXPath+ query can be evaluated on an XML document by computing the simulation of the query tree by the document graph. • For an XML document, its simulation quotient is the smallest covering index for BPQ+. This, together with the previously-known result stated above, leads to the following: For BPQ covering indexes of XML documents, Bisimulation - Simulation = Negation. • For an XML document, its simulation quotient, with the idref edges ignored throughout, is the smallest covering index for TPQ.

For any XML document, its simulation quotient is never larger than its bisimulation quotient; in some instances, it is exponentially smaller. Our last two results show that disallowing negation in the queries could substantially reduce the size of the smallest covering index.

/pdf/covering-indexes-for-xml-queries-bisimulation-simulation-1m632ikikb.pdf

Covering indexes for XML queries: bisimulation - simulation = negation

Tree Pattern Queries (TPQ), Branching Path Queries (BPQ), and Core XPath (CXPath) are subclasses of the XML query language XPath, TPQ ⊂ BPQ ⊂ CXPath ⊂ XPath. Let TPQ = TPQ ⊂ BPQ ⊂ CXPath ⊂ XPath denote the corresponding subclasses, consisting of queries that do not involve the boolean negation operator not in their predicates. Simulation and bisimulation are two different binary relations on graph vertices that have previously been studied in connection with some of these classes. For instance, TPQ queries can be minimized using simulation. Most relevantly, for an XML document, its bisimulation quotient is the smallest index that covers (i.e., can be used to answer) all BPQ queries. Our results are as follows: • Evaluating a CXPath query on an XML document is equivalent to computing the simulation of the query tree by the document graph. • For an XML document, its simulation quotient is the smallest covering index for BPQ. This, together with the previously-known result stated above, leads to the following: For BPQ covering indexes of XML documents, Bisimulation − Simulation = Negation. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the VLDB copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Very Large Data Base Endowment. To copy otherwise, or to republish, requires a fee and/or special permission from the Endowment. Proceedings of the 29th VLDB Conference, Berlin, Germany, 2003 • For an XML document, its simulation quotient, with the idref edges ignored throughout, is the smallest covering index for TPQ. For any XML document, its simulation quotient is never larger than its bisimulation quotient; in some instances, it is exponentially smaller. Our last two results show that disallowing negation in the queries could reduce the size of the smallest covering index.

Paper ID 238: Covering Indexes for XML Queries: Bisimulation – Simulation = Negation

The longest increasing subsequence problem is as follows: Given a sequence of n real numbers, find a longest increasing subsequence of . There is a well-known O(n lg n)-time comparison tree algorithm for solving this problem. Also, a tight Ω(n lg n) lower bound in the comparison tree model is known. We prove a tight Ω(n lg n) lower bound in the more powerful algebraic decision tree model. The above lower bounds also apply to the apparently simpler problem of finding the length of a longest increasing subsequence.

Prakash Ramanan

Papers

Efficient provenance storage

Efficient algorithms for minimizing tree pattern queries

Covering indexes for XML queries: bisimulation - simulation = negation

Paper ID 238: Covering Indexes for XML Queries: Bisimulation – Simulation = Negation

Tight Ω(n lg n) lower bound for finding a longest increasing subsequence