There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

The book Knowledge Discovery in Databases, edited by Piatetsky-Shapiro and Frawley [PSF91], is an early collection of research papers on knowledge discovery from data. The book Advances in Knowledge Discovery and Data Mining, edited by Fayyad, Piatetsky-Shapiro, Smyth, and Uthurusamy [FPSSe96], is a collection of later research results on knowledge discovery and data mining. There have been many data mining books published in recent years, including Predictive Data Mining by Weiss and Indurkhya [WI98], Data Mining Solutions: Methods and Tools for Solving Real-World Problems by Westphal and Blaxton [WB98], Mastering Data Mining: The Art and Science of Customer Relationship Management by Berry and Linofi [BL99], Building Data Mining Applications for CRM by Berson, Smith, and Thearling [BST99], Data Mining: Practical Machine Learning Tools and Techniques by Witten and Frank [WF05], Principles of Data Mining (Adaptive Computation and Machine Learning) by Hand, Mannila, and Smyth [HMS01], The Elements of Statistical Learning by Hastie, Tibshirani, and Friedman [HTF01], Data Mining: Introductory and Advanced Topics by Dunham, and Data Mining: Multimedia, Soft Computing, and Bioinformatics by Mitra and Acharya [MA03]. There are also books containing collections of papers on particular aspects of knowledge discovery, such as Machine Learning and Data Mining: Methods and Applications edited by Michalski, Brakto, and Kubat [MBK98], and Relational Data Mining edited by Dzeroski and Lavrac [De01], as well as many tutorial notes on data mining in major database, data mining and machine learning conferences.

/pdf/data-mining-concepts-and-techniques-2nd-edition-zx1xx1f5e3.pdf

Data Mining: Concepts and Techniques (2nd edition)

We discuss the design of an acquisitional query processor for data collection in sensor networks. Acquisitional issues are those that pertain to where, when, and how often data is physically acquired (sampled) and delivered to query processing operators. By focusing on the locations and costs of acquiring data, we are able to significantly reduce power consumption over traditional passive systems that assume the a priori existence of data. We discuss simple extensions to SQL for controlling data acquisition, and show how acquisitional issues influence query optimization, dissemination, and execution. We evaluate these issues in the context of TinyDB, a distributed query processor for smart sensor devices, and show how acquisitional techniques can provide significant reductions in power consumption on our sensor devices.

/pdf/tinydb-an-acquisitional-query-processing-system-for-sensor-2akih312yn.pdf

TinyDB: an acquisitional query processing system for sensor networks

Most statistical and machine-learning algorithms assume that the data is a random sample drawn from a stationary distribution. Unfortunately, most of the large databases available for mining today violate this assumption. They were gathered over months or years, and the underlying processes generating them changed during this time, sometimes radically. Although a number of algorithms have been proposed for learning time-changing concepts, they generally do not scale well to very large databases. In this paper we propose an efficient algorithm for mining decision trees from continuously-changing data streams, based on the ultra-fast VFDT decision tree learner. This algorithm, called CVFDT, stays current while making the most of old data by growing an alternative subtree whenever an old one becomes questionable, and replacing the old with the new when the new becomes more accurate. CVFDT learns a model which is similar in accuracy to the one that would be learned by reapplying VFDT to a moving window of examples every time a new example arrives, but with O(1) complexity per example, as opposed to O(w), where w is the size of the window. Experiments on a set of large time-changing data streams demonstrate the utility of this approach.

/pdf/mining-time-changing-data-streams-a9kghvc13p.pdf

Mining time-changing data streams

In the data stream scenario, input arrives very rapidly and there is limited memory to store the input. Algorithms have to work with one or few passes over the data, space less than linear in the input size or time significantly less than the input size. In the past few years, a new theory has emerged for reasoning about algorithms that work within these constraints on space, time, and number of passes. Some of the methods rely on metric embeddings, pseudo-random computations, sparse approximation theory and communication complexity. The applications for this scenario include IP network traffic analysis, mining text message streams and processing massive data sets in general. Researchers in Theoretical Computer Science, Databases, IP Networking and Computer Systems are working on the data stream challenges. This article is an overview and survey of data stream algorithmics and is an updated version of [1].

/pdf/data-streams-algorithms-and-applications-5fo85u5tcy.pdf

Data streams: algorithms and applications

In this paper, we describe a scenario for a distributed marketplace on the Web where resource discovery agents find out about resources that may want to join the marketplace and electronic commerce is realized through buying agents representing the customers and the selling agents representing the resources like electronic catalogs.

We propose a possible architecture which is based on the emerging technologies and standards. In this architecture, the resources expose their metadata using Resource Description Framework (RDF) to be accessed by the resource discovery agents and their content through Extensible Markup Language (XML) to be accessed by the selling agents by using Document Object Model (DOM). The marketplace contains Document Type Definitions (DTDs) and a dictionary of synonyms to be used by the buying agents to help the customer to specify the item s/he wishes to purchase. Distribution infrastructure is CORBA and Web on which the buying and selling agents find out about each other using Trading Object Services. The modifications necessary to the proposed architecture considering only the available technology are also discussed.

METU-Emar: An Agent-Based Electronic Marketplace on the Web

Traffic-aware real-time route planning has recently been an application of increasing interest for metropolitan cities with busy traffic. This paper approaches the problem from a stream processing point of view and proposes a general architecture to solve it. This work is inspired by a real use case and is implemented on an industry-strength stream processing engine. Experimental results on this implementation demonstrate the scalability of this approach in terms of increasing data and query rates.

http://people.csail.mit.edu/tatbul/publications/iwgs11.pdf

Real-time route planning with stream processing systems: a case study for the city of Lucerne

In this paper, we propose to demonstrate a "stream-as-you-go" approach that minimizes the data transfer time of data- and compute-intensive scientific applications deployed in the cloud, by making them incrementally processable. We describe a system that implements this approach based on the IBM InfoSphere Streams computing platform deployed over Amazon EC2. The functionality, performance, and usability of the system will be demonstrated through two DNA sequence analysis applications.

/pdf/incremental-dna-sequence-analysis-in-the-cloud-3rx6ds82p0.pdf

Incremental DNA sequence analysis in the cloud

Work ow management systems aim to automate the execution of business processes One of the objectives of the work ow systems is to include the already existing applications such as legacy applications as well as new applications which are termed as tasks into the system and provide synchronized execution among them To achieve this a mechanism is necessary to support the communication between the tasks and the system The communication mechanism should handle the transfer of data necessary for the execution of the tasks and for the scheduling of the tasks Another point to be noted is the necessity of the handling user tasks that have to be performed by the users of the work ow system Since the trend is toward distributed execution to avoid the bottlenecks due to the nature of central systems we considered these issues in a distributed execution environment Therefore in this paper task handling in a truely distributed work ow management system that is being developed at METU namely METUFlow is described Yet the techniques described are general enough to be applicable to any work ow management system

Task Handling in Workflow Management Systems.

Data stream management systems may be subject to higher input rates than their resources can handle. In this case, results get delayed and Quality of Service (QoS) at system outputs may fall below acceptable levels. Load shedding addresses this problem by allowing data loss in exchange for reduced latency. Drop operators are placed at carefully chosen points in a query plan, in order to relieve overload with minimal loss in answer quality. In this paper, we describe a load shedding technique for queries consisting of one or more aggregate operators with sliding windows. We introduce a sophisticated drop operator, called a “Windowed Drop”. This operator is aware of window properties (i.e., window size and window slide) of downstream aggregate operators in the query plan. Accordingly, it logically partitions the stream into windows and probabilistically decides which windows to drop. This decision is further encoded into tuples by marking the ones that are disallowed from starting new windows. Unlike earlier approaches, our approach preserves integrity of windows throughout a query plan, and always delivers subsets of original query answers with minimal degradation in overall QoS utility.

Nesime Tatbul

Papers

METU-Emar: An Agent-Based Electronic Marketplace on the Web

Real-time route planning with stream processing systems: a case study for the city of Lucerne

Incremental DNA sequence analysis in the cloud

Task Handling in Workflow Management Systems.

Window-aware Load Shedding for Data Streams