http://www.cs.unibo.it/~montesi/CBD/Articoli/SurvyBigData2.pdf

Data-intensive applications, challenges, techniques and technologies: A survey on Big Data

KDD is a complex and demanding task While a large number of methods has been established for numerous problems, many challenges remain to be solved New tasks emerge requiring the development of new methods or processing schemes Like in software development, the development of such solutions demands for careful analysis, specification, implementation, and testing Rapid prototyping is an approach which allows crucial design decisions as early as possible A rapid prototyping system should support maximal re-use and innovative combinations of existing methods, as well as simple and quick integration of new onesThis paper describes Yale, a free open-source environment forKDD and machine learning Yale provides a rich variety of methods whichallows rapid prototyping for new applications and makes costlyre-implementations unnecessary Additionally, Yale offers extensive functionality for process evaluation and optimization which is a crucial property for any KDD rapid prototyping tool Following the paradigm of visual programming eases the design of processing schemes While the graphical user interface supports interactive design, the underlying XML representation enables automated applications after the prototyping phaseAfter a discussion of the key concepts of Yale, we illustrate the advantages of rapid prototyping for KDD on case studies ranging from data pre-processing to result visualization These case studies cover tasks like feature engineering, text mining, data stream mining and tracking drifting concepts, ensemble methods and distributed data mining This variety of applications is also reflected in a broad user base, we counted more than 40,000 downloads during the last twelve months

YALE: rapid prototyping for complex data mining tasks

The age of big data is now coming. But the traditional data 
analytics may not be able to handle such large quantities of data. The question that arises now is, how to develop a high performance platform to efficiently analyze big data and 
how to design an appropriate mining algorithm to find the useful things from big data. To deeply discuss this issue, this paper begins with a brief introduction to data analytics, followed by the discussions of big data analytics. 
Some important open issues and further research directions will also be presented for the next step of big data analytics.

/pdf/big-data-analytics-a-survey-2m5v4g89fb.pdf

Big data analytics: a survey

The development and application of data mining algorithms requires the use of powerful software tools. As the number of available tools continues to grow, the choice of the most suitable tool becomes increasingly difficult. This paper attempts to support the decision-making process by discussing the historical development and presenting a range of existing state-of-the-art data mining and related tools. Furthermore, we propose criteria for the tool categorization based on different user groups, data structures, data mining tasks and methods, visualization and interaction styles, import and export options for data and models, platforms, and license policies. These criteria are then used to classify data mining tools into nine different types. The typical characteristics of these types are explained and a selection of the most important tools is categorized. This paper is organized as follows: the first section Historical Development and State-of-the-Art highlights the historical development of data mining software until present; the criteria to compare data mining software are explained in the second section Criteria for Comparing Data Mining Software. The last section Categorization of Data Mining Software into Different Types proposes a categorization of data mining software and introduces typical software tools for the different types. © 2011 John Wiley & Sons, Inc. WIREs Data Mining Knowl Discov 2011 1 431-443 DOI: 10.1002/widm.24

Data mining tools

The practical relevance of process mining is increasing as more and more event data become available. Process mining techniques aim to discover, monitor and improve real processes by extracting knowledge from event logs. The two most prominent process mining tasks are: (i) process discovery: learning a process model from example behavior recorded in an event log, and (ii) conformance checking: diagnosing and quantifying discrepancies between observed behavior and modeled behavior. The increasing volume of event data provides both opportunities and challenges for process mining. Existing process mining techniques have problems dealing with large event logs referring to many different activities. Therefore, we propose a generic approach to decompose process mining problems. The decomposition approach is generic and can be combined with different existing process discovery and conformance checking techniques. It is possible to split computationally challenging process mining problems into many smaller problems that can be analyzed easily and whose results can be combined into solutions for the original problems.

/pdf/decomposing-petri-nets-for-process-mining-a-generic-approach-4z2emcsrfd.pdf

Decomposing Petri nets for process mining: A generic approach

Data mining algorithms are widely used today for the analysis of large corporate and scientific datasets stored in databases and data archives. Industry, science, and commerce fields often need to analyze very large datasets maintained over geographically distributed sites by using the computational power of distributed and parallel systems. The grid can play a significant role in providing an effective computational support for distributed knowledge discovery applications. For the development of data mining applications on grids we designed a system called KNOWLEDGE GRID. This paper describes the KNOWLEDGE GRID framework and presents the toolset provided by the KNOWLEDGE GRID for implementing distributed knowledge discovery. The paper discusses how to design and implement data mining applications by using the KNOWLEDGE GRID tools starting from searching grid resources, composing software and data components, and executing the resulting data mining process on a grid. Some performance results are also discussed.

/pdf/distributed-data-mining-on-grids-services-tools-and-1dgylxq3xt.pdf

Distributed data mining on grids: services, tools, and applications

The continuous increase of data volumes available from many sources raises new challenges for their effective understanding. Knowledge discovery in large data repositories involves processes and activities that are computationally intensive, collaborative, and distributed in nature. The Grid is a profitable infrastructure that can be effectively exploited for handling distributed data mining and knowledge discovery. To achieve this goal, advanced software tools and services are needed to support the development of KDD applications. The Knowledge Grid is a high-level framework providing Grid-based knowledge discovery tools and services. Such services allow users to create and manage complex knowledge discovery applications that integrate data sources and data mining tools provided as distributed services on the Grid. All of these services are currently being re-designed and re-implemented as WSRF-compliant Grid Services. This paper highlights design aspects and implementation choices involved in such a process.

Distributed data mining services leveraging WSRF

Service-oriented middleware for distributed data mining on the grid

Today a large number of scientific and commercial applications often require to analyse large data sets maintained over geographically distributed sites by using the computational power of distributed high-performance environments. Advances in networking technology and computational infrastructure made it possible to construct large-scale distributed computing platforms, called computational grids, that provide dependable, consistent, and pervasive access to high-end computational resources. Grids can play a significant role in providing an effective computational support for distributed data mining applications. Currently we are developing a software system for geographically distributed knowledge discovery applications called KNOWLEDGE GRID,which is designed on top of computational grid mechanisms, provided by grid environments such as Glob us. In this paper we present an integrated toolset named VEGA (Visual Environment for Grid Applications), which allows a Knowledge Grid user to develop and execute distributed data mining computations in a simple and effective way.

/pdf/a-data-mining-toolset-for-distributed-high-performance-1fec0pkhr8.pdf

A data mining toolset for distributed high- performance platforms

The increasing use of computers in all the areas of human activities is resulting in huge collections of digital data. Databases are common everywhere and are used as repositories of every kind of data. Knowledge discovery techniques and tools are used today to analyze those very large data sets to identify interesting patterns and trends in them. When data is maintained over geographically distributed sites the computational power of distributed and parallel systems can be exploited for knowledge discovery in databases. In this scenario the Grid can provide an effective computational support for distributed knowledge discovery on large data sets. To this purpose we designed a system called Knowledge Grid This chapter describes the Knowledge Grid architecture and discusses some related systems and models recently proposed for knowledge discovery on Grids. The chapter presents also how to design and implement distributed data mining applications by using the Knowledge Grid tools starting from searching Grid resources, composing software and data elements, and executing the resulting application on a Grid.

Antonio Congiusta

Papers

Distributed data mining on grids: services, tools, and applications

Distributed data mining services leveraging WSRF

Service-oriented middleware for distributed data mining on the grid

A data mining toolset for distributed high- performance platforms

Grid-Based Data Mining and Knowledge Discovery