Data Mining Practical Machine Learning Tools and Techniques

Scientific facilities such as the Advanced Light Source (ALS) and Joint Genome Institute and projects such as the Materials Project have an increasing need to capture, store, and analyze dynamic semi-structured data and metadata. A similar growth of semi-structured data within large Internet service providers has led to the creation of NoSQL data stores for scalable indexing and MapReduce for scalable parallel analysis. MapReduce and NoSQL stores have been applied to scientific data. Hadoop, the most popular open source implementation of MapReduce, has been evaluated, utilized and modified for addressing the needs of different scientific analysis problems. ALS and the Materials Project are using MongoDB, a document oriented NoSQL store. However, there is a limited understanding of the performance trade-offs of using these two technologies together.In this paper we evaluate the performance, scalability and fault-tolerance of using MongoDB with Hadoop, towards the goal of identifying the right software environment for scientific data analysis.

/pdf/performance-evaluation-of-a-mongodb-and-hadoop-platform-for-uga3ttt9iy.pdf

Performance evaluation of a MongoDB and hadoop platform for scientific data analysis

While Big Data gradually become a hot topic of research and business and has been everywhere used in many industries, Big Data security and privacy has been increasingly concerned. However, there is an obvious contradiction between Big Data security and privacy and the widespread use of Big Data. In this paper, we firstly reviewed the enormous benefits and challenges of security and privacy in Big Data. Then, we present some possible methods and techniques to ensure Big Data security and privacy.

Big Data security and privacy: A review

We are constantly being told that we live in the Information Era - the Age of BIG data. It is clearly apparent that organizations need to employ data-driven decision making to gain competitive advantage. Processing, integrating and interacting with more data should make it better data, providing both more panoramic and more granular views to aid strategic decision making. This is made possible via Big Data exploiting affordable and usable Computational and Storage Resources. Many offerings are based on the Map-Reduce and Hadoop paradigms and most focus solely on the analytical side. Nonetheless, in many respects it remains unclear what Big Data actually is, current offerings appear as isolated silos that are difficult to integrate and/or make it difficult to better utilize existing data and systems. Paper addresses this lacunae by characterising the facets of Big Data and proposing a framework in which Big Data applications can be developed. The framework consists of three Stages and seven Layers to divide Big Data application into modular blocks. The aim is to enable organizations to better manage and architect a very large Big Data application to gain competitive advantage by allowing management to have a better handle on data processing.

Big Data Framework

With the advancement in the database technology NoSQL databases are becoming more and more popular now a days. The cloud-based NoSQL database MongoDB is one of them. As one can know the data modeling strategies for relational and non-relational databases differ drastically. Modeling of data in MongoDB database depends on the data and characteristics of MongoDB. Due to variation in data models MongoDB based application performance gets affected. In the present paper we have applied two different modeling styles as embedding of documents and normalization on collections. With the embedding feature we may face situation where documents grow in size after creation which may degrade the performance of database. The maximum document size allowed in MongoDB is limited. With references we get maximum flexibility than embedding but client-side applications must issue follow-up queries to resolve the references. The joins in this case cannot be effectively used. Hence there is need for defining the strategy of extent of normalization and embedding to get better performance in the mixed situation. The paper discussed here shows the variation in the performance along with the change in the modeling style with reference to normalization and embedding and it gives the base to find the extent of normalization and embedding for reducing query execution time.

A study of normalization and embedding in MongoDB

Data aggregation is one of the key features used in databases, especially for Business Intelligence (e.g., ETL, OLAP) and analytics/data mining. When considering SQL databases, aggregation is used to prepare and visualize data for deeper analyses. However, these operations are often impossible on very large volumes of data regarding memory-and-time-consumption. In this paper, we show how NoSQL databases such as MongoDB and its key-value stores, thanks to the native MapReduce algorithm, can provide an efficient framework to aggregate large volumes of data. We provide basic material about the MapReduce algorithm, the different NoSQL databases (read intensive vs. write intensive). We investigate how to efficiently modelize the data framework for BI and analytics. For this purpose, we focus on read intensive NoSQL databases using MongoDB and we show how NoSQL and MapReduce can help handling large volumes of data.

/pdf/reduce-you-say-what-nosql-can-do-for-data-aggregation-and-bi-4tomi8sghl.pdf

Reduce, You Say: What NoSQL Can Do for Data Aggregation and BI in Large Repositories

https://hal-lirmm.ccsd.cnrs.fr/lirmm-01239175/document

Spatio-temporal data classification through multidimensional sequential patterns: Application to crop mapping in complex landscape

multidimensional sequential patterns: Application to crop mapping in complex landscape

Paper recommendation to researchers has been extensively studied in the last years, and many methods have been investigated for this purpose. In this paper, we propose a novel approach embedding the whole process for selecting papers of interest given some keywords. Our approach is based on a workflow integrating fuzzy clustering of the papers, the computation of a representative summary paper per cluster using OWA operators, and ranking, in order to answer user queries adequately. The originality of our method relies in the introduction of fuzziness for more flexibility in the approach. The use of representative papers allows us to summarize sets of papers into a single representative one, thus simplifying the users interactions with the huge number of papers from the literature.

https://hal-lirmm.ccsd.cnrs.fr/lirmm-00803915/document

Paper Recommendation System: A Global and Soft Approach

Gradual patterns have been studied for many years as they contain precious information. They have been integrated in many expert systems and rule-based systems, for instance to reason on knowledge such as {\\em ''the greater the number of turns, the greater the number of car crashes\"}. In many cases, this knowledge has been considered as a rule {\\em ''the greater the number of turns $\\rightarrow$ the greater the number of car crashes\"} Historically, works have thus been focused on the representation of such rules, studying how implication could be defined, especially fuzzy implication. These rules were defined by experts who were in charge to describe the systems they were working on in order to turn them to operate automatically. More recently, approaches have been proposed in order to mine databases for automatically discovering such knowledge. Several approaches have been studied, the main scientific topics being: how to determine what is an relevant gradual pattern, and how to discover them as efficiently as possible (in terms of both memory and CPU usage). However, in some cases, end-users are not interested in raw level knowledge, and are rather interested in trends. Moreover, it may be the case that no relevant pattern can be discovered at a low level of granularity (e.g. city), whereas some can be discovered at a higher level (e.g. county). In this paper, we thus extend gradual pattern approaches in order to consider multiple level gradual patterns. For this purpose, we consider two aggregation policies, namely horizontal and vertical.

https://hal-lirmm.ccsd.cnrs.fr/lirmm-00803916/document

Michel Sala

Papers

Reduce, You Say: What NoSQL Can Do for Data Aggregation and BI in Large Repositories

Spatio-temporal data classification through multidimensional sequential patterns: Application to crop mapping in complex landscape

multidimensional sequential patterns: Application to crop mapping in complex landscape

Paper Recommendation System: A Global and Soft Approach

M2LGP: Mining Multiple Level Gradual Patterns