In the information era, data is crucial in decision making. Most data sets contain impurities that need to be weeded out before any meaningful decision can be made from the data. Hence, data cleaning is essential and often takes more than 80 percent of time and resources of the data analyst. Adequate tools and techniques must be used for data cleaning. There exist a lot of data cleaning tools but it is unclear how to choose them in various situations. This research aims at helping researchers and organizations choose the right tools for data cleaning. This article conducts a comparative study of four commonly used data cleaning tools on two real data sets and answers the research question of which tool will be useful based on different scenario.

A Comparative Study of Data Cleaning Tools

Many factors affect the success of Machine Learning on a given task. First of all, we need quality data. Data preprocessing is a factor that directly affects the quality of the intellectual analysis process because solving problems with the initial unprocessed sample does not give the expected result, which can lead to erroneous conclusions [1]. This can be caused by a number of errors, such as the repetition of data, the impossible value attributes, missing values, and so on. Such data can occur for a variety of reasons, such as entering data, using different formats or units of measurement, incorrectly deleting recurring value records, and so on [2]. The results of the algorithms used after the detection of errors in the initial sample, logically impossible values of features in the description of objects, and data preprocessing by removing such objects from the sample gives more reliable results.This article proposes to define a range of possible values for each pair of quantitative features. A incorrect object’s data can be identified on entering by the values of a pair of features, that do not fall into the appropriate ranges.

Data preprocessing on input

Data Cleansing or (data scrubbing) is an activity involving a process of detecting and correcting the errors and inconsistencies in data warehouse. Thus poor quality data i.e.; dirty data present in a data mart can be avoided using various data cleaning strategies, and thus leading to more accurate and hence reliable decision making. The quality data can only be produced by cleaning the data and pre-processing it prior to loading it in the data warehouse. As not all the algorithms address the problems related to every type of dirty data, one has to prioritize the need of its organization and use the algorithm according to their requirements and occurrence of dirty data. This paper focuses on the two data cleaning algorithms: Alliance Rules and HADCLEAN and their approaches towards the data quality. It also includes a comparison of the various factors and aspects common to both. General Terms

/pdf/a-comparative-analysis-of-data-cleaning-approaches-to-dirty-3l79exns0l.pdf

A Comparative Analysis of Data Cleaning Approaches to Dirty Data

/pdf/a-naive-gain-approach-to-intrusion-detection-systems-3o6onsagit.pdf

Sonal Porwal

Papers

A Comparative Analysis of Data Cleaning Approaches to Dirty Data

A Naïve Gain Approach to Intrusion Detection Systems