Statistical Analysis with Missing Data

https://isiarticles.com/bundles/Article/pre/pdf/8179.pdf

A hybrid method for imputation of missing values using optimized fuzzy c-means with support vector regression and a genetic algorithm

In real-life data, information is frequently lost in data mining, caused by the presence of missing values in attributes. Several schemes have been studied to overcome the drawbacks produced by missing values in data mining tasks; one of the most well known is based on preprocessing, formerly known as imputation. In this work, we focus on a classification task with twenty-three classification methods and fourteen different imputation approaches to missing values treatment that are presented and analyzed. The analysis involves a group-based approach, in which we distinguish between three different categories of classification methods. Each category behaves differently, and the evidence obtained shows that the use of determined missing values imputation methods could improve the accuracy obtained for these methods. In this study, the convenience of using imputation methods for preprocessing data sets with missing values is stated. The analysis suggests that the use of particular imputation methods conditioned to the groups is required.

https://sci2s.ugr.es/sites/default/files/ficherosPublicaciones/1408_2012-luengo-garcia-herrera-KAIS.pdf

On the choice of the best imputation methods for missing values considering three groups of classification methods

Missing value imputation on missing completely at random data using multilayer perceptrons

Machine learning has been the corner stone in analysing and extracting information from data and often a problem of missing values is encountered. Missing values occur because of various factors like missing completely at random, missing at random or missing not at random. All these may result from system malfunction during data collection or human error during data pre-processing. Nevertheless, it is important to deal with missing values before analysing data since ignoring or omitting missing values may result in biased or misinformed analysis. In literature there have been several proposals for handling missing values. In this paper, we aggregate some of the literature on missing data particularly focusing on machine learning techniques. We also give insight on how the machine learning approaches work by highlighting the key features of missing values imputation techniques, how they perform, their limitations and the kind of data they are most suitable for. We propose and evaluate two methods, the k nearest neighbor and an iterative imputation method (missForest) based on the random forest algorithm. Evaluation is performed on the Iris and novel power plant fan data with induced missing values at missingness rate of 5% to 20%. We show that both missForest and the k nearest neighbor can successfully handle missing values and offer some possible future research direction.

/pdf/a-survey-on-missing-data-in-machine-learning-5d62yrtdu1.pdf

A survey on missing data in machine learning.

In KDD procedure, to fill in missing data typically requires a very large investment of time and energy – often 80% to 90% of a data analysis project is spent in making the data reliable enough so that the results can be trustful. In this paper, we propose a SVM regression based algorithm for filling in missing data, i.e. set the decision attribute (output attribute) as the condition attribute (input attribute) and the condition attribute as the decision attribute, then use SVM regression to predict the condition attribute values. SARS data set experimental results show that SVM regression method has the highest precision. The method with which the value of the example that has the minimum distance to the example with missing value will be taken to fill in the missing values takes the second place, and the mean and median methods have lower precision.

/pdf/a-svm-regression-based-approach-to-filling-in-missing-values-43jjyxtbvi.pdf

A SVM regression based approach to filling in missing values

SARS is an acute infectious disease and can cause a large amount of death. Up until now we have not known it well. With the experimental results of micronutrients of 30 SARS patients and 30 non-SARS patients, using rough set theory we induce some classification rules. Attribute reduction results show that micronutrients Fe, Ca, K and Na are necessary and sufficient for classification, whereas micronutrients Zn, Cu and Mg are not necessary or are redundant. Additionally, we find that micronutrient Ca has a strong correlation to SARS. The classification results of 30 other examples show that the rough set classification method is available

Chen Guoshun

Papers

A SVM regression based approach to filling in missing values

Rough Set Based Classification rules generation for SARS Patients