In the current era of big data, huge quantities of valuable data, which may be of different levels of veracity, are being generated at a rapid rate. Embedded into these big data are implicit, previously unknown and potentially useful information and valuable knowledge that can be discovered by data science solutions, which apply techniques like data mining. There has been a trend that more and more collections of these big data have been made openly available in science, government and non-profit organizations so that people could collaboratively study and analysis these open big data. In this article, we focus on open big data for public transit because public transit (e.g., bus) as a means of transportation is a vital part of many people’s lives. As time is a precious resource, bus delays could negatively affect commuters’ plans. Unfortunately, they are inevitable. Hence, many existing works focused on predicting bus delays. However, predicting on-time or early buses is also important. For instance, commuters who come to a bus stop on time may still miss their buses if the buses leave early. So, in this article, we examine open big data about bus performance (e.g., early, on-time, and late stops). We analyze the data with frequent pattern mining and make predictions with decision-tree based classification. For illustration, we perform predictive analytics on real-life open big data available on Winnipeg Open Data Portal, about bus performance from Winnipeg Transit. It shows the benefits of predictive analytics on open big data for supporting smart transportation services.

Predictive analytics on open big data for supporting smart transportation services.

Today, predictive analytics applications became an urgent desire in higher educational institutions. Predictive analytics used advanced analytics that encompasses machine learning implementation to derive high-quality performance and meaningful information for all education levels. Mostly know that student grade is one of the key performance indicators that can help educators monitor their academic performance. During the past decade, researchers have proposed many variants of machine learning techniques in education domains. However, there are severe challenges in handling imbalanced datasets for enhancing the performance of predicting student grades. Therefore, this paper presents a comprehensive analysis of machine learning techniques to predict the final student grades in the first semester courses by improving the performance of predictive accuracy. Two modules will be highlighted in this paper. First, we compare the accuracy performance of six well-known machine learning techniques namely Decision Tree (J48), Support Vector Machine (SVM), Naive Bayes (NB), K-Nearest Neighbor (kNN), Logistic Regression (LR) and Random Forest (RF) using 1282 real student’s course grade dataset. Second, we proposed a multiclass prediction model to reduce the overfitting and misclassification results caused by imbalanced multi-classification based on oversampling Synthetic Minority Oversampling Technique (SMOTE) with two features selection methods. The obtained results show that the proposed model integrates with RF give significant improvement with the highest f-measure of 99.5%. This proposed model indicates the comparable and promising results that can enhance the prediction performance model for imbalanced multi-classification for student grade prediction.

/pdf/multiclass-prediction-model-for-student-grade-prediction-uonni0lhkr.pdf

Multiclass Prediction Model for Student Grade Prediction Using Machine Learning

Intelligent diagnosis is an important manner for mechanical fault diagnosis in the era of industrial big data, and deep network has received extensive attention in this field because of automatically learning features and classifying entered samples. As a classic deep learning model, Convolutional Neural Network has been applied in mechanical intelligent fault diagnosis. However, the limitation is that entered samples must be balanced to achieve satisfactory recognition rate. During the operation of machinery, the normal samples are abundant and the fault samples are rare. Therefore, the recognition rate of the minority category is minor when processing the imbalanced data with Convolutional Neural Network. To solve the above problem, an intelligent classification method for imbalanced mechanical data based on Deep Cost Adaptive Convolutional Network is proposed. According to this model, first, it learns intrinsic state characteristics in mechanical raw signals through multiple convolution and pooling operations. Second, it maps these characteristics to mechanical health condition by fully connected layers. Finally, the cost adaptive loss function adaptively assigns different misclassification costs for all categories and keeps updating them in training process to effectively classify the imbalanced mechanical data. The proposed method is verified by bearing data and milling cutter data with different imbalanced ratio, and compared with other methods. The experimental results show that the proposed method is robust and is able to effectively classify the imbalanced mechanical data.

/pdf/deep-cost-adaptive-convolutional-network-a-classification-5b2py9ugyu.pdf

Deep Cost Adaptive Convolutional Network: A Classification Method for Imbalanced Mechanical Data

The tragic pandemic of COVID-19, due to the Severe Acute Respiratory Syndrome coronavirus-2 or SARS-CoV-2, has shaken the entire world, and has significantly disrupted healthcare systems in many countries. Because of the existing challenges and controversies to testing for COVID-19, improved and cost-effective methods are needed to detect the disease. For this purpose, machine learning (ML) has emerged as a strong forecasting method for detecting COVID-19 from chest X-ray images. In this paper, we used a Deep Learning Method (DLM) to detect COVID-19 using chest X-ray (CXR) images. Radiographic images are readily available and can be used effectively for COVID-19 detection compared to other expensive and time-consuming pathological tests. We used a dataset of 10,040 samples, of which 2143 had COVID-19, 3674 had pneumonia (but not COVID-19), and 4223 were normal (not COVID-19 or pneumonia). Our model had a detection accuracy of 96.43% and a sensitivity of 93.68%. The area under the ROC curve was 99% for COVID-19, 97% for pneumonia (but not COVID-19 positive), and 98% for normal cases. In conclusion, ML approaches may be used for rapid analysis of CXR images and thus enable radiologists to filter potential candidates in a time-effective manner to detect COVID-19.

/pdf/an-efficient-deep-learning-model-to-detect-covid-19-using-16xzvsz8.pdf

An Efficient Deep Learning Model to Detect COVID-19 Using Chest X-ray Images

Recently, imbalanced data classification has received much attention due to its wide applications. In the literature, existing researches have attempted to improve the classification performance by considering various factors such as the imbalanced distribution, cost-sensitive learning, data space improvement, and ensemble learning. Nevertheless, most of the existing methods focus on only part of these main aspects/factors. In this work, we propose a novel imbalanced data classification model that considers all these main aspects. To evaluate the performance of our proposed model, we have conducted experiments based on 14 public datasets. The results show that our model outperforms the state-of-the-art methods in terms of recall, G-mean, F-measure and AUC.

/pdf/a-novel-model-for-imbalanced-data-classification-1d4nbun773.pdf

A Novel Model for Imbalanced Data Classification.

Classification is one of the most important tasks in machine learning with a huge number of real-life applications. In many practical classification problems, the available information for making object classification is partial or incomplete because some attribute values can be missing due to various reasons. These missing values can significantly affect the efficacy of the classification model. So it is crucial to develop effective techniques to impute these missing values. A number of methods have been introduced for solving classification problem with missing values. However they have various problems. So, we introduce an effective method for imputing missing values using the correlation among the attributes. Other methods which consider correlation for imputing missing values works better either for categorical or numeric data, or designed for a particular application only. Moreover they will not work if all the records have at least one missing attribute. Our method, Model based Missing value Imputation using Correlation (MMIC), can effectively impute both categorical and numeric data. It uses an effective model based technique for filling the missing values attribute wise and reusing then effectively using the model. Extensive performance analyzes show that our proposed approach achieves high performance in imputing missing values and thus increases the efficacy of the classifier. The experimental results also show that our method outperforms various existing methods for handling missing data in classification.

An effective method for classification with missing values

In machine learning, classification and regression are two of the most noteworthy key topics since they occur extensively in numerous real-world applications. However, real life data is hardly ever found balanced, rather skewed data is the common occurrence. This poses some serious challenges to the standard techniques of classification and regression. The performance and effectiveness of these techniques are substantially affected by overfitting, creating a bias towards the majority class. In recent years, quite a few number of methods have been introduced for classification of imbalanced data. But most of them are designed for binary classes and it is difficult or inefficient to extend them for multiple classes. Moreover, data imbalance problem occurs frequently in regression analysis too, with only a handful of algorithms robust enough to tackle to this problem. In this paper, we propose an effective recursive method for multi-class classification with imbalanced data. Our proposed algorithm partitions and balances the data, and is applied recursively coupled with ensemble techniques. Furthermore, we also extend our proposed method to solve the data imbalance problem in regression analysis. Experimental results demonstrate that the proposed recursive technique is effective and improves the performance when compared to existing methods for classification and regression with imbalanced distribution.

An Effective Recursive Technique for Multi-Class Classification and Regression for Imbalanced Data

In the field of Data Mining, classification and regression plays a vital role as they are useful in various real-life domains. Most of the real-life data suffer from data imbalance problem. The performances of the standard algorithms are hindered for the data imbalance problem. A number of methods have been introduced for imbalance data classification. However, most of them are designed for binary class imbalance problems. Furthermore, they suffer from various problems like loss of useful information, likelihood of overfitting, unexpected mistakes etc. On the other hand, data imbalance problem exists in regression analysis also, although very few existing methods consider this problem. Hence, we propose an effective recursive based ensemble method for multi-class imbalance data classification. We also extend our method to propose an effective recursive based method for solving the data imbalance problem in regression. Extensive performance analyses show that our proposed approach achieves high performance in multi-class classification on class imbalance data and regression analysis on skewed or imbalance data. The experimental results also show that our method outperforms various existing methods for imbalance classification and regression.

An Effective Ensemble Method for Multi-class Classification and Regression for Imbalanced Data

Traffic Congestion (TC) is a crucial problem in many urban areas, leading to extreme delays in daily lives. Intelligent traffic light control is a solution that dynamically balances the traffic duration based on different road traffic flows. Deep reinforcement learning (Deep RL) is a prevalent method to adapt traffic signal control in active environments nowadays. Many researchers recently developed deep RL-based intelligent and adaptive traffic signal control systems based on different states (e.g., speed and the position of vehicles) in a particular environment. However, deep RL has not employed the control of the traffic signal by observing traffic flow in any busy road. In this paper, we proposed a deep Q-network (DQN) method based on different traffic flow information to control the traffic signal dynamically. The simulation results show improved results in terms of the average delays of the vehicles.

A Study on Deep Reinforcement Learning Based Traffic Signal Control for Mitigating Traffic Congestion

In today’s networked world, graph data is becoming increasingly more ubiquitous as the complexities, layers and hierarchies of real life data demand to be represented in a structured manner. The goal of graph mining is to extract frequent and interesting subgraph from large graph databases. It is even more significant to perform correlation analysis within these frequent subgraphs, as such relations may provide us with valuable information. However, unfortunately much work has not been done in this field even though its necessity is enormous. In this paper, we propose two measures that help us discover such correlation among frequent subgraphs. Our measures are based on the observation that elements in graphs exhibit the tendency to occur both connected and disconnected. Evaluation results show the effectiveness and practicality of our measures in real life datasets.

Tahira Alam

Papers

An effective method for classification with missing values

An Effective Recursive Technique for Multi-Class Classification and Regression for Imbalanced Data

An Effective Ensemble Method for Multi-class Classification and Regression for Imbalanced Data

A Study on Deep Reinforcement Learning Based Traffic Signal Control for Mitigating Traffic Congestion

Discovering Correlation in Frequent Subgraphs