Showing papers in "Expert Systems With Applications in 2017"

Learning from class-imbalanced data

Abstract: 527 articles related to imbalanced data and rare events are reviewed.Viewing reviewed papers from both technical and practical perspectives.Summarizing existing methods and corresponding statistics by a new taxonomy idea.Categorizing 162 application papers into 13 domains and giving introduction.Some opening questions are discussed at the end of this manuscript. Rare events, especially those that could potentially negatively impact society, often require humans decision-making responses. Detecting rare events can be viewed as a prediction task in data mining and machine learning communities. As these events are rarely observed in daily life, the prediction task suffers from a lack of balanced data. In this paper, we provide an in depth review of rare event detection from an imbalanced learning perspective. Five hundred and seventeen related papers that have been published in the past decade were collected for the study. The initial statistics suggested that rare events detection and imbalanced learning are concerned across a wide range of research areas from management science to engineering. We reviewed all collected papers from both a technical and a practical point of view. Modeling methods discussed include techniques such as data preprocessing, classification algorithms and model evaluation. For applications, we first provide a comprehensive taxonomy of the existing application domains of imbalanced learning, and then we detail the applications for each category. Finally, some suggestions from the reviewed papers are incorporated with our experiences and judgments to offer further research directions for the imbalanced learning and rare event detection fields.

Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN

Abstract: A divide-and-conquer method classifying sentence types before sentiment analysis.Classifying sentence types by the number of opinion targets a sentence contain.A data-driven approach automatically extract features from input sentences. Different types of sentences express sentiment in very different ways. Traditional sentence-level sentiment classification research focuses on one-technique-fits-all solution or only centers on one special type of sentences. In this paper, we propose a divide-and-conquer approach which first classifies sentences into different types, then performs sentiment analysis separately on sentences from each type. Specifically, we find that sentences tend to be more complex if they contain more sentiment targets. Thus, we propose to first apply a neural network based sequence model to classify opinionated sentences into three types according to the number of targets appeared in a sentence. Each group of sentences is then fed into a one-dimensional convolutional neural network separately for sentiment classification. Our approach has been evaluated on four sentiment classification datasets and compared with a wide range of baselines. Experimental results show that: (1) sentence type classification can improve the performance of sentence-level sentiment analysis; (2) the proposed approach achieves state-of-the-art results on several benchmarking datasets.

Collaborative filtering and deep learning based recommendation system for cold start items

Abstract: Recommender system is a specific type of intelligent systems, which exploits historical user ratings on items and/or auxiliary information to make recommendations on items to the users. It plays a critical role in a wide range of online shopping, e-commercial services and social networking applications. Collaborative filtering (CF) is the most popular approaches used for recommender systems, but it suffers from complete cold start (CCS) problem where no rating record are available and incomplete cold start (ICS) problem where only a small number of rating records are available for some new items or users in the system. In this paper, we propose two recommendation models to solve the CCS and ICS problems for new items, which are based on a framework of tightly coupled CF approach and deep learning neural network. A specific deep neural network SADE is used to extract the content features of the items. The state of the art CF model, timeSVD++, which models and utilizes temporal dynamics of user preferences and item features, is modified to take the content features into prediction of ratings for cold start items. Extensive experiments on a large Netflix rating dataset of movies are performed, which show that our proposed recommendation models largely outperform the baseline models for rating prediction of cold start items. The two proposed recommendation models are also evaluated and compared on ICS items, and a flexible scheme of model retraining and switching is proposed to deal with the transition of items from cold start to non-cold start status. The experiment results on Netflix movie recommendation show the tight coupling of CF approach and deep learning neural network is feasible and very effective for cold start item recommendation. The design is general and can be applied to many other recommender systems for online shopping and social networking applications. The solution of cold start item problem can largely improve user experience and trust of recommender systems, and effectively promote cold start items.

Deep learning networks for stock market analysis and prediction

Abstract: Deep learning networks are applied to stock market analysis and prediction.A comprehensive analysis with different data representation methods is offered.Five-minute intraday data from the Korean KOSPI stock market is used.The network applied to residuals of autoregressive model improves prediction.Covariance estimation for market structure analysis is improved with the network. We offer a systematic analysis of the use of deep learning networks for stock market analysis and prediction. Its ability to extract features from a large set of raw data without relying on prior knowledge of predictors makes deep learning potentially attractive for stock market prediction at high frequencies. Deep learning algorithms vary considerably in the choice of network structure, activation function, and other model parameters, and their performance is known to depend heavily on the method of data representation. Our study attempts to provides a comprehensive and objective assessment of both the advantages and drawbacks of deep learning algorithms for stock market analysis and prediction. Using high-frequency intraday stock returns as input data, we examine the effects of three unsupervised feature extraction methodsprincipal component analysis, autoencoder, and the restricted Boltzmann machineon the networks overall ability to predict future market behavior. Empirical results suggest that deep neural networks can extract additional information from the residuals of the autoregressive model and improve prediction performance; the same cannot be said when the autoregressive model is applied to the residuals of the network. Covariance estimation is also noticeably improved when the predictive network is applied to covariance-based market structure analysis. Our study offers practical insights and potentially useful directions for further investigation into how deep learning networks can be effectively used for stock market analysis and prediction.

A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring

Abstract: Credit scoring is an effective tool for banks to properly guide decision profitably on granting loans. Ensemble methods, which according to their structures can be divided into parallel and sequential ensembles, have been recently developed in the credit scoring domain. These methods have proven their superiority in discriminating borrowers accurately. However, among the ensemble models, little consideration has been provided to the following: (1) highlighting the hyper-parameter tuning of base learner despite being critical to well-performed ensemble models; (2) building sequential models (i.e., boosting, as most have focused on developing the same or different algorithms in parallel); and (3) focusing on the comprehensibility of models. This paper aims to propose a sequential ensemble credit scoring model based on a variant of gradient boosting machine (i.e., extreme gradient boosting (XGBoost)). The model mainly comprises three steps. First, data pre-processing is employed to scale the data and handle missing values. Second, a model-based feature selection system based on the relative feature importance scores is utilized to remove redundant variables. Third, the hyper-parameters of XGBoost are adaptively tuned with Bayesian hyper-parameter optimization and used to train the model with selected feature subset. Several hyper-parameter optimization methods and baseline classifiers are considered as reference points in the experiment. Results demonstrate that Bayesian hyper-parameter optimization performs better than random search, grid search, and manual search. Moreover, the proposed model outperforms baseline models on average over four evaluation measures: accuracy, error rate, the area under the curve (AUC) H measure (AUC-H measure), and Brier score. The proposed model also provides feature importance scores and decision chart, which enhance the interpretability of credit scoring model.

Whale Optimization Algorithm and Moth-Flame Optimization for multilevel thresholding image segmentation

Abstract: Two metaheuristic algorithms (WOA and MFO) are used.These algorithms are applied to multilevel thresholding image segmentation.MFO and WOA are better than compared algorithms.MFO is better than WOA for higher number of thresholds. Determining the optimal thresholding for image segmentation has got more attention in recent years since it has many applications. There are several methods used to find the optimal thresholding values such as Otsu and Kapur based methods. These methods are suitable for bi-level thresholding case and they can be easily extended to the multilevel case, however, the process of determining the optimal thresholds in the case of multilevel thresholding is time-consuming. To avoid this problem, this paper examines the ability of two nature inspired algorithms namely: Whale Optimization Algorithm (WOA) and Moth-Flame Optimization (MFO) to determine the optimal multilevel thresholding for image segmentation. The MFO algorithm is inspired from the natural behavior of moths which have a special navigation style at night since they fly using the moonlight, whereas, the WOA algorithm emulates the natural cooperative behaviors of whales. The candidate solutions in the adapted algorithms were created using the image histogram, and then they were updated based on the characteristics of each algorithm. The solutions are assessed using the Otsus fitness function during the optimization operation. The performance of the proposed algorithms has been evaluated using several of benchmark images and has been compared with five different swarm algorithms. The results have been analyzed based on the best fitness values, PSNR, and SSIM measures, as well as time complexity and the ANOVA test. The experimental results showed that the proposed methods outperformed the other swarm algorithms; in addition, the MFO showed better results than WOA, as well as provided a good balance between exploration and exploitation in all images at small and high threshold numbers.

Machine learning models and bankruptcy prediction

Abstract: Machine learning models show improved bankruptcy prediction accuracy over traditional models.Various models were tested using different accuracy metrics.Boosting, bagging, and random forest models provide better results. There has been intensive research from academics and practitioners regarding models for predicting bankruptcy and default events, for credit risk management. Seminal academic research has evaluated bankruptcy using traditional statistics techniques (e.g. discriminant analysis and logistic regression) and early artificial intelligence models (e.g. artificial neural networks). In this study, we test machine learning models (support vector machines, bagging, boosting, and random forest) to predict bankruptcy one year prior to the event, and compare their performance with results from discriminant analysis, logistic regression, and neural networks. We use data from 1985 to 2013 on North American firms, integrating information from the Salomon Center database and Compustat, analysing more than 10,000 firm-year observations. The key insight of the study is a substantial improvement in prediction accuracy using machine learning techniques especially when, in addition to the original Altmans Z-score variables, we include six complementary financial indicators. Based on Carton and Hofer (2006), we use new variables, such as the operating margin, change in return-on-equity, change in price-to-book, and growth measures related to assets, sales, and number of employees, as predictive variables. Machine learning models show, on average, approximately 10% more accuracy in relation to traditional models. Comparing the best models, with all predictive variables, the machine learning technique related to random forest led to 87% accuracy, whereas logistic regression and linear discriminant analysis led to 69% and 50% accuracy, respectively, in the testing sample. We find that bagging, boosting, and random forest models outperform the others techniques, and that all prediction accuracy in the testing sample improves when the additional variables are included. Our research adds to the discussion of the continuing debate about superiority of computational methods over statistical techniques such as in Tsai, Hsu, and Yen (2014) and Yeh, Chi, and Lin (2014). In particular, for machine learning mechanisms, we do not find SVM to lead to higher accuracy rates than other models. This result contradicts outcomes from Danenas and Garsva (2015) and Cleofas-Sanchez, Garcia, Marques, and Senchez (2016), but corroborates, for instance, Wang, Ma, and Yang (2014), Liang, Lu, Tsai, and Shih (2016), and Cano etal. (2017). Our study supports the applicability of the expert systems by practitioners as in Heo and Yang (2014), Kim, Kang, and Kim (2015) and Xiao, Xiao, and Wang (2016).

Enhancing deep learning sentiment analysis with ensemble techniques in social applications

Abstract: A taxonomy that classifies ensemble models in the literature is presented.Surface and deep features integration is explored to improve classification.Several ensembles of classifiers and features are proposed and evaluated.Performance of the proposed models is evaluated on several sentiment datasets. Deep learning techniques for Sentiment Analysis have become very popular. They provide automatic feature extraction and both richer representation capabilities and better performance than traditional feature based techniques (i.e., surface methods). Traditional surface approaches are based on complex manually extracted features, and this extraction process is a fundamental question in feature driven methods. These long-established approaches can yield strong baselines, and their predictive capabilities can be used in conjunction with the arising deep learning methods. In this paper we seek to improve the performance of deep learning techniques integrating them with traditional surface approaches based on manually extracted features. The contributions of this paper are sixfold. First, we develop a deep learning based sentiment classifier using a word embeddings model and a linear machine learning algorithm. This classifier serves as a baseline to compare to subsequent results. Second, we propose two ensemble techniques which aggregate our baseline classifier with other surface classifiers widely used in Sentiment Analysis. Third, we also propose two models for combining both surface and deep features to merge information from several sources. Fourth, we introduce a taxonomy for classifying the different models found in the literature, as well as the ones we propose. Fifth, we conduct several experiments to compare the performance of these models with the deep learning baseline. For this, we use seven public datasets that were extracted from the microblogging and movie reviews domain. Finally, as a result, a statistical study confirms that the performance of these proposed models surpasses that of our original baseline on F1-Score.

Multi-level hybrid support vector machine and extreme learning machine based on modified K-means for intrusion detection system

Abstract: Reduction the 10%KDD training dataset up to 99.8% by using modified K-means.New high quality training datasets are constructed for training SVM and ELM.Multi-level model is proposed to improve the performance of detection accuracy.Improve the detection rate of DoS, U2R and R2L attacks.Overall accuracy of 95.75% is achieved with whole Corrected KDD dataset. Intrusion detection has become essential to network security because of the increasing connectivity between computers. Several intrusion detection systems have been developed to protect networks using different statistical methods and machine learning techniques. This study aims to design a model that deals with real intrusion detection problems in data analysis and classify network data into normal and abnormal behaviors. This study proposes a multi-level hybrid intrusion detection model that uses support vector machine and extreme learning machine to improve the efficiency of detecting known and unknown attacks. A modified K-means algorithm is also proposed to build a high-quality training dataset that contributes significantly to improving the performance of classifiers. The modified K-means is used to build new small training datasets representing the entire original training dataset, significantly reduce the training time of classifiers, and improve the performance of intrusion detection system. The popular KDD Cup 1999 dataset is used to evaluate the proposed model. Compared with other methods based on the same dataset, the proposed model shows high efficiency in attack detection, and its accuracy (95.75%) is the best performance thus far.

An up-to-date comparison of state-of-the-art classification algorithms

Abstract: Up-to-date report on the accuracy and efficiency of state-of-the-art classifiers.We compare the accuracy of 11 classification algorithms pairwise and groupwise.We examine separately the training, parameter-tuning, and testing time.GBDT and Random Forests yield highest accuracy, outperforming SVM.GBDT is the fastest in testing, Naive Bayes the fastest in training. Current benchmark reports of classification algorithms generally concern common classifiers and their variants but do not include many algorithms that have been introduced in recent years. Moreover, important properties such as the dependency on number of classes and features and CPU running time are typically not examined. In this paper, we carry out a comparative empirical study on both established classifiers and more recently proposed ones on 71 data sets originating from different domains, publicly available at UCI and KEEL repositories. The list of 11 algorithms studied includes Extreme Learning Machine (ELM), Sparse Representation based Classification (SRC), and Deep Learning (DL), which have not been thoroughly investigated in existing comparative studies. It is found that Stochastic Gradient Boosting Trees (GBDT) matches or exceeds the prediction performance of Support Vector Machines (SVM) and Random Forests (RF), while being the fastest algorithm in terms of prediction efficiency. ELM also yields good accuracy results, ranking in the top-5, alongside GBDT, RF, SVM, and C4.5 but this performance varies widely across all data sets. Unsurprisingly, top accuracy performers have average or slow training time efficiency. DL is the worst performer in terms of accuracy but second fastest in prediction efficiency. SRC shows good accuracy performance but it is the slowest classifier in both training and testing.

An improved Opposition-Based Sine Cosine Algorithm for global optimization

Abstract: A new method to solve global optimization and engineering problems called OBSCA.The proposed method improves the SCA by using opposite-based learning.We apply the OBSCA over mathematical benchmark functions.We test OBSCA in engineering optimization problems.Comparisons support the improvement on the performance of OBCSA. Real life optimization problems require techniques that properly explore the search spaces to obtain the best solutions. In this sense, it is common that traditional optimization algorithms fail in local optimal values. The Sine Cosine Algorithms (SCA) has been recently proposed; it is a global optimization approach based on two trigonometric functions. SCA uses the sine and cosine functions to modify a set of candidate solutions; such operators create a balance between exploration and exploitation of the search space. However, like other similar approaches, SCA tends to be stuck into sub-optimal regions that it is reflected in the computational effort required to find the best values. This situation occurs due that the operators used for exploration do not work well to analyze the search space. This paper presents an improved version of SCA that considers the opposition based learning (OBL) as a mechanism for a better exploration of the search space generating more accurate solutions. OBL is a machine learning strategy commonly used to increase the performance of metaheuristic algorithms. OBL considers the opposite position of a solution in the search space. Based on the objective function value, the OBL selects the best element between the original solution and its opposite position; this task increases the accuracy of the optimization process. The hybridization of concepts from different fields is crucial in intelligent and expert systems; it helps to combine the advantages of algorithms to generate more efficient approaches. The proposed method is an example of this combination; it has been tested over several benchmark functions and engineering problems. Such results support the efficacy of the proposed approach to find the optimal solutions in complex search spaces.

A bibliometric-based survey on AHP and TOPSIS techniques

Abstract: A bibliometric based survey on AHP and TOPSIS methods has been conducted.Scopus database was employed to retrieve the required data for this analysis.Assessment of quantitative and qualitative bibliometric indicators was obtainable.Efficacy of these methods promotes the development of related research.More scientific research interests will be devoted to these methods in the future. In recent years, the employment of multiple criteria decision analysis (MCDA) techniques in solving complex real-world problems has increased exponentially. The willingness to build advanced decision models, with higher capabilities to support decision making in a wide range of applications, promotes the integration of MCDA techniques with efficient systems such as intelligence and expert systems, geographic information systems, etc. Amongst the most applied MCDA techniques are Analytic Hierarchy Process (AHP) and Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS). The development of a comprehensive perspective on research activities associated with the applications of these methods provides insights into the contributions of countries, institutes, authors and journals towards the advancements of these methods. Furthermore, it helps in identifying the status and trends of research. This in turn will help researchers in shaping up and improving future research activities and investments. To meet these aims, a bibliometric analysis based on data harvested from Scopus database was carried out to identify a set of bibliometric performance indicators (i.e. quantitative indicators such as productivity, and qualitative indicators such as citations and Hirsch index (h-index)). Additionally, bibliometric visualization maps were employed to identify the hot spots of research. The total research output was 10,188 documents for AHP and 2412 documents for TOPSIS. China took a leading position in AHP research (3513 documents; 34.5%). It was also the leading country in TOPSIS research (846 documents; 35.1%). The most collaborated country in AHP research was the United States, while in case of TOPSIS it was China. The United States had gained the highest h-index (78) in AHP research, while in TOPSIS it was Taiwan with h-index of 46. Expert Systems with Applications journal was the most productive journal in AHP (204; 2.0%) and TOPSIS research (125; 5.2%), simultaneously. University of Tehran, Iran and Islamic Azad University, Iran were the most productive institutions in AHP (173; 1.7%) and TOPSIS (115; 4.8%) research, simultaneously. The major hot topics that utilized AHP and will continue to be active include different applications of geographic information systems, risk modeling and supply chain management. While for TOPSIS, they are supply chain management and sustainability research. Overall, this analysis has shown increasing recognition of powerful of MCDA techniques to support strategic decisions. The efficacy of these methods in the previous context promotes their progress and advancements.

An integrated decision support system based on ANN and Fuzzy_AHP for heart failure risk prediction

Abstract: This study proposed a hybrid decision support method (ANN and Fuzzy_AHP) for heart failure prediction.The performance of the proposed method was examined using three performance metrics.From the evaluations results, the proposed method performed better than the conventional ANN approachThe proposed method would provide improved and realistic result for efficient therapy administration. Heart failure (HF) has been considered as one of the deadliest human diseases worldwide and the accurate prediction of HF risks would be vital for HF prevention and treatment. To predict HF risks, decision support systems based on artificial neural networks (ANN) have been widely proposed in previous studies. Generally, these existing ANN-based systems usually assumed that HF attributes have equal risk contribution to the HF diagnosis. However, several previous investigations have shown that the risk contributions of the attributes would be different. Thus the equal risk assumption concept associated with existing ANN methods would not properly reflect the diagnosis status of HF patients. In this study, the commonly used 13 HF attributes were considered and their contributions were determined by an experienced cardiac clinician. And Fuzzy analytic hierarchy process (Fuzzy_AHP) technique was used to compute the global weights for the attributes based on their individual contribution. Then the global weights that represent the contributions of the attributes were applied to train an ANN classifier for the prediction of HF risks in patients. The performance of the newly proposed decision support system based on the integration of ANN and Fuzzy_AHP methods was evaluated by using online clinical dataset of 297 HF patients and compared with that of the conventional ANN method. Our result shows that the proposed method could achieve an average prediction accuracy of 91.10%, which is 4.40% higher in comparison to that of the conventional ANN method. In addition, the newly proposed method also had better performance than seven previous methods that reported prediction accuracies in the range of 57.85-89.01%. The improvement of the HF risk prediction in the current study might be due to both the various contributions of the HF attributes and the proposed hybrid method. These findings suggest that the proposed method could be used to accurately predict HF risks in the clinic.

Sentiment analysis leveraging emotions and word embeddings

Abstract: Sentiment analysis and opinion mining are valuable for extraction of useful subjective information out of text documents. These tasks have become of great importance, especially for business and marketing professionals, since online posted products and services reviews impact markets and consumers shifts. This work is motivated by the fact that automating retrieval and detection of sentiments expressed for certain products and services embeds complex processes and pose research challenges, due to the textual phenomena and the language specific expression variations. This paper proposes a fast, flexible, generic methodology for sentiment detection out of textual snippets which express people’s opinions in different languages. The proposed methodology adopts a machine learning approach with which textual documents are represented by vectors and are used for training a polarity classification model. Several documents’ vector representation approaches have been studied, including lexicon-based, word embedding-based and hybrid vectorizations. The competence of these feature representations for the sentiment classification task is assessed through experiments on four datasets containing online user reviews in both Greek and English languages, in order to represent high and weak inflection language groups. The proposed methodology requires minimal computational resources, thus, it might have impact in real world scenarios where limited resources is the case.

Forecasting daily stock market return using dimensionality reduction

Abstract: A data mining procedure to forecast daily stock market return is proposed.The raw data includes 60 financial and economic features over a 10-year period.Combining ANNs with PCA gives slightly higher classification accuracy.Combining ANNs with PCA provides significantly higher risk-adjusted profits. In financial markets, it is both important and challenging to forecast the daily direction of the stock market return. Among the few studies that focus on predicting daily stock market returns, the data mining procedures utilized are either incomplete or inefficient, especially when a large amount of features are involved. This paper presents a complete and efficient data mining process to forecast the daily direction of the S&P 500 Index ETF (SPY) return based on 60 financial and economic features. Three mature dimensionality reduction techniques, including principal component analysis (PCA), fuzzy robust principal component analysis (FRPCA), and kernel-based principal component analysis (KPCA) are applied to the whole data set to simplify and rearrange the original data structure. Corresponding to different levels of the dimensionality reduction, twelve new data sets are generated from the entire cleaned data using each of the three different dimensionality reduction methods. Artificial neural networks (ANNs) are then used with the thirty-six transformed data sets for classification to forecast the daily direction of future market returns. Moreover, the three different dimensionality reduction methods are compared with respect to the natural data set. A group of hypothesis tests are then performed over the classification and simulation results to show that combining the ANNs with the PCA gives slightly higher classification accuracy than the other two combinations, and that the trading strategies guided by the comprehensive classification mining procedures based on PCA and ANNs gain significantly higher risk-adjusted profits than the comparison benchmarks, while also being slightly higher than those strategies guided by the forecasts based on the FRPCA and KPCA models.

Deep learning for decentralized parking lot occupancy detection

Abstract: We propose an effective CNN architecture for visual parking occupancy detectionThe CNN architecture is small enough to run on smart camerasThe proposed solution performs and generalizes better than other SotA approachesWe provide a new training/validation dataset for parking occupancy detection A smart camera is a vision system capable of extracting application-specific information from the captured images The paper proposes a decentralized and efficient solution for visual parking lot occupancy detection based on a deep Convolutional Neural Network (CNN) specifically designed for smart cameras This solution is compared with state-of-the-art approaches using two visual datasets: PKLot, already existing in literature, and CNRPark-EXT The former is an existing dataset, that allowed us to exhaustively compare with previous works The latter dataset has been created in the context of this research, accumulating data across various seasons of the year, to test our approach in particularly challenging situations, exhibiting occlusions, and diverse and difficult viewpoints This dataset is public available to the scientific community and is another contribution of our research Our experiments show that our solution outperforms and generalizes the best performing approaches on both datasets The performance of our proposed CNN architecture on the parking lot occupancy detection task, is comparable to the well-known AlexNet, which is three orders of magnitude larger

An overview of fault tree analysis and its application in model based dependability analysis

Abstract: I provide an overview of the Fault Tree Analysis method.I review different extensions of fault trees.A number of model-based dependability analysis approaches are reviewed.I outline the future outlook for model-based dependability analysis. Fault Tree Analysis (FTA) is a well-established and well-understood technique, widely used for dependability evaluation of a wide range of systems. Although many extensions of fault trees have been proposed, they suffer from a variety of shortcomings. In particular, even where software tool support exists, these analyses require a lot of manual effort. Over the past two decades, research has focused on simplifying dependability analysis by looking at how we can synthesise dependability information from system models automatically. This has led to the field of model-based dependability analysis (MBDA). Different tools and techniques have been developed as part of MBDA to automate the generation of dependability analysis artefacts such as fault trees. Firstly, this paper reviews the standard fault tree with its limitations. Secondly, different extensions of standard fault trees are reviewed. Thirdly, this paper reviews a number of prominent MBDA techniques where fault trees are used as a means for system dependability analysis and provides an insight into their working mechanism, applicability, strengths and challenges. Finally, the future outlook for MBDA is outlined, which includes the prospect of developing expert and intelligent systems for dependability analysis of complex open systems under the conditions of uncertainty.

The impact of microblogging data for stock market prediction: Using Twitter to predict returns, volatility, trading volume and survey sentiment indices

Abstract: In this paper, we propose a robust methodology to assess the value of microblogging data to forecast stock market variables: returns, volatility and trading volume of diverse indices and portfolios. The methodology uses sentiment and attention indicators extracted from microblogs (a large Twitter dataset is adopted) and survey indices (AAII and II, USMC and Sentix), diverse forms to daily aggregate these indicators, usage of a Kalman Filter to merge microblog and survey sources, a realistic rolling windows evaluation, several Machine Learning methods and the Diebold-Mariano test to validate if the sentiment and attention based predictions are valuable when compared with an autoregressive baseline. We found that Twitter sentiment and posting volume were relevant for the forecasting of returns of S&P 500 index, portfolios of lower market capitalization and some industries. Additionally, KF sentiment was informative for the forecasting of returns. Moreover, Twitter and KF sentiment indicators were useful for the prediction of some survey sentiment indicators. These results confirm the usefulness of microblogging data for financial expert systems, allowing to predict stock market behavior and providing a valuable alternative for existing survey measures with advantages (e.g., fast and cheap creation, daily frequency).

Smart health

Abstract: Organized evaluation of various big data and smart system technology in healthcare context.Proposed a conceptual model on Big data enabled Smart Healthcare System Framework (BSHSF).We extract some depth information (some relevant examples) about advanced healthcare system.In depth study about state-of-the-art big data and smart healthcare system in parallel. In the era of big data, recent developments in the area of information and communication technologies (ICT) are facilitating organizations to innovate and grow. These technological developments and wide adaptation of ubiquitous computing enable numerous opportunities for government and companies to reconsider healthcare prospects. Therefore, big data and smart healthcare systems are independently attracting extensive attention from both academia and industry. The combination of both big data and smart systems can expedite the prospects of the healthcare industry. However, a thorough study of big data and smart systems together in the healthcare context is still absent from the existing literature. The key contributions of this article include an organized evaluation of various big data and smart system technologies and a critical analysis of the state-of-the-art advanced healthcare systems. We describe the three-dimensional structure of a paradigm shift. We also extract three broad technical branches (3T) contributing to the promotion of healthcare systems. More specifically, we propose a big data enabled smart healthcare system framework (BSHSF) that offers theoretical representations of an intra and inter organizational business model in the healthcare context. We also mention some examples reported in the literature, and then we contribute to pinpointing the potential opportunities and challenges of applying BSHSF to healthcare business environments. We also make five recommendations for effectively applying `BSHSF to the healthcare industry. To the best of our knowledge, this is the first in-depth study about state-of-the-art big data and smart healthcare systems in parallel. The managerial implication of this article is that organizations can use the findings of our critical analysis to reinforce their strategic arrangement of smart systems and big data in the healthcare context, and hence better leverage them for sustainable organizational invention.

A feature weighted support vector machine and K-nearest neighbor algorithm for stock market indices prediction

Abstract: This study investigates stock market indices prediction that is an interesting and important research in the areas of investment and applications, as it can get more profits and returns at lower risk rate with effective exchange strategies. To realize accurate prediction, various methods have been tried, among which the machine learning methods have drawn attention and been developed. In this paper, we propose a basic hybridized framework of the feature weighted support vector machine as well as feature weighted K-nearest neighbor to effectively predict stock market indices. We first establish a detailed theory of feature weighted SVM for the data classification assigning different weights for different features with respect to the classification importance. Then, to get the weights, we estimate the importance of each feature by computing the information gain. Lastly, we use feature weighted K-nearest neighbor to predict future stock market indices by computing k weighted nearest neighbors from the historical dataset. Experiment results on two well known Chinese stock market indices like Shanghai and Shenzhen stock exchange indices are finally presented to test the performance of our established model. With our proposed model, it can achieve a better prediction capability to Shanghai Stock Exchange Composite Index and Shenzhen Stock Exchange Component Index in the short, medium and long term respectively. The proposed algorithm can also be adapted to other stock market indices prediction.

Multilevel thresholding using grey wolf optimizer for image segmentation

Abstract: Multilevel thresholding is one of the most important areas in the field of image segmentation. However, the computational complexity of multilevel thresholding increases exponentially with the increasing number of thresholds. To overcome this drawback, a new approach of multilevel thresholding based on Grey Wolf Optimizer (GWO) is proposed in this paper. GWO is inspired from the social and hunting behaviour of the grey wolves. This metaheuristic algorithm is applied to multilevel thresholding problem using Kapur's entropy and Otsu's between class variance functions. The proposed method is tested on a set of standard test images. The performances of the proposed method are then compared with improved versions of PSO (Particle Swarm Optimization) and BFO (Bacterial Foraging Optimization) based multilevel thresholding methods. The quality of the segmented images is computed using Mean Structural SIMilarity (MSSIM) index. Experimental results suggest that the proposed method is more stable and yields solutions of higher quality than PSO and BFO based methods. Moreover, the proposed method is found to be faster than BFO but slower than the PSO based method.

A comparative study of machine learning classifiers for modeling travel mode choice

Abstract: A comparison of 7 classifiers for travel mode prediction is performed.Prediction accuracy and variable importance for each travel mode is investigated.Among the investigated classifiers, random forest performs best.Trip distance followed by the number of cars are the most important variables.The importance of other variables varies with travel mode and classifier. The analysis of travel mode choice is an important task in transportation planning and policy making in order to understand and predict travel demands. While advances in machine learning have led to numerous powerful classifiers, their usefulness for modeling travel mode choice remains largely unexplored. Using extensive Dutch travel diary data from the years 2010 to 2012, enriched with variables on the built and natural environment as well as on weather conditions, this study compares the predictive performance of seven selected machine learning classifiers for travel mode choice analysis and makes recommendations for model selection. In addition, it addresses the importance of different variables and how they relate to different travel modes. The results show that random forest performs significantly better than any other of the investigated classifiers, including the commonly used multinomial logit model. While trip distance is found to be the most important variable, the importance of the other variables varies with classifiers and travel modes. The importance of the meteorological variables is highest for support vector machine, while temperature is particularly important for predicting bicycle and public transport trips. The results suggest that the analysis of variable importance with respect to the different classifiers and travel modes is essential for a better understanding and effective modeling of peoples travel behavior.

Genetic algorithm based model for optimizing bank lending decisions

Abstract: The bank lending decisions in credit crunch environments are big challenge.This NP-hard optimization problem is solved using a proposed GA based model.The proposed model is tested using two scenarios with simulated and real data.The real data is collected from Southern Louisiana Credit Union.The proposed model increased the bank profit and improved the system performance. To avoid the complexity and time consumption of traditional statistical and mathematical programming, intelligent techniques have gained great attention in different financial research areas, especially in banking decisions optimization. However, choosing optimum bank lending decisions that maximize the bank profit in a credit crunch environment is still a big challenge. For that, this paper proposes an intelligent model based on the Genetic Algorithm (GA) to organize bank lending decisions in a highly competitive environment with a credit crunch constraint (GAMCC). GAMCC provides a framework to optimize bank objectives when constructing the loan portfolio, by maximizing the bank profit and minimizing the probability of bank default in a search for a dynamic lending decision. Compared to the state-of-the art methods, GAMCC is considered a better intelligent tool that enables banks to reduce the loan screening time by a range of 12%50%. Moreover, it greatly increases the bank profit by a range of 3.9%8.1%.

New directional bat algorithm for continuous optimization problems

Abstract: Bat algorithm (BA) is a recent optimization algorithm based on swarm intelligence and inspiration from the echolocation behavior of bats. One of the issues in the standard bat algorithm is the premature convergence that can occur due to the low exploration ability of the algorithm under some conditions. To overcome this deficiency, directional echolocation is introduced to the standard bat algorithm to enhance its exploration and exploitation capabilities. In addition to such directional echolocation, three other improvements have been embedded into the standard bat algorithm to enhance its performance. The new proposed approach, namely the directional Bat Algorithm (dBA), has been then tested using several standard and non-standard benchmarks from the CEC’2005 benchmark suite. The performance of dBA has been compared with ten other algorithms and BA variants using non-parametric statistical tests. The statistical test results show the superiority of the directional bat algorithm.

A feature reduced intrusion detection system using ANN classifier

Abstract: A Feature reduced Intrusion Detection System has been proposed.Pre-processing is done to compensate less occurring and frequent occurring attacks.Feature reduction has been done on basis of information gain and correlation.A classifier based on artificial neural network has been used.Comparison with state of art methods has been done. Rapid increase in internet and network technologies has led to considerable increase in number of attacks and intrusions. Detection and prevention of these attacks has become an important part of security. Intrusion detection system is one of the important ways to achieve high security in computer networks and used to thwart different attacks. Intrusion detection systems have curse of dimensionality which tends to increase time complexity and decrease resource utilization. As a result, it is desirable that important features of data must be analyzed by intrusion detection system to reduce dimensionality. This work proposes an intelligent system which first performs feature ranking on the basis of information gain and correlation. Feature reduction is then done by combining ranks obtained from both information gain and correlation using a novel approach to identify useful and useless features. These reduced features are then fed to a feed forward neural network for training and testing on KDD99 dataset. Pre-processing of KDD-99 dataset has been done to normalize number of instances of each class before training. The system then behaves intelligently to classify test data into attack and non-attack classes. The aim of the feature reduced system is to achieve same degree of performance as a normal system. The system is tested on five different test datasets and both individual and average results of all datasets are reported. Comparison of proposed method with and without feature reduction is done in terms of various performance metrics. Comparisons with recent and relevant approaches are also tabled. Results obtained for proposed method are really encouraging.

Deep learning for biological image classification

Abstract: A number of industries use human inspection to visually classify the quality of their products and the raw materials used in the production process, this process could be done automatically through digital image processing. The industries are not always interested in the most accurate technique for a given problem, but most appropriate for the expected results, there must be a balance between accuracy and computational cost. This paper investigates the classification of the quality of wood boards based on their images. For such, it compares the use of deep learning, particularly Convolutional Neural Networks, with the combination of texture-based feature extraction techniques and traditional techniques: Decision tree induction algorithms, Neural Networks, Nearest neighbors and Support vector machines. Reported studies show that Deep Learning techniques applied to image processing tasks have achieved predictive performance superior to traditional classification techniques, mainly in high complex scenarios. One of the reasons pointed out is their embedded feature extraction mechanism. Deep Learning techniques directly identify and extract features, considered by them to be relevant, in a given image dataset. However, empirical results for the image data set have shown that the texture descriptor method proposed, regardless of the strategy employed is very competitive when compared with Convolutional Neural Network for all the performed experiments. The best performance of the texture descriptor method could be caused by the nature of the image dataset. Finally are pointed out some perspectives of futures developments with the application of Active learning and Semi supervised methods.

Text summarization using unsupervised deep learning

Abstract: Unsupervised extractive summarization of emails using a deep auto-encoder with excellent performance.Ensemble Noisy Auto-Encoder runs noisy inputs through one trained network, enhancing performance.Summaries are highly informative and semantically similar to human abstracts. We present methods of extractive query-oriented single-document summarization using a deep auto-encoder (AE) to compute a feature space from the term-frequency (tf) input. Our experiments explore both local and global vocabularies. We investigate the effect of adding small random noise to local tf as the input representation of AE, and propose an ensemble of such noisy AEs which we call the Ensemble Noisy Auto-Encoder (ENAE). ENAE is a stochastic version of an AE that adds noise to the input text and selects the top sentences from an ensemble of noisy runs. In each individual experiment of the ensemble, a different randomly generated noise is added to the input representation. This architecture changes the application of the AE from a deterministic feed-forward network to a stochastic runtime model. Experiments show that the AE using local vocabularies clearly provide a more discriminative feature space and improves the recall on average 11.2%. The ENAE can make further improvements, particularly in selecting informative sentences. To cover a wide range of topics and structures, we perform experiments on two different publicly available email corpora that are specifically designed for text summarization. We used ROUGE as a fully automatic metric in text summarization and we presented the average ROUGE-2 recall for all experiments.

Detecting driving stress in physiological signals based on multimodal feature analysis and kernel classifiers

Abstract: Monitoring driving status has great potential in helping us decline the occurrence probability of traffic accidents and the aim of this research is to develop a novel system for driving stress detection based on multimodal feature analysis and kernel-based classifiers. Physiological signals such as electrocardiogram, galvanic skin response and respiration were record from fourteen drives executed in a prescribed route at real drive environments. Features were widely extracted from time, spectral and wavelet multi-domains. In order to search for the optimal feature sets, Sparse Bayesian Learning (SBL) and Principal Component Analysis (PCA) were combined and adopted. Kernel-based classifiers were employed to improve the accuracy of stress detection task. Analysis I used features from 10 s intervals of data which were recorded during well-defined rest, highway and city driving conditions to discriminate three levels of diving stress achieving an averaging accuracy over 99% at per-drive level and 89% in cross-drive validation. Analysis II made continuous stress evaluation throughout a complete driving test attaining a high coincidence with the true road situation especially at the switching interval of traffic conditions. Experimental results reveal that different levels of driving stress can be characterized by specific set of physiological measures. These physiological measures could be applied to in-vehicle intelligent systems in various approaches to help the drivers better manage their negative driving status. Our design scheme for driving stress detection could also facilitate the development of similar in-vehicle expert systems, such as driver's emotion management, driver's sleeping onset monitoring, and human-computer interaction (HCI).

Multi-class sentiment classification

Abstract: A framework for multi-class sentiment classification is proposed.A total of 3600 comparative experiments are conducted.Performances of different feature selection/machine learning algorithms are compared.The results are valuable for further studies on multi-class sentiment classification. Multi-class sentiment classification has extensive application backgrounds, whereas studies on this issue are still relatively scarce. In this paper, a framework for multi-class sentiment classification is proposed, which includes two parts: 1) selecting important features of texts using the feature selection algorithm, and 2) training multi-class sentiment classifier using the machine learning algorithm. Then, experiments are conducted for comparing the performances of four popular feature selection algorithms (document frequency, CHI statistics, information gain and gain ratio) and five popular machine learning algorithms (decision tree, nave Bayes, support vector machine, radial basis function neural network and K-nearest neighbor) in multi-class sentiment classification. The experiments are conducted on three public datasets which include twelve data subsets, and 10-fold cross validation is used to obtain the classification accuracy concerning each combination of feature selection algorithm, machine learning algorithm, feature set size and data subset. Based on the obtained 3600 classification accuracies (4 feature selection algorithms 5 machine learning algorithms 15 feature set sizes 12 data subsets), the average classification accuracy of each algorithm is calculated, and the Wilcoxon test is used to verify the existence of significant difference between different algorithms in multi-class sentiment classification. The results show that, in terms of classification accuracy, gain ratio performs best among the four feature selection algorithms and support vector machine performs best among the five machine learning algorithms. In terms of execution time, the similar comparisons are also conducted. The obtained results would be valuable for further improving the existing multi-class sentiment classifiers and developing new multi-class sentiment classifiers.

A comparative study on base classifiers in ensemble methods for credit scoring

Abstract: In the last years, the application of artificial intelligence methods on credit risk assessment has meant an improvement over classic methods. Small improvements in the systems about credit scoring and bankruptcy prediction can suppose great profits. Then, any improvement represents a high interest to banks and financial institutions. Recent works show that ensembles of classifiers achieve the better results for this kind of tasks. In this paper, it is extended a previous work about the selection of the best base classifier used in ensembles on credit data sets. It is shown that a very simple base classifier, based on imprecise probabilities and uncertainty measures, attains a better trade-off among some aspects of interest for this type of studies such as accuracy and area under ROC curve (AUC). The AUC measure can be considered as a more appropriate measure in this grounds, where the different type of errors have different costs or consequences. The results shown here present to this simple classifier as an interesting choice to be used as base classifier in ensembles for credit scoring and bankruptcy prediction, proving that not only the individual performance of a classifier is the key point to be selected for an ensemble scheme.