Label noise is an important issue in classification, with many potential negative consequences. For example, the accuracy of predictions may decrease, whereas the complexity of inferred models and the number of necessary training samples may increase. Many works in the literature have been devoted to the study of label noise and the development of techniques to deal with label noise. However, the field lacks a comprehensive survey on the different types of label noise, their consequences and the algorithms that consider label noise. This paper proposes to fill this gap. First, the definitions and sources of label noise are considered and a taxonomy of the types of label noise is proposed. Second, the potential consequences of label noise are discussed. Third, label noise-robust, label noise cleansing, and label noise-tolerant algorithms are reviewed. For each category of approaches, a short discussion is proposed to help the practitioner to choose the most suitable technique in its own particular field of application. Eventually, the design of experiments is also discussed, what may interest the researchers who would like to test their own algorithms. In this paper, label noise consists of mislabeled instances: no additional information is assumed to be available like e.g., confidences on labels.

https://romisatriawahono.net/lecture/rm/survey/machine%20learning/Frenay%20-%20Classification%20in%20the%20Presence%20of%20Label%20Noise%20-%202014.pdf

Classification in the Presence of Label Noise: A Survey

An attack graph is a succinct representation of all paths through a system that end in a state where an intruder has successfully achieved his goal. Today Red Teams determine the vulnerability of networked systems by drawing gigantic attack graphs by hand. Constructing attack graphs by hand is tedious, error-prone, and impractical for large systems. By viewing an attack as a violation of a safety property, we can use off-the-shelf model checking technology to produce attack graphs automatically: a successful path from the intruder's viewpoint is a counterexample produced by the model checker In this paper we present an algorithm for generating attack graphs using model checking as a subroutine. Security analysts use attack graphs for detection, defense and forensics. In this paper we present a minimization analysis technique that allows analysts to decide which minimal set of security measures would guarantee the safety of the system. We provide a formal characterization of this problem: we prove that it is polynomially equivalent to the minimum hitting set problem and we present a greedy algorithm with provable bounds. We also present a reliability analysis technique that allows analysts to perform a simple cost-benefit trade-off depending on the likelihoods of attacks. By interpreting attack graphs as Markov Decision Processes we can use the value iteration algorithm to compute the probabilities of intruder success for each attack the graph.

Two Formal Analyses of Attack Graphs

Because learning an optimal Bayesian network classifier is an NP-hard problem, learning-improved naive Bayes has attracted much attention from researchers. In this paper, we summarize the existing improved algorithms and propose a novel Bayes model: hidden naive Bayes (HNB). In HNB, a hidden parent is created for each attribute which combines the influences from all other attributes. We experimentally test HNB in terms of classification accuracy, using the 36 UCI data sets selected by Weka, and compare it to naive Bayes (NB), selective Bayesian classifiers (SBC), naive Bayes tree (NBTree), tree-augmented naive Bayes (TAN), and averaged one-dependence estimators (AODE). The experimental results show that HNB significantly outperforms NB, SBC, NBTree, TAN, and AODE. In many data mining applications, an accurate class probability estimation and ranking are also desirable. We study the class probability estimation and ranking performance, measured by conditional log likelihood (CLL) and the area under the ROC curve (AUC), respectively, of naive Bayes and its improved models, such as SBC, NBTree, TAN, and AODE, and then compare HNB to them in terms of CLL and AUC. Our experiments show that HNB also significantly outperforms all of them.

A Novel Bayes Model: Hidden Naive Bayes

Deep learning with noisy labels: exploring techniques and remedies in medical image analysis

In the last two decades, there has been significant advancement in heuristics for inducing Bayesian belief networks for the purpose of automatic distillation of knowledge from masses of data with target concepts. However, there are various circumstances where we are confronted to fix a set of most influencing variables in modelling of class variable. This arises in provision of confidence measures on set of variables used in the structure learning of data. In this study, we have tweaked empirical as well as theoretical aspects of various feature selection evaluators, their corresponding searching methods under six well known scoring functions in K2 which is a notable structure learning technique in Bayesian belief network. We have come up with some useful findings for overall computationally efficient approach among eleven evaluators. This analysis is useful in inducing better structure from the given dataset in imparting improved performance metric for Bayesian belief network.

A review of feature selection techniques in structure learning

In order to form a good generalization from a set of training instances, a clean training dataset is important. Unfortunately, real world data is never as perfect as we would like it to be and can often suffered from corruptions. In this paper, a new approach is proposed to identify and correct mislabeled training instances. For a given instance, we employ a Bayesian classifier to evaluate the probabilities of the instance belonging to all considered class labels. Then information entropy calculated from the probability distributions is used to evaluate the typicality of the instance belonging to considered class labels. Finally, the instance with low entropy, but with error prediction result, would be identified as mislabeled instance. Experimental results indicate that our approach gains comparative or better performance than previous techniques.

Identifying and Correcting Mislabeled Training Instances

Most cyber-attacks are not single attack actions. They are multi-step attacks composed by a set of attack actions. Although techniques used by attackers can be diverse, attack patterns are generally finite. So we need to find attack steps that are correlated in an attack scenario. By studying the patterns of multi-step cyber attacks, an algorithm is presented for correlating multi-step cyber attacks and constructing attack scenario system based on modeling multi-step cyber attacks. When alerts appear, the algorithm turns them into corresponding attack models based on the knowledge base and correlates them, whether alert or not is based on the weighted cost in the attack path graph and the attack degree of the corresponding host. And attack scenarios can be constructed by correlating the attack path graphs. Moreover, the model can detect intrusion alerts in real time and revise the attack scenarios. Experiments on the DARPA IDS test dataset show the validity of the algorithm.

Correlating Multi-Step Attack and Constructing Attack Scenarios Based on Attack Pattern Modeling

In our daily life, short texts have been everywhere especially since the emergence of social network. There are countless short texts in online media like twitter, online Q&A sites and so on. Discovering topics is quite valuable in various application domains such as content recommendation and text characterization. Traditional topic models like LDA are widely applied for sorts of tasks, but when it comes to short text scenario, these models may get stuck due to the lack of words. Recently, a popular model named BTM uses word cooccurrence relationship to solve the sparsity problem and is proved effectively. However, both BTM and extended models ignore the inside relationship between words. From our perspectives, more related words should appear in the same topic. Based on this idea, we propose a model named RIBSTM which makes use of RNN for relationship learning and IDF for filtering high-frequency words. Experiments on two real-world short text datasets show great utility of our model.

Don't forget the quantifiable relationship between words: using recurrent neural network for short text topic discovery

Load balancing device is an important part of cloud platform. One of the most common applications of load balancing is to provide a single powerful virtual machine from multiple servers. In multi-core environment, the load balancing device can run multiple physically parallel load-balancing processes to increase overall performance. An important issue when operating a load-balanced service is how to send all requests in a user session consistently to the same backend server, i.e. session maintaining. Most of multiprocessing load balancing solutions use shared memory and lock when manage session. By modifying Linux kernel, we avoid using shared memory and implement a lock-free multiprocessing load balancing solution.

A Lock-Free Solution for Load Balancing in Multi-Core Environment

Inspired by recent works in Aspect-Based Sentiment Analysis (ABSA) on product reviews and faced with more complex posts on social media platforms mentioning multiple entities as well as multiple aspects, we define a novel task called Multi-Entity Aspect-Based Sentiment Analysis (ME-ABSA). This task aims at fine-grained sentiment analysis of (entity, aspect) combinations, making the well-studied ABSA task a special case of it. To address the task, we propose an innovative method that models Context memory, Entity memory and Aspect memory, called CEA method. Our experimental results show that our CEA method achieves a significant gain over several baselines, including the state-of-the-art method for the ABSA task, and their enhanced versions, on datasets for ME-ABSA and ABSA tasks. The in-depth analysis illustrates the significant advantage of the CEA method over baseline methods for several hard-to-predict post types. Furthermore, we show that the CEA method is capable of generalizing to new (entity, aspect) combinations with little loss of accuracy. This observation indicates that data annotation in real applications can be largely simplified.

Chongjun Wang

Papers

Identifying and Correcting Mislabeled Training Instances

Correlating Multi-Step Attack and Constructing Attack Scenarios Based on Attack Pattern Modeling

Don't forget the quantifiable relationship between words: using recurrent neural network for short text topic discovery

A Lock-Free Solution for Load Balancing in Multi-Core Environment

Multi-Entity Aspect-Based Sentiment Analysis With Context, Entity and Aspect Memory