Often the best performing supervised learning models are ensembles of hundreds or thousands of base-level classifiers. Unfortunately, the space required to store this many classifiers, and the time required to execute them at run-time, prohibits their use in applications where test sets are large (e.g. Google), where storage space is at a premium (e.g. PDAs), and where computational power is limited (e.g. hea-ring aids). We present a method for "compressing" large, complex ensembles into smaller, faster models, usually without significant loss in performance.

Model compression

Label noise is an important issue in classification, with many potential negative consequences. For example, the accuracy of predictions may decrease, whereas the complexity of inferred models and the number of necessary training samples may increase. Many works in the literature have been devoted to the study of label noise and the development of techniques to deal with label noise. However, the field lacks a comprehensive survey on the different types of label noise, their consequences and the algorithms that consider label noise. This paper proposes to fill this gap. First, the definitions and sources of label noise are considered and a taxonomy of the types of label noise is proposed. Second, the potential consequences of label noise are discussed. Third, label noise-robust, label noise cleansing, and label noise-tolerant algorithms are reviewed. For each category of approaches, a short discussion is proposed to help the practitioner to choose the most suitable technique in its own particular field of application. Eventually, the design of experiments is also discussed, what may interest the researchers who would like to test their own algorithms. In this paper, label noise consists of mislabeled instances: no additional information is assumed to be available like e.g., confidences on labels.

https://romisatriawahono.net/lecture/rm/survey/machine%20learning/Frenay%20-%20Classification%20in%20the%20Presence%20of%20Label%20Noise%20-%202014.pdf

Classification in the Presence of Label Noise: A Survey

A method for operating a voice trigger is provided. In some implementations, the method is performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors. The method includes receiving a sound input. The sound input may correspond to a spoken word or phrase, or a portion thereof. The method includes determining whether at least a portion of the sound input corresponds to a predetermined type of sound, such as a human voice. The method includes, upon a determination that at least a portion of the sound input corresponds to the predetermined type, determining whether the sound input includes predetermined content, such as a predetermined trigger word or phrase. The method also includes, upon a determination that the sound input includes the predetermined content, initiating a speech-based service, such as a voice-based digital assistant.

Voice trigger for a digital assistant

A system and method of detecting an aberrant message is provided. An ordered set of words within the message is detected. The set of words found within the message is linked to a corresponding set of expected words, the set of expected words having semantic attributes. A set of grammatical structures represented in the message is detected, based on the ordered set of words and the semantic attributes of the corresponding set of expected words. A cognitive noise vector comprising a quantitative measure of a deviation between grammatical structures represented in the message and an expected measure of grammatical structures for a message of the type is then determined. The cognitive noise vector may be processed by higher levels of the neural network and/or an external processor.

Intelligent control with hierarchical stacked neural networks

Cross-validation is a very commonly employed technique used to evaluate classifier performance. However, it can potentially introduce dataset shift, a harmful factor that is often not taken into account and can result in inaccurate performance estimation. This paper analyzes the prevalence and impact of partition-induced covariate shift on different k-fold cross-validation schemes. From the experimental results obtained, we conclude that the degree of partition-induced covariate shift depends on the cross-validation scheme considered. In this way, worse schemes may harm the correctness of a single-classifier performance estimation and also increase the needed number of repetitions of cross-validation to reach a stable performance estimation.

Study on the Impact of Partition-Induced Dataset Shift on $k$ -Fold Cross-Validation

Cross-validation has often been applied in machine learning research for estimating the accuracies of classifiers. In this work, we propose an extension to this method, called distribution-balanced stratified cross-validation (DBSCV), which improves the estimation quality by providing balanced intraclass distributions when partitioning a data set into multiple folds. We have tested DBSCV on nine real-world and three artificial domains using the C4.5 decision trees classifier. The results show that DBSCV performs better (has smaller biases) than the regular stratified crossvalidationin most cases, especially when the number of folds is small. The analysis and experiments based on three artificial data sets also reveal that DBSCV is particularly effective when multiple intraclass clusters exist in a data set.

Distribution-balanced stratified cross-validation for accuracy estimation

According to various embodiments of the present invention, an automated technique is implemented for resolving and merging fields accurately and reliably, given a set of duplicated records that represents a same entity. In at least one embodiment, a system is implemented that uses a machine learning (ML) method, to train a model from training data, and to learn from users how to efficiently resolve and merge fields. In at least one embodiment, the method of the present invention builds feature vectors as input for its ML method. In at least one embodiment, the system and method of the present invention apply Hierarchical Based Sequencing (HBS) and/or Multiple Output Relaxation (MOR) models in resolving and merging fields. Training data for the ML method can come from any suitable source or combination of sources.

Resolving and merging duplicate records using machine learning

According to various embodiments of the present invention, user performance and/or motivation for a computing system may be maximized by optimizing one or more target components of a user interface of the computing system. The target components may be aspects of the user interface that is perceived by the user. One or more input features and one or more output features may be identified, and data regarding these input and output features may be gathered. This data may be compared with the results generated by a set of candidate artificial intelligence algorithms to determine which of them provides the best fit with the data collected. Then, the selected artificial intelligence algorithm may be applied to the user interface to iteratively change the target components over time until the optimal settings for each user are discovered.

Adaptive User Interfaces

Techniques are described herein for predicting one or more behaviors by an email recipient and, more specifically, to machine learning techniques for predicting one or more behaviors of an email recipient, changing one or more components in the email to increase the likelihood of a behavior, and determining and/or scheduling an optimal time to send the email. Some advantages of the embodiments disclosed herein may include, without limitation, the ability to predict the behavior of the email recipient and suggest the characteristics of an email which will increase the likelihood of a positive behavior, such as a reading or responding to the email, visiting a website, calling a sales representative, or opening an email attachment.

Email optimization for predicted recipient behavior: suggesting changes in an email to increase the likelihood of an outcome

During the data collecting and labeling process it is possible for noise to be introduced into a data set. As a result, the quality of the data set degrades and experiments and inferences derived from the data set become less reliable. In this paper we present an algorithm, called ANR (automatic noise reduction), as a filtering mechanism to identify and remove noisy data items whose classes have been mislabeled. The underlying mechanism behind ANR is based on a framework of multi-layer artificial neural networks. ANR assigns each data item a soft class label in the form of a class probability vector, which is initialized to the original class label and can be modified during training. When the noise level is reasonably small (< 30%), the non-noisy data is dominant in determining the network architecture and its output, and thus a mechanism for correcting mislabeled data can be provided by aligning class probability vector with the network output. With a learning procedure for class probability vector based on its difference from the network output, the probability of a mislabeled class gradually becomes smaller while that of the correct class becomes larger, which eventually causes a correction of mislabeled data after sufficient training. After training, those data items whose classes have been relabeled are then treated as noisy data and removed from the data set. We evaluate the performance of the ANR based on 12 data sets drawn from the UCI data repository. The results show that ANR is capable of identifying a significant portion of noisy data. An average increase in accuracy of 24.5% can be achieved at a noise level of 25% by using ANR as a training data filter for a nearest neighbor classifier, as compared to the one without using ANR.

/pdf/a-noise-filtering-method-using-neural-networks-dnqpvljbhp.pdf

Xinchuan Zeng

Papers

Distribution-balanced stratified cross-validation for accuracy estimation

Resolving and merging duplicate records using machine learning

Adaptive User Interfaces

Email optimization for predicted recipient behavior: suggesting changes in an email to increase the likelihood of an outcome

A noise filtering method using neural networks