Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Statistical approaches to processing natural language text have become dominant in recent years This foundational text is the first comprehensive introduction to statistical natural language processing (NLP) to appear The book contains all the theory and algorithms needed for building NLP tools It provides broad but rigorous coverage of mathematical and linguistic foundations, as well as detailed discussion of statistical methods, allowing students and researchers to construct their own implementations The book covers collocation finding, word sense disambiguation, probabilistic parsing, information retrieval, and other applications

/pdf/foundations-of-statistical-natural-language-processing-1ou5e6c2q6.pdf

Foundations of Statistical Natural Language Processing

Data Mining Practical Machine Learning Tools and Techniques

2 1 The innovative transport systems and the CityMobil project 10 1.1 The research questions 10 2 The state of art in the field of automatic transport systems 12 2.1 Case studies and demand studies for innovative transport systems 12 3 The design and implementation of surveys 14 3.1 Definition of experimental design 14 3.2 Questionnaire design and delivery 16 3.3 First analyses on the collected sample 18 4 Calibration of Logit Multionomial demand models 21 4.1 Methodology 21 4.2 Calibration of the “full” model. 22 4.3 Calibration of the “final” model 24 4.4 The demand analysis through the final Multinomial Logit model 25 5 The analysis of interaction between the demand and socioeconomic attributes 31 5.1 Methodology 31 5.2 Application of Mixed Logit models to the demand 31 5.3 Analysis of the interactions between demand and socioeconomic attributes through Mixed Logit models 32 5.4 Mixed Logit model and interaction between age and the demand for the CTS 38 5.5 Demand analysis with Mixed Logit model 39 6 Final analyses and conclusions 45 6.1 Comparison between the results of the analyses 45 6.2 Conclusions 48 6.3 Answers to the research questions and future developments 52

The European commission

Feature selection techniques have become an apparent need in many bioinformatics applications. In addition to the large pool of techniques that have already been developed in the machine learning and data mining fields, specific applications in bioinformatics have led to a wealth of newly proposed techniques.

In this article, we make the interested reader aware of the possibilities of feature selection, providing a basic taxonomy of feature selection techniques, and discussing their use, variety and potential in a number of both common as well as upcoming bioinformatics applications.

Contact: yvan.saeys@psb.ugent.be

Supplementary information: http://bioinformatics.psb.ugent.be/supplementary_data/yvsae/fsreview

A review of feature selection techniques in bioinformatics

We investigate the usefulness of evolutionary algorithms in three incarnations of the problem of feature relevance assignment in memory-based language processing (MBLP): feature weighting, feature ordering and feature selection. We use a simple genetic algorithm (GA) for this problem on two typical tasks in natural language processing: morphological synthesis and unknown word tagging. We find that GA feature selection always significantly outperforms the MBLP variant without selection and that feature ordering and weighting with GA significantly outperforms a situation where no weighting is used. However, GA selection does not significantly do better than simple iterative feature selection methods, and GA weighting and ordering reach only similar performance as current information-theoretic feature weighting methods.

/pdf/genetic-algorithms-for-feature-relevance-assignment-in-1rfrqcc4z6.pdf

Genetic algorithms for feature relevance assignment in memory-based language processing

In a large corpus (2.9 million tokens) of chat conversations, we studied the impact of Flemish adolescents’ social background on non-standard writing. We found significant correlations between different aspects of social class (level of education, home language and profession of the parents) and all examined deviations from formal written standard Dutch. Clustering several social variables might not only lead to a better operationalization of the complex phenomenon of social class, it certainly allows for discriminating social groups with distinct linguistic practices: lower class teenagers used each of the non-standard features much more often and in some cases in a different way than their upper class peers. Possible explanations concern discrepancies in terms of both linguistic proficiency and linguistic attitudes. Our findings emphasize the importance of including social background as an independent variable in variationist studies on youngsters’ computer-mediated communication.

Adolescents’ social background and non-standard writing in online communication

Neural machine translation (NMT) is nowadays commonly applied at the subword level, using byte-pair encoding. A promising alternative approach focuses on character-level translation, which simplifies processing pipelines in NMT considerably. This approach, however, must consider relatively longer sequences, rendering the training process prohibitively expensive. In this paper, we discuss a novel, Transformer-based approach, that we compare, both in speed and in quality to the Transformer at subword and character levels, as well as previously developed character-level models. We evaluate our models on 4 language pairs from WMT'15: DE-EN, CS-EN, FI-EN and RU-EN. The proposed novel architecture can be trained on a single GPU and is 34% percent faster than the character-level Transformer; still, the obtained results are at least on par with it. In addition, our proposed model outperforms the subword-level model in FI-EN and shows close results in CS-EN. To stimulate further research in this area and close the gap with subword-level NMT, we make all our code and models publicly available.

Character-level Transformer-based Neural Machine Translation

Coreference resolution, determining the appropriate discourse referent for an anaphoric expression, is an essential but difficult task in natural language processing. It has been observed that an important source of errors in machine-learning based approaches to this task, is the wrong disambiguation of the third person singular neuter pronoun as either referential or non-referential. In this paper, we investigate whether a machine learning based approach can be successfully applied to the disambiguation of the neuter pronoun in Dutch and show a modest potential effect of this disambiguation on the results of a machine learning based coreference resolution system for Dutch.

/pdf/disambiguation-of-the-neuter-pronoun-and-its-effect-on-2bvkvigbdm.pdf

Disambiguation of the neuter pronoun and its effect on pronominal coreference resolution

Regular Papers.- Exceptional Model Mining.- A Joint Topic and Perspective Model for Ideological Discourse.- Effective Pruning Techniques for Mining Quasi-Cliques.- Efficient Pairwise Multilabel Classification for Large-Scale Problems in the Legal Domain.- Fitted Natural Actor-Critic: A New Algorithm for Continuous State-Action MDPs.- A New Natural Policy Gradient by Stationary Distribution Metric.- Towards Machine Learning of Grammars and Compilers of Programming Languages.- Improving Classification with Pairwise Constraints: A Margin-Based Approach.- Metric Learning: A Support Vector Approach.- Support Vector Machines, Data Reduction, and Approximate Kernel Matrices.- Mixed Bregman Clustering with Approximation Guarantees.- Hierarchical, Parameter-Free Community Discovery.- A Genetic Algorithm for Text Classification Rule Induction.- Nonstationary Gaussian Process Regression Using Point Estimates of Local Smoothness.- Kernel-Based Inductive Transfer.- State-Dependent Exploration for Policy Gradient Methods.- Client-Friendly Classification over Random Hyperplane Hashes.- Large-Scale Clustering through Functional Embedding.- Clustering Distributed Sensor Data Streams.- A Novel Scalable and Data Efficient Feature Subset Selection Algorithm.- Robust Feature Selection Using Ensemble Feature Selection Techniques.- Effective Visualization of Information Diffusion Process over Complex Networks.- Actively Transfer Domain Knowledge.- A Unified View of Matrix Factorization Models.- Parallel Spectral Clustering.- Classification of Multi-labeled Data: A Generative Approach.- Pool-Based Agnostic Experiment Design in Linear Regression.- Distribution-Free Learning of Bayesian Network Structure.- Assessing Nonlinear Granger Causality from Multivariate Time Series.- Clustering Via Local Regression.- Decomposable Families of Itemsets.- Transferring Instances for Model-Based Reinforcement Learning.- A Simple Model for Sequences of Relational State Descriptions.- Semi-Supervised Boosting for Multi-Class Classification.- A Joint Segmenting and Labeling Approach for Chinese Lexical Analysis.- Transferred Dimensionality Reduction.- Multiple Manifolds Learning Framework Based on Hierarchical Mixture Density Model.- Estimating Sales Opportunity Using Similarity-Based Methods.- Learning MDP Action Models Via Discrete Mixture Trees.- Continuous Time Bayesian Networks for Host Level Network Intrusion Detection.- Data Streaming with Affinity Propagation.- Semi-supervised Discriminant Analysis Via CCCP.- Demo Papers.- A Visualization-Based Exploratory Technique for Classifier Comparison with Respect to Multiple Metrics and Multiple Domains.- Pleiades: Subspace Clustering and Evaluation.- SEDiL: Software for Edit Distance Learning.- Monitoring Patterns through an Integrated Management and Mining Tool.- A Knowledge-Based Digital Dashboard for Higher Learning Institutions.- SINDBAD and SiQL: An Inductive Database and Query Language in the Relational Model.

Walter Daelemans

Papers

Genetic algorithms for feature relevance assignment in memory-based language processing

Adolescents’ social background and non-standard writing in online communication

Character-level Transformer-based Neural Machine Translation

Disambiguation of the neuter pronoun and its effect on pronominal coreference resolution

Machine learning and knowledge discovery in databases : European conference, ECML PKDD 2008, Antwerp, Belgium, September 15-19, 2008 : proceedings