Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Neural machine translation (NMT) models typically operate with a fixed vocabulary, but translation is an open-vocabulary problem. Previous work addresses the translation of out-of-vocabulary words by backing off to a dictionary. In this paper, we introduce a simpler and more effective approach, making the NMT model capable of open-vocabulary translation by encoding rare and unknown words as sequences of subword units. This is based on the intuition that various word classes are translatable via smaller units than words, for instance names (via character copying or transliteration), compounds (via compositional translation), and cognates and loanwords (via phonological and morphological transformations). We discuss the suitability of different word segmentation techniques, including simple character ngram models and a segmentation based on the byte pair encoding compression algorithm, and empirically show that subword models improve over a back-off dictionary baseline for the WMT 15 translation tasks English!German and English!Russian by up to 1.1 and 1.3 BLEU, respectively.

/pdf/neural-machine-translation-of-rare-words-with-subword-units-3w3a5ojqnb.pdf

Neural Machine Translation of Rare Words with Subword Units

We describe an open-source toolkit for statistical machine translation whose novel contributions are (a) support for linguistically motivated factors, (b) confusion network decoding, and (c) efficient data formats for translation models and language models. In addition to the SMT decoder, the toolkit also includes a wide variety of tools for training, tuning and applying the system to many translation tasks.

/pdf/moses-open-source-toolkit-for-statistical-machine-3pmfuwk8uc.pdf

Moses: Open Source Toolkit for Statistical Machine Translation

This introductory text to statistical machine translation (SMT) provides all of the theories and methods needed to build a statistical machine translator, such as Google Language Tools and Babelfish. In general, statistical techniques allow automatic translation systems to be built quickly for any language-pair using only translated texts and generic software. With increasing globalization, statistical machine translation will be central to communication and commerce. Based on courses and tutorials, and classroom-tested globally, it is ideal for instruction or self-study, for advanced undergraduates and graduate students in computer science and/or computational linguistics, and researchers in natural language processing. The companion website provides open-source corpora and tool-kits.

/pdf/statistical-machine-translation-1yr1jtzcjp.pdf

Statistical Machine Translation

We present a statistical phrase-based translation model that uses hierarchical phrases---phrases that contain subphrases. The model is formally a synchronous context-free grammar but is learned from a bitext without any syntactic information. Thus it can be seen as a shift to the formal machinery of syntax-based translation systems without any linguistic commitment. In our experiments using BLEU as a metric, the hierarchical phrase-based model achieves a relative improvement of 7.5% over Pharaoh, a state-of-the-art phrase-based system.

/pdf/a-hierarchical-phrase-based-model-for-statistical-machine-1xejmbxuk5.pdf

A Hierarchical Phrase-Based Model for Statistical Machine Translation

This paper presents a statistical machine translation trained on normalized corpora. The automatic paraphrasing is carried out by inducing paraphrasing expressions from a bilingual corpus. Then, the normalization is treated as a specific paraphrase of a given input determined by the frequency in a corpus. The experimental results on Japanese-to-English translation with normalized English corpus exhibited the reduction of word-error-rate by 8% and the improvement of subjective evaluation from 70% into 72.5%.

/pdf/statistical-machine-translation-on-paraphrased-corpora-513snsnv8e.pdf

Statistical Machine Translation on Paraphrased Corpora.

[Object] An object is to provide an apparatus for attaining highly precise word alignment. [Solution] The apparatus includes: selecting means receiving a bilingual sentence pair and a word alignment for the bilingual sentence pair, for successively selecting words f j of a sentence in a first language in a prescribed order; and a recurrent neural network (RNN) 100 , computing, for all words of the sentence in the first language, a score 102 representing a probability that a word pair consisting of the word f j and a word e a _ {j} aligned with the word f j by a word alignment a j in a second language of the bilingual sentence pair is a correct word pair, and based on this score, for computing a score of the word alignment a j . When computing a score of word pair (f j , e a _ {j} ), RNN 100 computes a score 102 of the word pair (f j , e a _ {j} ) based on all word alignments a 1 j-1 selected by the selecting means prior to the word f j of the word pair (f j , e a _ {j} ), of the word alignments a j , by means of a recurrent connection 118.

Word alignment score computing apparatus, word alignment apparatus, and computer program

This paper presents a novel method for nested named entity recognition. As a layered method, our method extends the prior second-best path recognition method by explicitly excluding the influence of the best path. Our method maintains a set of hidden states at each time step and selectively leverages them to build a different potential function for recognition at each level. In addition, we demonstrate that recognizing innermost entities first results in better performance than the conventional outermost entities first scheme. We provide extensive experimental results on ACE2004, ACE2005, and GENIA datasets to show the effectiveness and efficiency of our proposed method.

/pdf/nested-named-entity-recognition-via-explicitly-excluding-the-knc8m2mkfx.pdf

Nested Named Entity Recognition via Explicitly Excluding the Influence of the Best Path

In this paper, we propose to use various global features for discriminative reranking in an SMT framework. We employ an online large-margin based training algorithm for the structural output support vector machines based on the margin infused relaxed algorithm. Besides the standard features used, such as decoder's scores, source and target sentences, alignments and part-of-speech tags, we include sentence type probabilities, posterior probabilities and back translation features for reranking. These features have been proved to be useful in other approaches in statistical machine translation but it is the first attempt to apply them in reranking. Our experimental results using 160K BTEC corpus show an improvement of 1–4 BLEU percentage points on Japanese/Chinese to English translation.

Discriminative reranking for SMT using various global features

This paper reports the NTT statistical translation system participating in the evaluation campaign of IWSLT 2005. The NTT system is based on a phrase translation model and utilizes a large number of features with a log-linear model. We studied the various features recently developed in this research field and evaluate the system using supplied data as well as publicly available Chinese, Japanese, and English data. Despite domain mismatch, additional data helped improve translation accuracy.

Taro Watanabe

Papers

Statistical Machine Translation on Paraphrased Corpora.

Word alignment score computing apparatus, word alignment apparatus, and computer program

Nested Named Entity Recognition via Explicitly Excluding the Influence of the Best Path

Discriminative reranking for SMT using various global features

The NTT Statistical Machine Translation System for IWSLT2005