Showing papers on "Phrase published in 2012"

A Comparison of Vector-based Representations for Semantic Composition

Abstract: In this paper we address the problem of modeling compositional meaning for phrases and sentences using distributional methods. We experiment with several possible combinations of representation and composition, exhibiting varying degrees of sophistication. Some are shallow while others operate over syntactic structure, rely on parameter learning, or require access to very large corpora. We find that shallow approaches are as good as more computationally intensive alternatives with regards to two particular tests: (1) phrase similarity and (2) paraphrase detection. The sizes of the involved training corpora and the generated vectors are not as important as the fit between the meaning representation and compositional method.

Sentence Simplification by Monolingual Machine Translation

Abstract: In this paper we describe a method for simplifying sentences using Phrase Based Machine Translation, augmented with a re-ranking heuristic based on dissimilarity, and trained on a monolingual parallel corpus. We compare our system to a word-substitution baseline and two state-of-the-art systems, all trained and tested on paired sentences from the English part of Wikipedia and Simple Wikipedia. Human test subjects judge the output of the different systems. Analysing the judgements shows that by relatively careful phrase-based paraphrasing our model achieves similar simplification results to state-of-the-art systems, while generating better formed output. We also argue that text readability metrics such as the Flesch-Kincaid grade level should be used with caution when evaluating the output of simplification systems.

Vector Space Models of Word Meaning and Phrase Meaning: A Survey

Abstract: Distributional models represent a word through the contexts in which it has been observed. They can be used to predict similarity in meaning, based on the distributional hypothesis, which states that two words that occur in similar contexts tend to have similar meanings. Distributional approaches are often implemented in vector space models. They represent a word as a point in high-dimensional space, where each dimension stands for a context item, and a word's coordinates represent its context counts. Occurrence in similar contexts then means proximity in space. In this survey we look at the use of vector space models to describe the meaning of words and phrases: the phenomena that vector space models address, and the techniques that they use to do so. Many word meaning phenomena can be described in terms of semantic similarity: synonymy, priming, categorization, and the typicality of a predicate's arguments. But vector space models can do more than just predict semantic similarity. They are a very flexible tool, because they can make use of all of linear algebra, with all its data structures and operations. The dimensions of a vector space can stand for many things: context words, or non-linguistic context like images, or properties of a concept. And vector space models can use matrices or higher-order arrays instead of vectors for representing more complex relationships. Polysemy is a tough problem for distributional approaches, as a representation that is learned from all of a word's contexts will conflate the different senses of the word. It can be addressed, using either clustering or vector combination techniques. Finally, we look at vector space models for phrases, which are usually constructed by combining word vectors. Vector space models for phrases can predict phrase similarity, and some argue that they can form the basis for a general-purpose representation framework for natural language semantics.

A Phrasal Expressions List

Abstract: There is little dispute that formulaic sequences form an important part of the lexicon, but to date there has been no principled way to prioritize the inclusion of such items in pedagogic materials, such as ESL/EFL textbooks or tests of vocabulary knowledge. While wordlists have been used for decades, they have only provided information about individual word forms (e.g. the General Service List (West 1953) and the Academic Word List (Coxhead 2000)). This article addresses this deficiency by presenting the PHRASal Expressions List (PHRASE List), a list of the 505 most frequent non-transparent multiword expressions in English, intended especially for receptive use. The rationale and development of the list are discussed, as well as its compatibility with British National Corpus single-word frequency lists. It is hoped that the PHRASE List will provide a basis for the systematic integration of multiword lexical items into teaching materials, vocabulary tests, and learning syllabuses.

Irony and Sarcasm: Corpus Generation and Analysis Using Crowdsourcing

Abstract: The ability to reliably identify sarcasm and irony in text can improve the performance of many Natural Language Processing (NLP) systems including summarization, sentiment analysis, etc. The existing sarcasm detection systems have focused on identifying sarcasm on a sentence level or for a specific phrase. However, often it is impossible to identify a sentence containing sarcasm without knowing the context. In this paper we describe a corpus generation experiment where we collect regular and sarcastic Amazon product reviews. We perform qualitative and quantitative analysis of the corpus. The resulting corpus can be used for identifying sarcasm on two levels: a document and a text utterance (where a text utterance can be as short as a sentence and as long as a whole document).

“That's So Gay!”: Examining the Covariates of Hearing This Expression Among Gay, Lesbian, and Bisexual College Students

Abstract: Objective: The investigators examined the health and well-being correlates of hearing the popular phrase “that's so gay” among gay, lesbian, and bisexual (GLB) emerging adults. Participants: Participants were 114 self-identified GLB students aged 18 to 25 years. Methods: An online survey was distributed to students at a large public university in the Midwest during winter 2009. Results: Participants’ social and physical well-being was negatively associated with hearing this phrase, specifically feeling isolated and experiencing physical health symptoms (ie, headaches, poor appetite, or eating problems). Conclusions: College professionals and student leaders must acknowledge that the phrase is a form of heterosexist harassment. As such, policies addressing diversity and harassment should address students’ use of this phrase, aiming to reduce its use. Additionally, colleges and universities should develop practices that counteract poorer well-being associated with hearing the phrase.

Continuous Space Translation Models for Phrase-Based Statistical Machine Translation

Abstract: This paper presents a new approach to perform the estimation of the translation model probabilities of a phrase-based statistical machine translation system. We use neural networks to directly learn the translation probability of phrase pairs using continuous representations. The system can be easily trained on the same data used to build standard phrase-based systems. We provide experimental evidence that the approach seems to be able to infer meaningful translation probabilities for phrase pairs not seen in the training data, or even predict a list of the most likely translations given a source phrase. The approach can be used to rescore n-best lists, but we also discuss an integration into the Moses decoder. A preliminary evaluation on the English/French IWSLT task achieved improvements in the BLEU score and a human analysis showed that the new model often chooses semantically better translations. Several extensions of this work are discussed.

Detecting potential significant errors in speech recognition results

Abstract: In some embodiments, the recognition results produced by a speech processing system (which may include two or more recognition results, including a top recognition result and one or more alternative recognition results) based on an analysis of a speech input, are evaluated for indications of potential significant errors. In some embodiments, the recognition results may be evaluated to determine whether a meaning of any of the alternative recognition results differs from a meaning of the top recognition result in a manner that is significant for a domain, such as the medical domain. In some embodiments, words and/or phrases that may be confused by an ASR system may be determined and associated in sets of words and/or phrases. Words and/or phrases that may be determined include those that change a meaning of a phrase or sentence when included in the phrase/sentence.

Combining Word-Level and Character-Level Models for Machine Translation Between Closely-Related Languages

Abstract: We propose several techniques for improving statistical machine translation between closely-related languages with scarce resources. We use character-level translation trained on n-gram-character-aligned bitexts and tuned using word-level BLEU, which we further augment with character-based transliteration at the word level and combine with a word-level translation model. The evaluation on Macedonian-Bulgarian movie subtitles shows an improvement of 2.84 BLEU points over a phrase-based word-level baseline.

Dynamic pass phrase security system (DPSS)

Abstract: There is disclosed an n-dimensional biometric security system as well as a method of identifying and validating a user through the use of a automated random one-time passphrase generation. The use of tailored templates to generate one-time phase phrase text as well as the use of update subscriptions of templates ensures a high level of security. A verification session preferably uses short, text-independent one-time pass phrases and secure audio tokens with master audio generated from an internal text-to-speech security processor. An automated enrollment process may be implemented in an ongoing and seamless fashion with a user's interactions with the system. Various calibration and tuning techniques are also disclosed.

Word-based self-indexes for natural language text

Abstract: The inverted index supports efficient full-text searches on natural language text collections. It requires some extra space over the compressed text that can be traded for search speed. It is usually fast for single-word searches, yet phrase searches require more expensive intersections. In this article we introduce a different kind of index. It replaces the text using essentially the same space required by the compressed text alone (compression ratio around 35p). Within this space it supports not only decompression of arbitrary passages, but efficient word and phrase searches. Searches are orders of magnitude faster than those over inverted indexes when looking for phrases, and still faster on single-word searches when little space is available. Our new indexes are particularly fast at counting the occurrences of words or phrases. This is useful for computing relevance of words or phrases.We adapt self-indexes that succeeded in indexing arbitrary strings within compressed space to deal with large alphabets. Natural language texts are then regarded as sequences of words, not characters, to achieve word-based self-indexes. We design an architecture that separates the searchable sequence from its presentation aspects. This permits applying case folding, stemming, removing stopwords, etc. as is usual on inverted indexes.

Methods and apparatus for acoustic disambiguation

Abstract: Techniques for disambiguating at least one text segment from at least one acoustically similar word and/or phrase. The techniques include identifying at least one text segment, in a textual representation having a plurality of text segments, having at least one acoustically similar word and/or phrase, annotating the textual representation with disambiguating information to help disambiguate the at least one text segment from the at least one acoustically similar word and/or phrase, and synthesizing a speech signal, at least in part, by performing text-to-speech synthesis on at least a portion of the textual representation that includes the at least one text segment, wherein the speech signal includes speech corresponding to the disambiguating information located proximate the portion of the speech signal corresponding to the at least one text segment.

Phrase frequency effects in language production.

Abstract: A classic debate in the psychology of language concerns the question of the grain-size of the linguistic information that is stored in memory. One view is that only morphologically simple forms are stored (e.g., ‘car’, ‘red’), and that more complex forms of language such as multi-word phrases (e.g., ‘red car’) are generated on-line from the simple forms. In two experiments we tested this view. In Experiment 1, participants produced noun+adjective and noun+noun phrases that were elicited by experimental displays consisting of colored line drawings and two superimposed line drawings. In Experiment 2, participants produced noun+adjective and determiner+noun+adjective utterances elicited by colored line drawings. In both experiments, naming latencies decreased with increasing frequency of the multi-word phrase, and were unaffected by the frequency of the object name in the utterance. These results suggest that the language system is sensitive to the distribution of linguistic information at grain-sizes beyond individual words.

Gestures Enhance Foreign Language Learning

Abstract: Language and gesture are highly interdependent systems that reciprocally influence each other. For example, performing a gesture when learning a word or a phrase enhances its retrieval compared to pure verbal learning. Although the enhancing effects of co-speech gestures on memory are known to be robust, the underlying neural mechanisms are still unclear. Here, we summarize the results of behavioral and neuroscientific studies. They indicate that the neural representation of words consists of complex multimodal networks connecting perception and motor acts that occur during learning. In this context, gestures can reinforce the sensorimotor representation of a word or a phrase, making it resistant to decay. Also, gestures can favor embodiment of abstract words by creating it from scratch. Thus, we propose the use of gesture as a facilitating educational tool that integrates body and mind.

Toward Statistical Machine Translation without Parallel Corpora

Abstract: We estimate the parameters of a phrase-based statistical machine translation system from monolingual corpora instead of a bilingual parallel corpus. We extend existing research on bilingual lexicon induction to estimate both lexical and phrasal translation probabilities for MT-scale phrase-tables. We propose a novel algorithm to estimate reordering probabilities from monolingual data. We report translation results for an end-to-end translation system using these monolingual features alone. Our method only requires monolingual corpora in source and target languages, a small bilingual dictionary, and a small bitext for tuning feature weights. In this paper, we examine an idealization where a phrase-table is given. We examine the degradation in translation performance when bilingually estimated translation probabilities are removed and show that 80%+ of the loss can be recovered with monolingually estimated features alone. We further show that our monolingual features add 1.5 BLEU points when combined with standard bilingually estimated phrase table features.

NiuTrans: An Open Source Toolkit for Phrase-based and Syntax-based Machine Translation

Abstract: We present a new open source toolkit for phrase-based and syntax-based machine translation. The toolkit supports several state-of-the-art models developed in statistical machine translation, including the phrase-based model, the hierachical phrase-based model, and various syntax-based models. The key innovation provided by the toolkit is that the decoder can work with various grammars and offers different choices of decoding algrithms, such as phrase-based decoding, decoding as parsing/tree-parsing and forest-based decoding. Moreover, several useful utilities were distributed with the toolkit, including a discriminative reordering model, a simple and fast language model, and an implementation of minimum error rate training for weight tuning.

Document-Wide Decoding for Phrase-Based Statistical Machine Translation

Abstract: Independence between sentences is an assumption deeply entrenched in the models and algorithms used for statistical machine translation (SMT), particularly in the popular dynamic programming beam search decoding algorithm. This restriction is an obstacle to research on more sophisticated discourse-level models for SMT. We propose a stochastic local search decoding method for phrase-based SMT, which permits free document-wide dependencies in the models. We explore the stability and the search parameters of this method and demonstrate that it can be successfully used to optimise a document-level semantic language model.

Maximum Expected BLEU Training of Phrase and Lexicon Translation Models

Abstract: This paper proposes a new discriminative training method in constructing phrase and lexicon translation models. In order to reliably learn a myriad of parameters in these models, we propose an expected BLEU score-based utility function with KL regularization as the objective, and train the models on a large parallel dataset. For training, we derive growth transformations for phrase and lexicon translation probabilities to iteratively improve the objective. The proposed method, evaluated on the Europarl German-to-English dataset, leads to a 1.1 BLEU point improvement over a state-of-the-art baseline translation system. In IWSLT 2011 Benchmark, our system using the proposed method achieves the best Chinese-to-English translation result on the task of translating TED talks.

Deletion versus pro-forms: an overly simple dichotomy?

Abstract: This paper examines an anaphoric construction, British English do, and locates it within the dichotomy in the ellipsis literature between deleted phrases and null pro-forms, concluding that the choice is a false one, in that pro-forms involve deletion as well; the question, then, is how to account for the differential permeability to dependencies that require external licensing of the various deleted constituents. British English do has some characteristics of a fully deleted phrase, and some of a pro-form. The paper proposes that deletion is involved in this construction, but of a smaller constituent than can host wh-movement or long quantifier-raising. Therefore, deletion must occur within the syntax, in order to bleed syntactic processes. It is further shown that, within a phase-based syntax, Voice must be a phase rather than v, but that both functional heads must exist, and offers a new explanation for the incompatibility of passive and British English do, as well as an account of why some languages, like English, lack impersonal passives, while others, such as Dutch, allow them.

A Phrase-Discovering Topic Model Using Hierarchical Pitman-Yor Processes

Abstract: Topic models traditionally rely on the bag-of-words assumption. In data mining applications, this often results in end-users being presented with inscrutable lists of topical unigrams, single words inferred as representative of their topics. In this article, we present a hierarchical generative probabilistic model of topical phrases. The model simultaneously infers the location, length, and topic of phrases within a corpus and relaxes the bag-of-words assumption within phrases by using a hierarchy of Pitman-Yor processes. We use Markov chain Monte Carlo techniques for approximate inference in the model and perform slice sampling to learn its hyperparameters. We show via an experiment on human subjects that our model finds substantially better, more interpretable topical phrases than do competing models.

Randomized visual phrases for object search

Abstract: Accurate matching of local features plays an essential role in visual object search. Instead of matching individual features separately, using the spatial context, e.g., bundling a group of co-located features into a visual phrase, has shown to enable more discriminative matching. Despite previous work, it remains a challenging problem to extract appropriate spatial context for matching. We propose a randomized approach to deriving visual phrase, in the form of spatial random partition. By averaging the matching scores over multiple randomized visual phrases, our approach offers three benefits: 1) the aggregation of the matching scores over a collection of visual phrases of varying sizes and shapes provides robust local matching; 2) object localization is achieved by simple thresholding on the voting map, which is more efficient than subimage search; 3) our algorithm lends itself to easy parallelization and also allows a flexible trade-off between accuracy and speed by adjusting the number of partition times. Both theoretical studies and experimental comparisons with the state-of-the-art methods validate the advantages of our approach.

The Effect of Learner Corpus Size in Grammatical Error Correction of ESL Writings

Abstract: English as a Second Language (ESL) learners’ writings contain various grammatical errors. Previous research on automatic error correction for ESL learners’ grammatical errors deals with restricted types of learners’ errors. Some types of errors can be corrected by rules using heuristics, while others are difficult to correct without statistical models using native corpora and/or learner corpora. Since adding error annotation to learners’ text is time-consuming, it was not until recently that large scale learner corpora became publicly available. However, little is known about the effect of learner corpus size in ESL grammatical error correction. Thus, in this paper, we investigate the effect of learner corpus size on various types of grammatical errors, using an error correction system based on phrase-based statistical machine translation (SMT) trained on a large scale errortagged learner corpus. We show that the phrase-based SMT approach is effective in correcting frequent errors that can be identified by local context, and that it is difficult for phrase-based SMT to correct errors that need long range contextual information.

Mobile device voice activation

Abstract: A mobile computerized device receives an indication of a first user input comprising a button actuation to initiate a push-to-talk voice search. The device receives from the user a spoken search phrase comprising one or more search terms, and receives an indication of a second user input comprising releasing the actuated button which indicates that the user has finished speaking the search phrase. The mobile device generates a search query using the one or more search terms in the spoken search phrase, responsive to receiving the second user input. In a further example, the computerized mobile device displays one or more likely text search phrases derived from the spoken search phrase via voice-to-text conversion, receives a user input indicating which of the likely text search phrases is an intended search phrase, and uses the intended search phrase as the one or more search terms used in generating the search query.

Linguistic properties of multi-word passphrases

Abstract: We examine patterns of human choice in a passphrase-based authentication system deployed by Amazon, a large online merchant. We tested the availability of a large corpus of over 100,000 possible phrases at Amazon's registration page, which prohibits using any phrase already registered by another user. A number of large, readily-available lists such as movie and book titles prove effective in guessing attacks, suggesting that passphrases are vulnerable to dictionary attacks like all schemes involving human choice. Extending our analysis with natural language phrases extracted from linguistic corpora, we find that phrase selection is far from random, with users strongly preferring simple noun bigrams which are common in natural language. The distribution of chosen passphrases is less skewed than the distribution of bigrams in English text, indicating that some users have attempted to choose phrases randomly. Still, the distribution of bigrams in natural language is not nearly random enough to resist offline guessing, nor are longer three- or four-word phrases for which we see rapidly diminishing returns.

Analysing the Effect of Out-of-Domain Data on SMT Systems

Abstract: In statistical machine translation (SMT), it is known that performance declines when the training data is in a different domain from the test data. Nevertheless, it is frequently necessary to supplement scarce in-domain training data with out-of-domain data. In this paper, we first try to relate the effect of the out-of-domain data on translation performance to measures of corpus similarity, then we separately analyse the effect of adding the out-of-domain data at different parts of the training pipeline (alignment, phrase extraction, and phrase scoring). Through experiments in 2 domains and 8 language pairs it is shown that the out-of-domain data improves coverage and translation of rare words, but may degrade the translation quality for more common words.

Two types of external argument licensing: The case of causers

Abstract: . This article argues that the thematic licensing of causer arguments is not a strictly lexical property but depends on the event configuration within the verbal phrase. The central observation leading to this conclusion is that three morphosyntactically different types of causer-DPs are subject to the same licensing condition: they are licit only in the context of a bi-eventive, resultative event structure. This licensing constellation is not only provided by lexically bi-eventive verbs, but also by overt syntactic composition of a mono-eventive verb with a secondary result predicate where the mono-eventive verb does not license causers on its own. The latter constellation argues against coding causer-roles in a verb’s lexical entry. Instead, it argues for an account that assumes event decomposition of lexically resultative verbs and some version of a configurational θ-theory. Concentrating on existing syntactic versions of such an account, it is shown that they need to be updated to cover the set of data presented in this paper. A central claim put forward is the need to dissociate the verbal layers introducing causative-resultative event structure (which acts as thematic licenser of causers) from those layers introducing external arguments syntactically (formal licensers). Concerning the latter, it is shown that causers, although thematically external arguments, are not necessarily introduced by a Voice projection on top of the verbal predicate.