Many Natural Language Processing applications nowadays rely on pre-trained word representations estimated from large text corpora such as news collections, Wikipedia and Web Crawl. In this paper, we show how to train high-quality word vector representations by using a combination of known tricks that are however rarely used together. The main result of our work is the new set of publicly available pre-trained models that outperform the current state of the art by a large margin on a number of tasks.

Advances in Pre-Training Distributed Word Representations

L'article, publie dans Science, sur une des premieres utilisations analytiques de Google Books, fondee sur les n-grammes (Google Ngrams) We constructed a corpus of digitized texts containing about 4% of all books ever printed. Analysis of this corpus enables us to investigate cultural trends quantitatively. We survey the vast terrain of "culturomics", focusing on linguistic and cultural phenomena that were reflected in the English language between 1800 and 2000. We show how this approach can ...

Quantitative Analysis of Culture Using Millions of Digitized Books

In Semantic Textual Similarity, systems rate the degree of semantic equivalence between two text snippets. This year, the participants were challenged with new data sets for English, as well as the introduction of Spanish, as a new language in which to assess semantic similarity. For the English subtask, we exposed the systems to a diversity of testing scenarios, by preparing additional OntoNotesWordNet sense mappings and news headlines, as well as introducing new genres, including image descriptions, DEFT discussion forums, DEFT newswire, and tweet-newswire headline mappings. For Spanish, since, to our knowledge, this is the first time that official evaluations are conducted, we used well-formed text, by featuring sentences extracted from encyclopedic content and newswire. The annotations for both tasks leveraged crowdsourcing. The Spanish subtask engaged 9 teams participating with 22 system runs, and the English subtask attracted 15 teams with 38 system runs.

/pdf/semeval-2014-task-10-multilingual-semantic-textual-1tp3uw8dic.pdf

SemEval-2014 Task 10: Multilingual Semantic Textual Similarity

Determining semantic similarity between texts is important in many tasks in information retrieval such as search, query suggestion, automatic summarization and image finding. Many approaches have been suggested, based on lexical matching, handcrafted patterns, syntactic parse trees, external sources of structured semantic knowledge and distributional semantics. However, lexical features, like string matching, do not capture semantic similarity beyond a trivial level. Furthermore, handcrafted patterns and external sources of structured semantic knowledge cannot be assumed to be available in all circumstances and for all domains. Lastly, approaches depending on parse trees are restricted to syntactically well-formed texts, typically of one sentence in length. We investigate whether determining short text similarity is possible using only semantic features---where by semantic we mean, pertaining to a representation of meaning---rather than relying on similarity in lexical or syntactic representations. We use word embeddings, vector representations of terms, computed from unlabelled data, that represent terms in a semantic space in which proximity of vectors can be interpreted as semantic similarity. We propose to go from word-level to text-level semantics by combining insights from methods based on external sources of semantic knowledge with word embeddings. A novel feature of our approach is that an arbitrary number of word embedding sets can be incorporated. We derive multiple types of meta-features from the comparison of the word vectors for short text pairs, and from the vector means of their respective word embeddings. The features representing labelled short text pairs are used to train a supervised learning algorithm. We use the trained model at testing time to predict the semantic similarity of new, unlabelled pairs of short texts We show on a publicly available evaluation set commonly used for the task of semantic similarity that our method outperforms baseline methods that work under the same conditions.

/pdf/short-text-similarity-with-word-embeddings-1h71ossbvq.pdf

Short Text Similarity with Word Embeddings

We describe three semantic text similarity systems developed for the *SEM 2013 STS shared task and the results of the corresponding three runs. All of them shared a word similarity feature that combined LSA word similarity and WordNet knowledge. The first, which achieved the best mean score of the 89 submitted runs, used a simple term alignment algorithm augmented with penalty terms. The other two runs, ranked second and fourth, used support vector regression models to combine larger sets of features.

/pdf/umbc-ebiquity-core-semantic-textual-similarity-systems-2k38rpjo6l.pdf

UMBC_EBIQUITY-CORE: Semantic Textual Similarity Systems

Semantic textual similarity is a measure of the degree of semantic equivalence between two pieces of text. We describe the SemSim system and its performance in the *SEM 2013 and SemEval-2014 tasks on semantic textual similarity. At the core of our system lies a robust distributional word similarity component that combines latent semantic analysis and machine learning augmented with data from several linguistic resources. We used a simple term alignment algorithm to handle longer pieces of text. Additional wrappers and resources were used to handle task specific challenges that include processing Spanish text, comparing text sequences of different lengths, handling informal words and phrases, and matching words with sense definitions. In the *SEM 2013 task on Semantic Textual Similarity, our best performing system ranked first among the 89 submitted runs. In the SemEval-2014 task on Multilingual Semantic Textual Similarity, we ranked a close second in both the English and Spanish subtasks. In the SemEval-2014 task on Cross-Level Semantic Similarity, we ranked first in Sentence---Phrase, Phrase---Word, and Word---Sense subtasks and second in the Paragraph---Sentence subtask.

/pdf/robust-semantic-text-similarity-using-lsa-machine-learning-21lv435ihu.pdf

Robust semantic text similarity using LSA, machine learning, and linguistic resources

We describe UMBC’s systems developed for the SemEval 2014 tasks on Multilingual Semantic Textual Similarity (Task 10) and Cross-Level Semantic Similarity (Task 3). Our best submission in the Multilingual task ranked second in both English and Spanish subtasks using an unsupervised approach. Our best systems for Cross-Level task ranked second in Paragraph-Sentence and first in both Sentence-Phrase and Word-Sense subtask. The system ranked first for the PhraseWord subtask but was not included in the official results due to a late submission.

/pdf/meerkat-mafia-multilingual-and-cross-level-semantic-textual-2c595ie4sz.pdf

Meerkat Mafia: Multilingual and Cross-Level Semantic Textual Similarity Systems

The human face forms an important interface to convey nonverbal emotional information. Facial expressions reflect an individual's reactions to personal thoughts or external stimuli. These can act as valuable supplementary biometric information to automated person identification systems. In this study, video segments of individuals were FACS coded to quantify facial expressions. The Action Unit (AU) frequencies, considered both individually and in specific combinations, served as features for person identification. The experiments confirm that these features of the facial behavior are well suited for biometric person identification. Considering both facial asymmetry and Action Unit combinations resulted in a significant improvement in the identification efficiency. Additionally, we observed the convergence in the identification process with the increase of the training data. Thus, given sufficient training data, facial behavior can serve as a reliable biometric modality.

/pdf/facial-behavior-as-a-soft-biometric-2d2qd8caqb.pdf

Facial behavior as a soft biometric

In the past few years, the Internet of Things has started to become a reality; however, its growth has been hampered by privacy and security concerns. One promising approach is to use Semantic Web technologies to mitigate privacy concerns in an informed, flexible way. We present CARLTON, a framework for managing data privacy for entities in a Physical Web deployment using Semantic Web technologies. CARLTON uses context-sensitive privacy policies to protect privacy of organizational and personnel data. We provide use case scenarios where natural language queries for data are handled by the system, and show how privacy policies may be used to manage data privacy in such scenarios, based on an ontology of concepts that can be used as rule antecedents in customizable privacy policies.

/pdf/semantic-knowledge-and-privacy-in-the-physical-web-11joy7jb1c.pdf

Abhay L. Kashyap

Papers

UMBC_EBIQUITY-CORE: Semantic Textual Similarity Systems

Robust semantic text similarity using LSA, machine learning, and linguistic resources

Meerkat Mafia: Multilingual and Cross-Level Semantic Textual Similarity Systems

Facial behavior as a soft biometric

Semantic knowledge and privacy in the physical web