Background
Secondary structure forms an important intermediate level of description of nucleic acids that encapsulates the dominating part of the folding energy, is often well conserved in evolution, and is routinely used as a basis to explain experimental findings. Based on carefully measured thermodynamic parameters, exact dynamic programming algorithms can be used to compute ground states, base pairing probabilities, as well as thermodynamic properties.

/pdf/viennarna-package-2-0-4q9zpqix5v.pdf

ViennaRNA Package 2.0

The measurement of meaning

비만아 부모의 돌봄 경험: q 방법론

We introduce a stochastic graph-based method for computing relative importance of textual units for Natural Language Processing. We test the technique on the problem of Text Summarization (TS). Extractive TS relies on the concept of sentence salience to identify the most important sentences in a document or set of documents. Salience is typically defined in terms of the presence of particular important words or in terms of similarity to a centroid pseudo-sentence. We consider a new approach, LexRank, for computing sentence importance based on the concept of eigenvector centrality in a graph representation of sentences. In this model, a connectivity matrix based on intra-sentence cosine similarity is used as the adjacency matrix of the graph representation of sentences. Our system, based on LexRank ranked in first place in more than one task in the recent DUC 2004 evaluation. In this paper we present a detailed analysis of our approach and apply it to a larger data set including data from earlier DUC evaluations. We discuss several methods to compute centrality using the similarity graph. The results show that degree-based methods (including LexRank) outperform both centroid-based methods and other systems participating in DUC in most of the cases. Furthermore, the LexRank with threshold method outperforms the other degree-based techniques including continuous LexRank. We also show that our approach is quite insensitive to the noise in the data that may result from an imperfect topical clustering of documents.

/pdf/lexrank-graph-based-lexical-centrality-as-salience-in-text-10aso8e2ok.pdf

LexRank: graph-based lexical centrality as salience in text summarization

If we take an existing supervised NLP system, a simple and general way to improve accuracy is to use unsupervised word representations as extra word features. We evaluate Brown clusters, Collobert and Weston (2008) embeddings, and HLBL (Mnih & Hinton, 2009) embeddings of words on both NER and chunking. We use near state-of-the-art supervised baselines, and find that each of the three word representations improves the accuracy of these baselines. We find further improvements by combining different word representations. You can download our word features, for off-the-shelf use in existing NLP systems, as well as our code, here: http://metaoptimize.com/projects/wordreprs/

/pdf/word-representations-a-simple-and-general-method-for-semi-1096uffw40.pdf

Word Representations: A Simple and General Method for Semi-Supervised Learning

We introduce precision-biased parsing: a parsing task which favors precision over recall by allowing the parser to abstain from decisions deemed uncertain. We focus on dependency-parsing and present an ensemble method which is capable of assigning parents to 84% of the text tokens while being over 96% accurate on these tokens. We use the precision-biased parsing task to solve the related high-quality parse-selection task: finding a subset of high-quality (accurate) trees in a large collection of parsed text. We present a method for choosing over a third of the input trees while keeping unlabeled dependency parsing accuracy of 97% on these trees. We also present a method which is not based on an ensemble but rather on directly predicting the risk associated with individual parser decisions. In addition to its efficiency, this method demonstrates that a parsing system can provide reasonable estimates of confidence in its predictions without relying on ensembles or aggregate corpus counts.

https://www.cs.bgu.ac.il/~yoavg/publications/emnlp2012confidencea.pdf

Precision-biased Parsing and High-Quality Parse Selection

We present an implemented procedure to select an appropriate connective to link two propositions. Each connective is defined as a set of constraints between features of the propositions it connects. Our focus has been to identify pragmatic features that can be produced by a deep generator to provide a simple representation of connectives. Using these features, we can account for a variety of connective usages. We describe how a surface generator can produce complex sentences when given these features in input. The selection procedure is implemented as part of a large functional unification grammar. Topic area: language generation

/pdf/a-procedure-for-the-selection-of-connectives-how-deep-is-the-dzv1trqa84.pdf

A Procedure for the Selection of Connectives: How Deep Is the Surface?

Masked language modeling (MLM) is one of the key sub-tasks in vision-language pretraining. In the cross-modal setting, tokens in the sentence are masked at random, and the model predicts the masked tokens given the image and the text. In this paper, we observe several key disadvantages of MLM in this setting. First, as captions tend to be short, in a third of the sentences no token is sampled. Second, the majority of masked tokens are stop-words and punctuation, leading to under-utilization of the image. We investigate a range of alternative masking strategies specific to the cross-modal setting that address these shortcomings, aiming for better fusion of text and image in the learned representation. When pre-training the LXMERT model, our alternative masking strategies consistently improve over the original masking strategy on three downstream tasks, especially in low resource settings. Further, our pre-training approach substantially outperforms the baseline model on a prompt-based probing task designed to elicit image objects. These results and our analysis indicate that our method allows for better utilization of the training data.

Data Efficient Masked Language Modeling for Vision and Language

Information Retrieval (IR) research has recently started addressing the information need of exploratory search, where the searcher may be unfamiliar with the domain or not have decided what is the goal of his query. A popular approach to support exploratory search is the usage of faceted search. The implementation of faceted search requires that documents be annotated by metadata in the form of attributes and hierarchical categories. In many applications, the metadata is maintained manually, in the form of a search ontology. Recent work has also investigated methods to automatically acquire such metadata from sample documents. In this work, we propose a new method to automatically evaluate the quality of such a search ontology. Our method relies on mapping ontology individuals to textual documents. On the basis of this mapping, we evaluate the adequacy of the ontology by translating ontology properties into properties over the textual corpora, which can be empirically tested using natural language processing techniques. This data-driven method provides concrete feedback to ontology maintainers and quantitative estimation of the functional adequacy of the ontology towards search experience improvement. We test this ontology evaluation method on an ontology in the Movies domain, that has been acquired automatically from the integration of multiple semi-structured and textual data sources (e.g., IMDb and Wikipedia). We automatically construct a domain corpus from a set of movie individuals by crawling the Web for movie reviews. The 1-1 relation between textual documents (reviews) and movie individuals in the ontology enables us to test the ontology terminological coverage, its classification potential and its capability to capture the topic of movies. Since the proposed ontology evaluation method relies only on the possibility to align textual documents to ontology individuals, it is domain independent and can be applied to various, well-documented domains.

/pdf/automatic-evaluation-of-search-ontologies-in-the-36u2h6r0ev.pdf

Automatic Evaluation of Search Ontologies in the Entertainment Domain Using Natural Language Processing

We study cross-lingual UMLS named entity linking, where mentions in a given source language are mapped to UMLS concepts, most of which are labeled in English. Our cross-lingual framework includes an offline unsupervised construction of a translated UMLS dictionary and a per-document pipeline which identifies UMLS candidate mentions and uses a fine-tuned pretrained transformer language model to filter candidates according to context. Our method exploits a small dataset of manually annotated UMLS mentions in the source language and uses this supervised data in two ways: to extend the unsupervised UMLS dictionary and to fine-tune the contextual filtering of candidate mentions in full documents.We demonstrate results of our approach on both Hebrew and English. We achieve new state-of-the-art (SOTA) results on the Hebrew Camoni corpus, +8.9 F1 on average across three communities in the dataset. We also achieve new SOTA on the English dataset MedMentions with +7.3 F1.

/pdf/cross-lingual-umls-named-entity-linking-using-umls-2t82mlqw.pdf

Michael Elhadad

Papers

Precision-biased Parsing and High-Quality Parse Selection

A Procedure for the Selection of Connectives: How Deep Is the Surface?

Data Efficient Masked Language Modeling for Vision and Language

Automatic Evaluation of Search Ontologies in the Entertainment Domain Using Natural Language Processing

Cross-Lingual UMLS Named Entity Linking using UMLS Dictionary Fine-Tuning