Contrastive models like CLIP have been shown to learn robust representations of images that capture both semantics and style. To leverage these representations for image generation, we propose a two-stage model: a prior that generates a CLIP image embedding given a text caption, and a decoder that generates an image conditioned on the image embedding. We show that explicitly generating image representations improves image diversity with minimal loss in photorealism and caption similarity. Our decoders conditioned on image representations can also produce variations of an image that preserve both its semantics and style, while varying the non-essential details absent from the image representation. Moreover, the joint embedding space of CLIP enables language-guided image manipulations in a zero-shot fashion. We use diffusion models for the decoder and experiment with both autoregressive and diffusion models for the prior, finding that the latter are computationally more efficient and produce higher-quality samples.

Hierarchical Text-Conditional Image Generation with CLIP Latents

Ethnologue: Languages of the World

Methodological Review: What can natural language processing do for clinical decision support?

Pharmacovigilance from social media: mining adverse drug reaction mentions using sequence labeling with word embedding cluster features.

Named Entity Recognition (NER) is a key component in NLP systems for question answering, information retrieval, relation extraction, etc. NER systems have been studied and developed widely for decades, but accurate systems using deep neural networks (NN) have only been introduced in the last few years. We present a comprehensive survey of deep neural network architectures for NER, and contrast them with previous approaches to NER based on feature engineering and other supervised or semi-supervised learning algorithms. Our results highlight the improvements achieved by neural networks, and show how incorporating some of the lessons learned from past work on feature-based NER systems can yield further improvements.

A Survey on Recent Advances in Named Entity Recognition from Deep Learning models

Learning visual representations of medical images is core to medical image understanding but its progress has been held back by the small size of hand-labeled datasets. Existing work commonly relies on transferring weights from ImageNet pretraining, which is suboptimal due to drastically different image characteristics, or rule-based label extraction from the textual report data paired with medical images, which is inaccurate and hard to generalize. We propose an alternative unsupervised strategy to learn medical visual representations directly from the naturally occurring pairing of images and textual data. Our method of pretraining medical image encoders with the paired text data via a bidirectional contrastive objective between the two modalities is domain-agnostic, and requires no additional expert input. We test our method by transferring our pretrained weights to 4 medical image classification tasks and 2 zero-shot retrieval tasks, and show that our method leads to image representations that considerably outperform strong baselines in most settings. Notably, in all 4 classification tasks, our method requires only 10% as much labeled training data as an ImageNet initialized counterpart to achieve better or comparable performance, demonstrating superior data efficiency.

/pdf/contrastive-learning-of-medical-visual-representations-from-15x0tzocxw.pdf

Contrastive Learning of Medical Visual Representations from Paired Images and Text

With the rapidly growing use of electronic health records, the possibility of large-scale clinical information extraction has drawn much attention. We aim to extract adverse drug events and effects from records. As the first step of this challenge, this study assessed (1) how much adverse-effect information is contained in records, and (2) automatic extracting accuracy of the current standard Natural Language Processing (NLP) system. Results revealed that 7.7% of records include adverse event information, and that 59% of them (4.5% in total) can be extracted automatically. This result is particularly encouraging, considering the massive amounts of records, which are increasing daily.

/pdf/extraction-of-adverse-drug-effects-from-clinical-records-61mdja43lz.pdf

Extraction of adverse drug effects from clinical records.

This paper describes the system that has been used by TeamX in SemEval-2014 Task 9 Subtask B. The system is a sentiment analyzer based on a supervised text categorization approach designed with following two concepts. Firstly, since lexicon features were shown to be effective in SemEval-2013 Task 2, various lexicons and pre-processors for them are introduced to enhance lexical information. Secondly, since a distribution of sentiment on tweets is known to be unbalanced, an weighting scheme is introduced to bias an output of a machine learner. For the test run, the system was tuned towards Twitter texts and successfully achieved high scoring results on Twitter data, average F1 70.96 on Twitter2014 and average F1 56.50 on Twitter2014Sarcasm.

/pdf/teamx-a-sentiment-analyzer-with-enhanced-lexicon-mapping-and-leyoj91os6.pdf

TeamX: A Sentiment Analyzer with Enhanced Lexicon Mapping and Weighting Scheme for Unbalanced Data

With the rapidly growing use of electronic health records, the possibility of large-scale clinical information extraction has drawn much attention. It is not, however, easy to extract information because these reports are written in natural language. To address this problem, this paper presents a system that converts a medical text into a table structure. This system's core technologies are (1) medical event recognition modules and (2) a negative event identification module that judges whether an event actually occurred or not. Regarding the latter module, this paper also proposes an SVM-based classifier using syntactic information. Experimental results demonstrate empirically that syntactic information can contribute to the method's accuracy.

https://www.cs.brandeis.edu/~marc/misc/proceedings/naacl-hlt-2009/BioNLP/pdf/BioNLP24.pdf

TEXT2TABLE: Medical Text Summarization System Based on Named Entity Recognition and Modality Identification

We propose a novel geolocation prediction model using a complex neural network. Geolocation prediction in social media has attracted many researchers to use information of various types. Our model unifies text, metadata, and user network representations with an attention mechanism to overcome previous ensemble approaches. In an evaluation using two open datasets, the proposed model exhibited a maximum 3.8% increase in accuracy and a maximum of 6.6% increase in accuracy@161 against previous models. We further analyzed several intermediate layers of our model, which revealed that their states capture some statistical characteristics of the datasets.

/pdf/unifying-text-metadata-and-user-network-representations-with-1esyk6ri0r.pdf

Yasuhide Miura

Papers

Contrastive Learning of Medical Visual Representations from Paired Images and Text

Extraction of adverse drug effects from clinical records.

TeamX: A Sentiment Analyzer with Enhanced Lexicon Mapping and Weighting Scheme for Unbalanced Data

TEXT2TABLE: Medical Text Summarization System Based on Named Entity Recognition and Modality Identification

Unifying Text, Metadata, and User Network Representations with a Neural Network for Geolocation Prediction