Open AccessPosted Content
Clinical Concept Extraction with Contextual Word Embedding
TLDR
The authors proposed a clinical concept extraction model for automatic annotation of clinical problems, treatments, and tests in clinical notes utilizing domain-specific contextual word embedding, which achieved the best performance among reported baseline models and outperformed the state-of-the-art models by 3.4%.Abstract:
Automatic extraction of clinical concepts is an essential step for turning the unstructured data within a clinical note into structured and actionable information. In this work, we propose a clinical concept extraction model for automatic annotation of clinical problems, treatments, and tests in clinical notes utilizing domain-specific contextual word embedding. A contextual word embedding model is first trained on a corpus with a mixture of clinical reports and relevant Wikipedia pages in the clinical domain. Next, a bidirectional LSTM-CRF model is trained for clinical concept extraction using the contextual word embedding model. We tested our proposed model on the I2B2 2010 challenge dataset. Our proposed model achieved the best performance among reported baseline models and outperformed the state-of-the-art models by 3.4% in terms of F1-score.read more
Citations
More filters
Journal ArticleDOI
BioBERT: a pre-trained biomedical language representation model for biomedical text mining.
TL;DR: This article proposed BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining), which is a domain-specific language representation model pre-trained on large-scale biomedical corpora.
Proceedings ArticleDOI
Publicly Available Clinical BERT Embeddings
Emily Alsentzer,John Murphy,William Boag,Wei-Hung Weng,Di Jindi,Tristan Naumann,Matthew B. A. McDermott +6 more
TL;DR: This paper explored and released BERT models for clinical text: one for generic clinical text and another for discharge summaries specifically, and demonstrated that using a domain-specific model yields performance improvements on 3/5 clinical NLP tasks, establishing a new state-of-the-art on the MedNLI dataset.
Posted Content
Publicly Available Clinical BERT Embeddings
Emily Alsentzer,John Murphy,Willie Boag,Wei-Hung Weng,Di Jin,Tristan Naumann,Matthew B. A. McDermott +6 more
TL;DR: This work explores and releases two BERT models for clinical text: one for generic clinical text and another for discharge summaries specifically, and demonstrates that using a domain-specific model yields performance improvements on 3/5 clinical NLP tasks, establishing a new state-of-the-art on the MedNLI dataset.
Journal ArticleDOI
Enhancing clinical concept extraction with contextual embeddings.
TL;DR: This article explored the space of possible options in utilizing these new models for clinical concept extraction, including comparing these to traditional word embedding methods (word2vec, GloVe, fastText).
Journal ArticleDOI
Building a PubMed knowledge graph.
Jian Xu,Sunkyu Kim,Min Song,Minbyul Jeong,Donghyeon Kim,Jaewoo Kang,Justin F. Rousseau,Xin Li,Weijia Xu,Vetle I. Torvik,Yi Bu,Chongyan Chen,Islam Akef Ebeid,Daifeng Li,Ying Ding +14 more
TL;DR: Wang et al. as mentioned in this paper constructed a PubMed knowledge graph (PKG) by extracting bio-entities from 29 million PubMed abstracts, disambiguating author names, integrating funding data through the National Institutes of Health (NIH) ExPORTER, collecting affiliation history and educational background of authors from ORCID®, and identifying fine-grained affiliation data from MapAffil.
References
More filters
Proceedings ArticleDOI
Glove: Global Vectors for Word Representation
TL;DR: A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure.
Posted Content
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: In this article, the adaptive estimates of lower-order moments are used for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimate of lowerorder moments.
Posted Content
Distributed Representations of Words and Phrases and their Compositionality
TL;DR: In this paper, the Skip-gram model is used to learn high-quality distributed vector representations that capture a large number of precise syntactic and semantic word relationships and improve both the quality of the vectors and the training speed.
Posted Content
Bidirectional LSTM-CRF Models for Sequence Tagging
Zhiheng Huang,Wei Xu,Kai Yu +2 more
TL;DR: This work is the first to apply a bidirectional LSTM CRF model to NLP benchmark sequence tagging data sets and it is shown that the BI-LSTM-CRF model can efficiently use both past and future input features thanks to a biddirectional L STM component.
Posted Content
Deep contextualized word representations
Matthew E. Peters,Mark Neumann,Mohit Iyyer,Matt Gardner,Christopher Clark,Kenton Lee,Luke Zettlemoyer +6 more
TL;DR: This article introduced a new type of deep contextualized word representation that models both complex characteristics of word use (e.g., syntax and semantics), and how these uses vary across linguistic contexts (i.e., to model polysemy).