This paper describes the SemEval-2014, Task 7 on the Analysis of Clinical Text and presents the evaluation results. It focused on two subtasks: (i) identification (Task A) and (ii) normalization (Task B) of diseases and disorders in clinical reports as annotated in the Shared Annotated Resources (ShARe) 1 corpus. This task was a follow-up to the ShARe/CLEF eHealth 2013 shared task, subtasks 1a and 1b, 2 but using a larger test set. A total of 21 teams competed in Task A, and 18 of those also participated in Task B. For Task A, the best system had a strict F1-score of 81.3, with a precision of 84.3 and recall of 78.6. For Task B, the same group had the best strict accuracy of 74.1. The organizers have made the text corpora, annotations, and evaluation tools available for future research and development at the shared task website. 3

/pdf/semeval-2015-task-14-analysis-of-clinical-text-51wxkoofx6.pdf

SemEval-2015 Task 14: Analysis of Clinical Text

Most state-of-the-art biomedical entity normalization systems, such as rule-based systems, merely rely on morphological information of entity mentions, but rarely consider their semantic information. In this paper, we introduce a novel convolutional neural network (CNN) architecture that regards biomedical entity normalization as a ranking problem and benefits from semantic information of biomedical entities. The CNN-based ranking method first generates candidates using handcrafted rules, and then ranks the candidates according to their semantic information modeled by CNN as well as their morphological information. Experiments on two benchmark datasets for biomedical entity normalization show that our proposed CNN-based ranking method outperforms traditional rule-based method with state-of-the-art performance. We propose a CNN architecture that regards biomedical entity normalization as a ranking problem. Comparison results show that semantic information is beneficial to biomedical entity normalization and can be well combined with morphological information in our CNN architecture for further improvement.

/pdf/cnn-based-ranking-for-biomedical-entity-normalization-2995b9no1d.pdf

CNN-based ranking for biomedical entity normalization.

We examine a key task in biomedical text processing, normalization of disorder mentions. We present a multi-pass sieve approach to this task, which has the advantage of simplicity and modularity. Our approach is evaluated on two datasets, one comprising clinical reports and the other comprising biomedical abstracts, achieving state-of-the-art results.

https://www.hlt.utdallas.edu/~vince/papers/acl15s-link.pdf

Sieve-Based Entity Linking for the Biomedical Domain

Medical concept normalization in social media posts with recurrent neural networks.

The number of scientific publications in the literature is steadily growing, containing our knowledge in the biomedical, health, and clinical sciences. Since there is currently no automatic archiving of the obtained results, much of this information remains buried in textual details not readily available for further usage or analysis. For this reason, natural language processing (NLP) and text mining methods are used for information extraction from such publications. In this paper, we review practices for Named Entity Recognition (NER) and Relation Detection (RD), allowing, e.g., to identify interactions between proteins and drugs or genes and diseases. This information can be integrated into networks to summarize large-scale details on a particular biomedical or clinical problem, which is then amenable for easy data management and further analysis. Furthermore, we survey novel deep learning methods that have recently been introduced for such tasks.

/pdf/named-entity-recognition-and-relation-detection-for-2d1hp93ahp.pdf

Named Entity Recognition and Relation Detection for Biomedical Information Extraction

This paper describes Team UWM’s system for the Task 7 of SemEval 2014 that does disorder mention extraction and normalization from clinical text. For the disorder mention extraction (Task A), the system was trained using Conditional Random Fields with features based on words, their POS tags and semantic types, as well as features based on MetaMap matches. For the disorder mention normalization (Task B), variations of disorder mentions were considered whenever exact matches were not found in the training data or in the UMLS. Suitable types of variations for disorder mentions were automatically learned using a new method based on edit distance patterns. Among nineteen participating teams, UWM ranked third in Task A with 0.755 strict F-measure and second in Task B with 0.66 strict accuracy.

/pdf/uwm-disorder-mention-extraction-from-clinical-text-using-206jzjrgqu.pdf

UWM: Disorder Mention Extraction from Clinical Text Using CRFs and Normalization Using Learned Edit Distance Patterns

https://dc.uwm.edu/cgi/viewcontent.cgi?article=1002&context=healthinfo_facart

Learning for clinical named entity recognition without manual annotations

/pdf/disease-name-extraction-from-clinical-text-using-conditional-3ctgh4l2md.pdf

Disease Name Extraction from Clinical Text Using Conditional Random Fields

In this paper the system that was developed by Team UWM for the Task 14 of SemEval 2015 competition is described. Task 14 included two tasks: Task 1 was identification of disorder mentions and their normalization, and Task 2 was identification of the following attributes for disorder mentions: the CUI of the disorder, negation indicator, subject, uncertainty indicator, course, severity, conditional, generic indicator, and body location. For Task 1, an earlier system was applied that uses Conditional Random Fields (CRFs) for disorder recognition and learned edit distance patterns for normalization. Task 2 was implemented by a simple method that finds the attribute terms around the disease mentions by matching them in the training data. Among all participants Team UWM was ranked fourth in Task 1, fourth in Task 2A (over gold-standard mentions) and third in Task 2B (over extracted mentions).

/pdf/uwm-a-simple-baseline-method-for-identifying-attributes-of-4xslmgfez4.pdf

UWM: A Simple Baseline Method for Identifying Attributes of Disease and Disorder Mentions in Clinical Text

Annotating clinical notes manually is very labor-intensive and needs expertise in the area of annotation. Thus annotation is a highly expensive task not only in human resource but also in financial aspects. Moreover mistakes, missed tags, and inconsistency are the common problems with manual annotations. The purpose of this research is to reduce humans as annotation effort for clinical notes, to improve consistency, and to decrease cost of annotation. The aim of this research is to annotate clinical texts to extract biomedical names and terms. In our research Unified Medical Language System (UMLS) is the reference meta thesaurus of names and terms used in biomedical and clinical domains. In this research we have done unsupervised and semi-supervised Named Entity Recognition (NER) through exact matching in UMLS. The data sets that have been used were provided by SemEval 2015 (task 14) natural language processing competition, including 199 clinical notes in training set and 133 notes in test set. The analysis that has been done so far can be divided into two steps: mapping and learning. The first step is to map all terms into UMLS that includes not only unigrams but also n-grams, usually n is 5. To achieve the best results of exact matching, we extracted UMLS terms of diseases and disorders based on semantic groups and mapped each n-gram to that part of UMLS. If there is a match, that is assumed to be a disease or disorder. When there is no match for n-grams (n>=2), to avoid low precisions, we supposed that unigrams must be noun phrases to be nominated as a disease/disorder. With this method we got 60% of f-score, and training files for next process (training CRFs) were generated. The second step involves using Conditional Random Fields (CRFs). The results generated in the first step were used to train the CRF. CRFs learn from training data the general contexts in which named entities occur. Also because of different levels of correctness in training files, we decided to modify training files before using them to train CRFs and to test on test data. Level of correctness means different accuracies of tagging in the data set. Because exact matching is not very accurate, the accuracy in different notes is variable. In some data it is very high and in some of them it is low. This results in an inconsistency in training files. To solve this problem we divided training files into ten groups. The CRF used only one group to be trained and to tag other groups, and results of exact matches and CRFs were combined (logic OR between results of CRF and exact match) together to get the final results. This was done for all other groups as well, and finally applied on test data. These two steps together are known as unsupervised disease named entity recognition, and the results show a difference of 10.3 percent between unsupervised and supervised approaches. By supervised learning we got 73% F-score while we got 62.7% by the proposed unsupervised approach. Another approach that was developed is semi supervised disease named entity recognition that used annotated files generated by unsupervised method and annotated files by human or gold standards. By this method we could improve 73% of F-score, that we got in supervised approach, to 74.2%. In the future some other refinements and extra tasks are going to be done. To improve the results, we are planning to use approximate matching by the process that is called normalization. Normalization means mapping a term in clinical notes to a preferred term in UMLS. These kinds of terms do not have exact matches, thus the way to find exact matches is to use normalization. Moreover we are going to do exact/approximate matching over discontinuous mentions in clinical texts. In these texts there are mentions including disconnected words in a sentence that together form a named entity. This essential step will extract those mentions that could not be extracted by exact match and normalization approaches. The last thing in our plan is to expand our developed system to a less supervised "Biomedical Named Entity Recognition (BNER)" to extract all biomedical and clinical terms. We will do this for other semantic groups in UMLS such as Activities and Behaviors, Anatomy, Devices, Phenomena, etc. Thus developing a less supervised annotating system for clinical notes could generate annotated notes with less cost of manual tagging, more consistent, and accurate enough. By using this approach it is feasible to extract tags of other semantic groups in UMLS, and finally it could be an advanced system to tag all the biomedical and clinical mentions based on semantic groups in UMLS.

Omid Ghiasvand

Papers

UWM: Disorder Mention Extraction from Clinical Text Using CRFs and Normalization Using Learned Edit Distance Patterns

Learning for clinical named entity recognition without manual annotations

Disease Name Extraction from Clinical Text Using Conditional Random Fields

UWM: A Simple Baseline Method for Identifying Attributes of Disease and Disorder Mentions in Clinical Text

Biomedical Named Entity Recognition with less Supervision