Natural language processing algorithms for mapping clinical text fragments onto ontology concepts: a systematic review and recommendations for future studies
TLDR
A list of sixteen recommendations regarding the usage of NLP systems and algorithms, usage of data, evaluation and validation, presentation of results, and generalizability of results was developed and believe will increase the reproducibility and reusability of future studies and NLP algorithms in medicine.Abstract:
Free-text descriptions in electronic health records (EHRs) can be of interest for clinical research and care optimization. However, free text cannot be readily interpreted by a computer and, therefore, has limited value. Natural Language Processing (NLP) algorithms can make free text machine-interpretable by attaching ontology concepts to it. However, implementations of NLP algorithms are not evaluated consistently. Therefore, the objective of this study was to review the current methods used for developing and evaluating NLP algorithms that map clinical text fragments onto ontology concepts. To standardize the evaluation of algorithms and reduce heterogeneity between studies, we propose a list of recommendations. Two reviewers examined publications indexed by Scopus, IEEE, MEDLINE, EMBASE, the ACM Digital Library, and the ACL Anthology. Publications reporting on NLP for mapping clinical text from EHRs to ontology concepts were included. Year, country, setting, objective, evaluation and validation methods, NLP algorithms, terminology systems, dataset size and language, performance measures, reference standard, generalizability, operational use, and source code availability were extracted. The studies’ objectives were categorized by way of induction. These results were used to define recommendations. Two thousand three hundred fifty five unique studies were identified. Two hundred fifty six studies reported on the development of NLP algorithms for mapping free text to ontology concepts. Seventy-seven described development and evaluation. Twenty-two studies did not perform a validation on unseen data and 68 studies did not perform external validation. Of 23 studies that claimed that their algorithm was generalizable, 5 tested this by external validation. A list of sixteen recommendations regarding the usage of NLP systems and algorithms, usage of data, evaluation and validation, presentation of results, and generalizability of results was developed. We found many heterogeneous approaches to the reporting on the development and evaluation of NLP algorithms that map clinical text to ontology concepts. Over one-fourth of the identified publications did not perform an evaluation. In addition, over one-fourth of the included studies did not perform a validation, and 88% did not perform external validation. We believe that our recommendations, alongside an existing reporting standard, will increase the reproducibility and reusability of future studies and NLP algorithms in medicine.read more
Citations
More filters
Journal ArticleDOI
Deep Learning Improved Clinical Target Volume Contouring Quality and Efficiency for Postoperative Radiation Therapy in Non-small Cell Lung Cancer.
Nan Bi,Jingbo Wang,Tao Zhang,Xinyuan Chen,Wenlong Xia,Junjie Miao,Kunpeng Xu,Linfang Wu,Quanrong Fan,Luhua Wang,Yexiong Li,Zongmei Zhou,Jianrong Dai +12 more
TL;DR: Compared with MC, the DLAC is a promising strategy to obtain superior accuracy, consistency, and efficiency for the PORT-CTV in NSCLC.
Journal ArticleDOI
Machine learning based natural language processing of radiology reports in orthopaedic trauma
A. W. Olthof,P Shouche,E M Fennema,Frank F A IJpma,R H C Koolstra,V M A Stirler,P. M. A. van Ooijen,Ludo J. Cornelissen +7 more
TL;DR: In this article, the authors compared different Machine Learning (ML) Natural Language Processing (NLP) methods to classify radiology reports in orthopaedic trauma for the presence of injuries.
Journal ArticleDOI
Evaluation of Automatic Segmentation Model With Dosimetric Metrics for Radiotherapy of Esophageal Cancer.
TL;DR: The automatic delineation could substitute the manual delineation for esophageal cancer radiotherapy planning based on the dosimetric evaluation in this study.
Journal ArticleDOI
Case Report: Utilizing AI and NLP to Assist with Healthcare and Rehabilitation During the COVID-19 Pandemic
Jay Carriere,Hareem Shafi,Katelyn Brehon,Kiran Pohar Manhas,Katie Churchill,Katie Churchill,Chester Ho,Chester Ho,Mahdi Tavakoli +8 more
TL;DR: In this article, Artificial Intelligence and Machine Learning (AI/ML) technologies such as Natural Language Processing (NLP) are used to assist with assessment and rehabilitation for acute and chronic conditions.
Journal ArticleDOI
Improving broad-coverage medical entity linking with semantic type prediction and large-scale datasets.
TL;DR: The authors proposed a semantic type prediction module to alleviate the problem of overgeneration of candidate concepts by filtering out irrelevant candidate concepts based on the predicted semantic type of a mention, which achieved state-of-the-art performance for biomedical text.
References
More filters
Journal ArticleDOI
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement
TL;DR: Moher et al. as mentioned in this paper introduce PRISMA, an update of the QUOROM guidelines for reporting systematic reviews and meta-analyses, which is used in this paper.
Journal Article
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement [in Spanish]
David Moher,Alessandro Liberati,Jennifer Tetzlaff,Douglas G. Altman,Gerd Antes,David C. Atkins,Virginia Barbour,N Barrowman,Jesse A. Berlin,J Clark,Mike Clarke,Deborah J. Cook,Roberto D'Amico,Jonathan J Deeks,Philip J. Devereaux,Kay Dickersin,M Egger,E Ernst,P C Gøtzsche,Jeremy M. Grimshaw,G Guyatt,Julian P T Higgins,Ioannidis Jpa.,Jos Kleijnen,Tom Lang,N Magrini,D McNamee,Lorenzo Moja,C Mulrow,M Napoli,Andrew D Oxman,B Pham,Drummond Rennie,Margaret Sampson,Kenneth F. Schulz,Paul G. Shekelle,David Tovey,Peter Tugwell +37 more
Journal ArticleDOI
A translation approach to portable ontology specifications
TL;DR: This paper describes a mechanism for defining ontologies that are portable over representation systems, basing Ontolingua itself on an ontology of domain-independent, representational idioms.
Journal ArticleDOI
The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies
E von Elm,Douglas G. Altman,Matthias Egger,Matthias Egger,Stuart J. Pocock,Peter C Gøtzsche,Jan P. Vandenbroucke +6 more
TL;DR: The STROBE Statement is a checklist of items that should be addressed in articles reporting on the 3 main study designs of analytical epidemiology: cohort, casecontrol, and cross-sectional studies; these recommendations are not prescriptions for designing or conducting studies.
Journal ArticleDOI
Rayyan-a web and mobile app for systematic reviews.
TL;DR: The strongest features of the app, identified and reported in user feedback, were its ability to help in screening and collaboration as well as the time savings it affords to users.