A Comparative Analysis of Active Learning for Biomedical Text Mining
15 Mar 2021-Vol. 4, Iss: 1, pp 23
TL;DR: Experiments show that AL has the potential to significantly reducing the cost of manual labelling, and AL-assisted pre-annotations accelerates the de novo annotation process with less annotation time required.
Abstract: An enormous amount of clinical free-text information, such as pathology reports, progress reports, clinical notes and discharge summaries have been collected at hospitals and medical care clinics. These data provide an opportunity of developing many useful machine learning applications if the data could be transferred into a learn-able structure with appropriate labels for supervised learning. The annotation of this data has to be performed by qualified clinical experts, hence, limiting the use of this data due to the high cost of annotation. An underutilised technique of machine learning that can label new data called active learning (AL) is a promising candidate to address the high cost of the label the data. AL has been successfully applied to labelling speech recognition and text classification, however, there is a lack of literature investigating its use for clinical purposes. We performed a comparative investigation of various AL techniques using ML and deep learning (DL)-based strategies on three unique biomedical datasets. We investigated random sampling (RS), least confidence (LC), informative diversity and density (IDD), margin and maximum representativeness-diversity (MRD) AL query strategies. Our experiments show that AL has the potential to significantly reducing the cost of manual labelling. Furthermore, pre-labelling performed using AL expediates the labelling process by reducing the time required for labelling.
Citations
More filters
Posted Content•
TL;DR: A variety of text representation methods, and model designs have blossomed in the context of NLP, including SOTA LMs are described, which can transform large volumes of text into effective vector representations capturing the same semantic information.
Abstract: Word representation has always been an important research area in the history of natural language processing (NLP). Understanding such complex text data is imperative, given that it is rich in information and can be used widely across various applications. In this survey, we explore different word representation models and its power of expression, from the classical to modern-day state-of-the-art word representation language models (LMS). We describe a variety of text representation methods, and model designs have blossomed in the context of NLP, including SOTA LMs. These models can transform large volumes of text into effective vector representations capturing the same semantic information. Further, such representations can be utilized by various machine learning (ML) algorithms for a variety of NLP related tasks. In the end, this survey briefly discusses the commonly used ML and DL based classifiers, evaluation metrics and the applications of these word embeddings in different NLP tasks.
52Â citations
Cites methods from "A Comparative Analysis of Active Le..."
...[121], and both models improved the classification results obtained by using BoW features....
[...]
30 Jun 2021
TL;DR: For a survey of word representation models and its power of expression, from the classical to modern-day state-of-the-art word representation language models (LMS), see as mentioned in this paper.
Abstract: Word representation has always been an important research area in the history of natural language processing (NLP). Understanding such complex text data is imperative, given that it is rich in information and can be used widely across various applications. In this survey, we explore different word representation models and its power of expression, from the classical to modern-day state-of-the-art word representation language models (LMS). We describe a variety of text representation methods, and model designs have blossomed in the context of NLP, including SOTA LMs. These models can transform large volumes of text into effective vector representations capturing the same semantic information. Further, such representations can be utilized by various machine learning (ML) algorithms for a variety of NLP-related tasks. In the end, this survey briefly discusses the commonly used ML- and DL-based classifiers, evaluation metrics, and the applications of these word embeddings in different NLP tasks.
46Â citations
TL;DR: In this paper, the authors developed machine learning models using selective key pathological categories to identify clinical test attributes that will aid in accurate early diagnosis of chronic kidney disease (CKD).
Abstract: Objective: Chronic kidney disease (CKD) is a major public health concern worldwide. High costs of late-stage diagnosis and insufficient testing facilities can contribute to high morbidity and mortality rates in CKD patients, particularly in less developed countries. Thus, early diagnosis aided by vital parameter analytics using affordable computer-aided diagnosis could not only reduce diagnosis costs but improve patient management and outcomes. Methods: In this study, we developed machine learning models using selective key pathological categories to identify clinical test attributes that will aid in accurate early diagnosis of CKD. Such an approach will save time and costs for diagnostic screening. We have also evaluated the performance of several classifiers with k-fold cross-validation on optimized datasets derived using these selected clinical test attributes. Results: Our results suggest that the optimized datasets with important attributes perform well in diagnosis of CKD using our proposed machine learning models. Furthermore, we evaluated clinical test attributes based on urine and blood tests along with clinical parameters that have low costs of acquisition. The predictive models with the optimized and pathologically categorized attributes set yielded high levels of CKD diagnosis accuracy with random forest (RF) classifier being the best performing. Conclusions: Our machine learning approach has yielded effective predictive analytics for CKD screening which can be developed as a resource to facilitate improved CKD screening for enhanced and timely treatment plans.
25Â citations
TL;DR: In this article, the authors presented a complete predictive model by combining consecutive transcriptomic data preprocessing procedures, followed by the proposed KmerFIDF method as a feature extraction method and linear discriminant analysis for dimensionality reduction.
Abstract: Multiple sclerosis is an autoimmune disease that causes psychological impacts and severe physical disabilities, including motor disabilities and partial blindness. This work introduces an early detection method for multiple sclerosis disease by analyzing transcriptomic microRNA data. By transforming this phenotype classification problem into a text mining problem, multiple sclerosis disease biomarkers can be obtained. To our knowledge, text mining methods have not been introduced previously in transcriptomic data analysis of multiple sclerosis disease. Hence, this work presents a complete predictive model by combining consecutive transcriptomic data preprocessing procedures, followed by the proposed KmerFIDF method as a feature extraction method and linear discriminant analysis for dimensionality reduction. Predictive machine learning methods can then be obtained accordingly. This study describes experimental work on a transcriptomic dataset of noncoding microRNA sequences denoted from relapsing-remitting multiple sclerosis patients before fingolimod treatment and after six consecutive months of treatment. The experimental results of the predictive methods with the proposed model report sensitivity, specificity, F1-score, and average accuracy scores of 96.4, 96.47, 95.6, and 97% with random forest, 92.89, 92.78, 93.2, and 94% with support vector machine and 91.95, 92.2, 93.1, and 94% with logistic regression, respectively. These promising results support the introduced model and the proposed KmerFIDF method in transcriptomic data analysis. Moreover, comparative experiments are conducted with two referenced studies. The obtained results show that the average reported accuracy scores of the proposed model outperform the referenced literature work.
7Â citations
20 Jan 2022
TL;DR: In this article , lower built-in self-test (LBIS T) mechanism is used to design a microprocessor and the proposed methodology is giving performance measure like power efficiency 97.5% and area had been attained.
Abstract: The major VLSI circuits like sequential circuits, linear chips and op amps are very important elements to provide many logic functions. Today's competitive devices like cell phone, tabs and note pads are most prominent and those are used to get function the 5G related operations. In this work lower built-in self-test (LBIS T) mechanism is used to designing a microprocessor. The proposed methodology is giving performance measure like power efficiency 97.5 % , improvement of delay is 2.5% and 32% development of area had been attained. This methodology attains more outperformance and compete with present technology. The proposed equipment and execution for our approach requiring a constrained range overhead (lower than 3% power) over conventional LBIS T.
7Â citations
References
More filters
Book•
01 Jan 1995TL;DR: Setting of the learning problem consistency of learning processes bounds on the rate of convergence ofLearning processes controlling the generalization ability of learning process constructing learning algorithms what is important in learning theory?
Abstract: Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing learning algorithms what is important in learning theory?.
40,147Â citations
Posted Content•
TL;DR: This paper proposed two novel model architectures for computing continuous vector representations of words from very large data sets, and the quality of these representations is measured in a word similarity task and the results are compared to the previously best performing techniques based on different types of neural networks.
Abstract: We propose two novel model architectures for computing continuous vector representations of words from very large data sets. The quality of these representations is measured in a word similarity task, and the results are compared to the previously best performing techniques based on different types of neural networks. We observe large improvements in accuracy at much lower computational cost, i.e. it takes less than a day to learn high quality word vectors from a 1.6 billion words data set. Furthermore, we show that these vectors provide state-of-the-art performance on our test set for measuring syntactic and semantic word similarities.
20,077Â citations
Posted Content•
TL;DR: In this paper, the authors propose to use a soft-searching model to find the parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly.
Abstract: Neural machine translation is a recently proposed approach to machine translation. Unlike the traditional statistical machine translation, the neural machine translation aims at building a single neural network that can be jointly tuned to maximize the translation performance. The models proposed recently for neural machine translation often belong to a family of encoder-decoders and consists of an encoder that encodes a source sentence into a fixed-length vector from which a decoder generates a translation. In this paper, we conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and propose to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly. With this new approach, we achieve a translation performance comparable to the existing state-of-the-art phrase-based system on the task of English-to-French translation. Furthermore, qualitative analysis reveals that the (soft-)alignments found by the model agree well with our intuition.
14,077Â citations
Proceedings Article•
28 Jun 2001TL;DR: This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
Abstract: We present conditional random fields , a framework for building probabilistic models to segment and label sequence data. Conditional random fields offer several advantages over hidden Markov models and stochastic grammars for such tasks, including the ability to relax strong independence assumptions made in those models. Conditional random fields also avoid a fundamental limitation of maximum entropy Markov models (MEMMs) and other discriminative Markov models based on directed graphical models, which can be biased towards states with few successor states. We present iterative parameter estimation algorithms for conditional random fields and compare the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
13,190Â citations
TL;DR: Recent work in the area of unsupervised feature learning and deep learning is reviewed, covering advances in probabilistic models, autoencoders, manifold learning, and deep networks.
Abstract: The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, autoencoders, manifold learning, and deep networks. This motivates longer term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation, and manifold learning.
11,201Â citations