scispace - formally typeset
Search or ask a question

Showing papers on "False positive paradox published in 2022"


Journal ArticleDOI
01 Apr 2022-Science
TL;DR: The T2T-CHM13 reference as discussed by the authors has been shown to universally improve read mapping and variant calling for 3202 and 17 globally diverse samples sequenced with short and long reads, respectively.
Abstract: Compared to its predecessors, the Telomere-to-Telomere CHM13 genome adds nearly 200 million base pairs of sequence, corrects thousands of structural errors, and unlocks the most complex regions of the human genome for clinical and functional study. We show how this reference universally improves read mapping and variant calling for 3202 and 17 globally diverse samples sequenced with short and long reads, respectively. We identify hundreds of thousands of variants per sample in previously unresolved regions, showcasing the promise of the T2T-CHM13 reference for evolutionary and biomedical discovery. Simultaneously, this reference eliminates tens of thousands of spurious variants per sample, including reduction of false positives in 269 medically relevant genes by up to a factor of 12. Because of these improvements in variant discovery coupled with population and functional genomic resources, T2T-CHM13 is positioned to replace GRCh38 as the prevailing reference for human genetics.

80 citations


Journal ArticleDOI
TL;DR: In this paper , the authors used permutation analysis to identify differentially expressed genes between two conditions using human population RNA-seq samples, and they found that two popular bioinformatics methods, DESeq2 and edgeR, have unexpectedly high false discovery rates.
Abstract: When identifying differentially expressed genes between two conditions using human population RNA-seq samples, we found a phenomenon by permutation analysis: two popular bioinformatics methods, DESeq2 and edgeR, have unexpectedly high false discovery rates. Expanding the analysis to limma-voom, NOISeq, dearseq, and Wilcoxon rank-sum test, we found that FDR control is often failed except for the Wilcoxon rank-sum test. Particularly, the actual FDRs of DESeq2 and edgeR sometimes exceed 20% when the target FDR is 5%. Based on these results, for population-level RNA-seq studies with large sample sizes, we recommend the Wilcoxon rank-sum test.

50 citations


Journal ArticleDOI
TL;DR: In this article , the authors used permutation analysis to identify differentially expressed genes between two conditions using human population RNA-seq samples, and they found that two popular bioinformatics methods, DESeq2 and edgeR, have unexpectedly high false discovery rates.
Abstract: When identifying differentially expressed genes between two conditions using human population RNA-seq samples, we found a phenomenon by permutation analysis: two popular bioinformatics methods, DESeq2 and edgeR, have unexpectedly high false discovery rates. Expanding the analysis to limma-voom, NOISeq, dearseq, and Wilcoxon rank-sum test, we found that FDR control is often failed except for the Wilcoxon rank-sum test. Particularly, the actual FDRs of DESeq2 and edgeR sometimes exceed 20% when the target FDR is 5%. Based on these results, for population-level RNA-seq studies with large sample sizes, we recommend the Wilcoxon rank-sum test.

41 citations


Journal ArticleDOI
TL;DR: Genome in a Bottle benchmarks are widely used to help validate clinical sequencing pipelines and develop variant calling and sequencing methods as mentioned in this paper , which includes more than 300,000 SNVs and 50,000 insertions or deletions (indels) and include 16% more exonic variants, many in challenging, clinically relevant genes not covered previously.
Abstract: Genome in a Bottle benchmarks are widely used to help validate clinical sequencing pipelines and develop variant calling and sequencing methods. Here we use accurate linked and long reads to expand benchmarks in 7 samples to include difficult-to-map regions and segmental duplications that are challenging for short reads. These benchmarks add more than 300,000 SNVs and 50,000 insertions or deletions (indels) and include 16% more exonic variants, many in challenging, clinically relevant genes not covered previously, such as PMS2. For HG002, we include 92% of the autosomal GRCh38 assembly while excluding regions problematic for benchmarking small variants, such as copy number variants, that should not have been in the previous version, which included 85% of GRCh38. It identifies eight times more false negatives in a short read variant call set relative to our previous benchmark. We demonstrate that this benchmark reliably identifies false positives and false negatives across technologies, enabling ongoing methods development.

40 citations


Journal ArticleDOI
TL;DR: In this article , a systematic investigation of existing DL-based vulnerability prediction approaches reveals that existing DLbased approaches suffer from challenges with the training data (e.g., data duplication, unrealistic distribution of vulnerable classes, etc.) and with the model choices (i.e., simple token-based models).
Abstract: Automated detection of software vulnerabilities is a fundamental problem in software security. Existing program analysis techniques either suffer from high false positives or false negatives. Recent progress in Deep Learning (DL) has resulted in a surge of interest in applying DL for automated vulnerability detection. Several recent studies have demonstrated promising results achieving an accuracy of up to 95 percent at detecting vulnerabilities. In this paper, we ask, “how well do the state-of-the-art DL-based techniques perform in a real-world vulnerability prediction scenario?” To our surprise, we find that their performance drops by more than 50 percent. A systematic investigation of what causes such precipitous performance drop reveals that existing DL-based vulnerability prediction approaches suffer from challenges with the training data (e.g., data duplication, unrealistic distribution of vulnerable classes, etc.) and with the model choices (e.g., simple token-based models). As a result, these approaches often do not learn features related to the actual cause of the vulnerabilities. Instead, they learn unrelated artifacts from the dataset (e.g., specific variable/function names, etc.). Leveraging these empirical findings, we demonstrate how a more principled approach to data collection and model design, based on realistic settings of vulnerability prediction, can lead to better solutions. The resulting tools perform significantly better than the studied baseline—up to 33.57 percent boost in precision and 128.38 percent boost in recall compared to the best performing model in the literature. Overall, this paper elucidates existing DL-based vulnerability prediction systems’ potential issues and draws a roadmap for future DL-based vulnerability prediction research.

36 citations


Journal ArticleDOI
TL;DR: This work presents the development of an ADAS (advanced driving assistance system) focused on driver drowsiness detection, whose objective is to alert drivers of their drowsy state to avoid road traffic accidents.
Abstract: This work presents the development of an ADAS (advanced driving assistance system) focused on driver drowsiness detection, whose objective is to alert drivers of their drowsy state to avoid road traffic accidents. In a driving environment, it is necessary that fatigue detection is performed in a non-intrusive way, and that the driver is not bothered with alarms when he or she is not drowsy. Our approach to this open problem uses sequences of images that are 60 s long and are recorded in such a way that the subject’s face is visible. To detect whether the driver shows symptoms of drowsiness or not, two alternative solutions are developed, focusing on the minimization of false positives. The first alternative uses a recurrent and convolutional neural network, while the second one uses deep learning techniques to extract numeric features from images, which are introduced into a fuzzy logic-based system afterwards. The accuracy obtained by both systems is similar: around 65% accuracy over training data, and 60% accuracy on test data. However, the fuzzy logic-based system stands out because it avoids raising false alarms and reaches a specificity (proportion of videos in which the driver is not drowsy that are correctly classified) of 93%. Although the obtained results do not achieve very satisfactory rates, the proposals presented in this work are promising and can be considered a solid baseline for future works.

30 citations


Journal ArticleDOI
TL;DR: In this paper , an approach that uses an expectation-maximization algorithm to generate taxonomic abundance profiles from full-length 16S rRNA reads is presented. But, this method is not optimized for the increased read length and error rate often observed in long read data.
Abstract: 16S ribosomal RNA-based analysis is the established standard for elucidating the composition of microbial communities. While short-read 16S rRNA analyses are largely confined to genus-level resolution at best, given that only a portion of the gene is sequenced, full-length 16S rRNA gene amplicon sequences have the potential to provide species-level accuracy. However, existing taxonomic identification algorithms are not optimized for the increased read length and error rate often observed in long-read data. Here we present Emu, an approach that uses an expectation-maximization algorithm to generate taxonomic abundance profiles from full-length 16S rRNA reads. Results produced from simulated datasets and mock communities show that Emu is capable of accurate microbial community profiling while obtaining fewer false positives and false negatives than alternative methods. Additionally, we illustrate a real-world application of Emu by comparing clinical sample composition estimates generated by an established whole-genome shotgun sequencing workflow with those returned by full-length 16S rRNA gene sequences processed with Emu.

30 citations


Journal ArticleDOI
TL;DR: It is demonstrated that RoBMA finds evidence for the absence of publication bias in Registered Replication Reports and reliably avoids false positives and is relatively robust to model misspecification and simulations show that it outperforms existing methods.
Abstract: Meta-analysis is an important quantitative tool for cumulative science, but its application is frustrated by publication bias. In order to test and adjust for publication bias, we extend model-averaged Bayesian meta-analysis with selection models. The resulting robust Bayesian meta-analysis (RoBMA) methodology does not require all-or-none decisions about the presence of publication bias, can quantify evidence in favor of the absence of publication bias, and performs well under high heterogeneity. By model-averaging over a set of 12 models, RoBMA is relatively robust to model misspecification and simulations show that it outperforms existing methods. We demonstrate that RoBMA finds evidence for the absence of publication bias in Registered Replication Reports and reliably avoids false positives. We provide an implementation in R so that researchers can easily use the new methodology in practice. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

29 citations


Journal ArticleDOI
TL;DR: The ability of AlphaFold to predict which peptides and proteins interact as well as its accuracy in modeling the resulting interaction complexes are benchmarked against established methods in the fields of peptide-protein interaction prediction and modeling.
Abstract: Protein interactions are key in vital biological process. In many cases, particularly often in regulation, this interaction is between a protein and a shorter peptide fragment. Such peptides are often part of larger disordered regions of other proteins. The flexible nature of peptides enable rapid, yet specific, regulation of important functions in the cell, such as the cell life-cycle. Because of this, understanding the molecular details of these interactions are crucial to understand and alter their function, and many specialized computational methods have been developed to study them. The recent release of AlphaFold and AlphaFold-Multimer has caused a leap in accuracy for computational modeling of proteins. In this study, the ability of AlphaFold to predict which peptides and proteins interact as well as its accuracy in modeling the resulting interaction complexes are benchmarked against established methods in the fields of peptide-protein interaction prediction and modeling. We find that AlphaFold-Multimer consistently produces predicted interaction complexes with a median DockQ of 0.47 for all 112 complexes investigated. Additionally, it can be used to separate interacting from non-interacting pairs of peptides and proteins with ROC-AUC and PR-AUC of 0.78 and 0.61, respectively, best among the method benchmarked. However, the most interestingly result is the possibility to improve AlphaFold by enabling dropout at inference to sample a wider part of the conformational space. This improves the median DockQ from 0.47 to 0.56 for rank 1 and the median best DockQ improves from 0.58 to 0.72. This scheme of generating more structures with AlphaFold should be generally useful for many application involving multiple states, flexible regions and disorder.

27 citations


Journal ArticleDOI
29 Apr 2022-PeerJ
TL;DR: Experimental results prove that the proposed improved firefly algorithm has significant potential in tackling machine learning hyper-parameters optimisation challenge and that it can be used for improving classification accuracy and average precision of network intrusion detection systems.
Abstract: The research proposed in this article presents a novel improved version of the widely adopted firefly algorithm and its application for tuning and optimising XGBoost classifier hyper-parameters for network intrusion detection. One of the greatest issues in the domain of network intrusion detection systems are relatively high false positives and false negatives rates. In the proposed study, by using XGBoost classifier optimised with improved firefly algorithm, this challenge is addressed. Based on the established practice from the modern literature, the proposed improved firefly algorithm was first validated on 28 well-known CEC2013 benchmark instances a comparative analysis with the original firefly algorithm and other state-of-the-art metaheuristics was conducted. Afterwards, the devised method was adopted and tested for XGBoost hyper-parameters optimisation and the tuned classifier was tested on the widely used benchmarking NSL-KDD dataset and more recent USNW-NB15 dataset for network intrusion detection. Obtained experimental results prove that the proposed metaheuristics has significant potential in tackling machine learning hyper-parameters optimisation challenge and that it can be used for improving classification accuracy and average precision of network intrusion detection systems.

25 citations


Journal ArticleDOI
TL;DR: In this paper , the authors developed BreastScreening-AI within two scenarios for the classification of multimodal images: (1) Clinician-only; and (2) clinician-AI.

Journal ArticleDOI
TL;DR: In this article , the authors compared the accuracy of 6 NITs between patients with or without Type 2 Diabetes mellitus (T2DM), and explained any differences, and adapted diagnostic algorithms for clinical practice accordingly.


Journal ArticleDOI
07 Mar 2022-Machines
TL;DR: In this article , the impact of the illumination level on the quantitative indicators of mechanical damage of the rolled strip is investigated, and the obtained images of defects at light levels in the range of 2-800 lx were recognized by a neural network model based on the U-net architecture with a decoder based on ResNet152.
Abstract: The impact of the illumination level on the quantitative indicators of mechanical damage of the rolled strip is investigated. To do so, a physical model experiment was conducted in the laboratory. The obtained images of defects at light levels in the range of 2–800 lx were recognized by a neural network model based on the U-net architecture with a decoder based on ResNet152. Two levels of illumination were identified, at which the total area of recognized defects increased: 50 lx and 300 lx. A quantitative assessment of the overall accuracy of defect recognition was conducted on the basis of comparison with data from images marked by an expert. The best recognition result (with Dice similarity coefficient DSC = 0.89) was obtained for the illumination of 300 lx. At lower light levels (less than 200 lx), some of the damage remained unrecognized. At high light levels (higher than 500 lx), a decrease in DSC was observed, mainly due to the fact that the surface objects are better visible and the recognized fragments become wider. In addition, more false-positives fragments were recognized. The obtained results are valuable for further adjustment of industrial systems for diagnosing technological defects on rolled metal strips.

Proceedings ArticleDOI
01 Jun 2022
TL;DR: SoftGroup as discussed by the authors performs bottom-up soft grouping followed by top-down refinement to mitigate the problems stemming from semantic prediction errors and suppresses false positive instances by learning to categorize them as background.
Abstract: Existing state-of-the-art 3D instance segmentation methods perform semantic segmentation followed by grouping. The hard predictions are made when performing semantic segmentation such that each point is associated with a single class. However, the errors stemming from hard decision propagate into grouping that results in (1) low overlaps between the predicted instance with the ground truth and (2) substantial false positives. To address the aforementioned problems, this paper proposes a 3D instance segmentation method referred to as SoftGroup by performing bottom-up soft grouping followed by top-down refinement. SoftGroup allows each point to be associated with multiple classes to mitigate the problems stemming from semantic prediction errors and suppresses false positive instances by learning to categorize them as background. Experimental results on different datasets and multiple evaluation metrics demonstrate the efficacy of SoftGroup. Its performance surpasses the strongest prior method by a significant margin of $+6.2\%$ on the ScanNet v2 hidden test set and $+6.8\%$ on S3DIS Area 5 in terms of $AP_{50}$ . Soft-Group is also fast, running at 345ms per scan with a sin-gle Titan X on ScanNet v2 dataset. The source code and trained models for both datasets are available at https://github.com/thangvubk/SoftGroup.git.

Journal ArticleDOI
TL;DR: Wang et al. as mentioned in this paper developed a deep learning-based system named ENDOANGEL-LD (lesion detection) to assist in detecting all focal gastric lesions and predicting neoplasms by WLE.

Journal ArticleDOI
TL;DR: In this article , the authors use simulations to demonstrate that the analysis approaches commonly used to test for rhythmic oscillations generate false positives in the presence of aperiodic temporal structure.
Abstract: The neural and perceptual effects of attention were traditionally assumed to be sustained over time, but recent work suggests that covert attention rhythmically switches between objects at 3-8 Hz. Here I use simulations to demonstrate that the analysis approaches commonly used to test for rhythmic oscillations generate false positives in the presence of aperiodic temporal structure. I then propose two alternative analyses that are better able to discriminate between periodic and aperiodic structure in time series. Finally, I apply these alternative analyses to published datasets and find no evidence for behavioural rhythms in attentional switching after accounting for aperiodic temporal structure. The techniques presented here will help clarify the periodic and aperiodic dynamics of perception and of cognition more broadly.

Proceedings ArticleDOI
05 Mar 2022
TL;DR: MVD, a statement-level Memory-related Vulnerability Detection approach based on flow-sensitive graph neural networks (FS-GNN), achieves better detection accuracy, outperforming both state-of-the-art DL-based and static analysis-based approaches.
Abstract: Memory-related vulnerabilities constitute severe threats to the security of modern software. Despite the success of deep learning-based approaches to generic vulnerability detection, they are still limited by the underutilization of flow information when applied for detecting memory-related vulnerabilities, leading to high false positives. In this paper, we propose MVD, a statement-level Memory-related Vulnerability Detection approach based on flow-sensitive graph neural networks (FS-GNN). FS-GNN is employed to jointly embed both unstructured information (i.e., source code) and structured information (i.e., control- and data-flow) to capture implicit memory-related vulnerability patterns. We evaluate MVD on the dataset which contains 4,353 real-world memory-related vulnerabilities, and compare our approach with three state-of-the-art deep learning-based approaches as well as five popular static analysis-based memory detectors. The experiment results show that MVD achieves better detection accuracy, outperforming both state-of-the-art DL-based and static analysis-based approaches. Furthermore, MVD makes a great trade-off between accuracy and efficiency.

Journal ArticleDOI
TL;DR: VulDeeLocator as discussed by the authors is a deep learning-based location-based vulnerability detector that can simultaneously achieve a high detection capability and a high locating precision, dubbed Vulnerability Deep Learning-based Locator.
Abstract: Automatically detecting software vulnerabilities is an important problem that has attracted much attention from the academic research community. However, existing vulnerability detectors still cannot achieve the vulnerability detection capability and the locating precision that would warrant their adoption for real-world use. In this paper, we present a vulnerability detector that can simultaneously achieve a high detection capability and a high locating precision, dubbed Vulnerability Deep learning-based Locator (VulDeeLocator). In the course of designing VulDeeLocator, we encounter difficulties including how to accommodate semantic relations between the definitions of types as well as macros and their uses across files, how to accommodate accurate control flows and variable define-use relations, and how to achieve high locating precision. We solve these difficulties by using two innovative ideas: (i) leveraging intermediate code to accommodate extra semantic information, and (ii) using the notion of granularity refinement to pin down locations of vulnerabilities. When applied to 200 files randomly selected from three real-world software products, VulDeeLocator detects 18 confirmed vulnerabilities (i.e., true-positives). Among them, 16 vulnerabilities correspond to known vulnerabilities; the other two are not reported in the National Vulnerability Database (NVD) but have been "silently" patched by the vendor of Libav when releasing newer versions.

Journal ArticleDOI
TL;DR: Zhang et al. as mentioned in this paper proposed to feed the informative feature representation from a joint-attention feature fusion network into adaptive non-maximum suppression (NMS) for a comprehensive performance enhancement.
Abstract: Attention mechanisms and Non-Maximum Suppression (NMS) have proven to be effective components in object detection. However, feature fusion of different scales and layers based on a single attention mechanism cannot always yield gratifying performance, and may introduce redundant information that makes the results worse than expected. NMS methods, on the other hand, generally face the single-constant threshold dilemma, namely, a lower threshold leads to the miss of highly overlapped instance objects while a higher one brings in more false positives. Therefore, how to optimize different dimensions of correlation in feature mapping and how to adaptively set the NMS threshold still hinder effective object detection. While independently addressing each will cause suboptimal detection, this paper proposes to feed the informative feature representation from a joint-attention feature fusion network into adaptive NMS for a comprehensive performance enhancement. Specifically, we embed two types of attention modules in a three-level Feature Pyramid Network (FPN): the channel-attention module is adopted for enhanced feature representation by re-evaluating relationships between channels from a global perspective; the position-attention module is used to exploit the correlation between features to discover rich contextual feature information. Furthermore, we develop dual-adaptive NMS to dynamically adjust the suppression thresholds according to instance objects density, namely, the threshold rises as instance objects gather and decays when objects appear sparsely. The proposed method is evaluated on the COCO dataset and extensive experimental results demonstrate its superior performance compared with existing methods.

Proceedings ArticleDOI
07 Feb 2022
TL;DR: This work shows that hard positives can give similar response maps to the corresponding pairs in visual scenes, and incorporates these hard positives by adding their response maps into a contrastive learning objective directly.
Abstract: The objective of this work is to localize the sound sources in visual scenes. Existing audio-visual works employ contrastive learning by assigning corresponding audio-visual pairs from the same source as positives while randomly mismatched pairs as negatives. However, these negative pairs may contain semantically matched audio-visual information. Thus, these semantically correlated pairs, "hard positives", are mistakenly grouped as negatives. Our key contribution is showing that hard positives can give similar response maps to the corresponding pairs. Our approach incorporates these hard positives by adding their response maps into a contrastive learning objective directly. We demonstrate the effectiveness of our approach on VGG-SS and SoundNet-Flickr test sets, showing favorable performance to the state-of-the-art methods.

Journal ArticleDOI
TL;DR: An updated and enhanced deep learning workflow to classify 50x50 um tiled image patches as TIL positive or negative based on the presence of 2 or more TILs in gigapixel whole slide images from the Cancer Genome Atlas (TCGA).
Abstract: The role of tumor infiltrating lymphocytes (TILs) as a biomarker to predict disease progression and clinical outcomes has generated tremendous interest in translational cancer research. We present an updated and enhanced deep learning workflow to classify 50x50 um tiled image patches (100x100 pixels at 20x magnification) as TIL positive or negative based on the presence of 2 or more TILs in gigapixel whole slide images (WSIs) from the Cancer Genome Atlas (TCGA). This workflow generates TIL maps to study the abundance and spatial distribution of TILs in 23 different types of cancer. We trained three state-of-the-art, popular convolutional neural network (CNN) architectures (namely VGG16, Inception-V4, and ResNet-34) with a large volume of training data, which combined manual annotations from pathologists (strong annotations) and computer-generated labels from our previously reported first-generation TIL model for 13 cancer types (model-generated annotations). Specifically, this training dataset contains TIL positive and negative patches from cancers in additional organ sites and curated data to help improve algorithmic performance by decreasing known false positives and false negatives. Our new TIL workflow also incorporates automated thresholding to convert model predictions into binary classifications to generate TIL maps. The new TIL models all achieve better performance with improvements of up to 13% in accuracy and 15% in F-score. We report these new TIL models and a curated dataset of TIL maps, referred to as TIL-Maps-23, for 7983 WSIs spanning 23 types of cancer with complex and diverse visual appearances, which will be publicly available along with the code to evaluate performance. Code Available at: https://github.com/ShahiraAbousamra/til_classification.

Proceedings ArticleDOI
28 Feb 2022
TL;DR: Amalfi, a machine-learning based approach for automatically detecting potentially malicious packages comprised of three complementary techniques, improves on the state of the art in that it is lightweight, requiring only a few seconds per package to extract features and run the classifiers, and gives good results in practice.
Abstract: The npm registry is one of the pillars of the JavaScript and Type-Script ecosystems, hosting over 1.7 million packages ranging from simple utility libraries to complex frameworks and entire applications. Each day, developers publish tens of thousands of updates as well as hundreds of new packages. Due to the overwhelming popularity of npm, it has become a prime target for malicious actors, who publish new packages or compromise existing packages to introduce malware that tampers with or exfiltrates sensitive data from users who install either these packages or any package that (transitively) depends on them. Defending against such attacks is essential to maintaining the integrity of the software supply chain, but the sheer volume of package updates makes comprehensive manual review infeasible. We present Amalfi, a machine-learning based approach for automatically detecting potentially malicious packages comprised of three complementary techniques. We start with classifiers trained on known examples of malicious and benign packages. If a package is flagged as malicious by a classifier, we then check whether it includes metadata about its source repository, and if so whether the package can be reproduced from its source code. Packages that are reproducible from source are not usually malicious, so this step allows us to weed out false positives. Finally, we also employ a simple textual clone-detection technique to identify copies of malicious packages that may have been missed by the classifiers, reducing the number of false negatives. Amalfi improves on the state of the art in that it is lightweight, requiring only a few seconds per package to extract features and run the classifiers, and gives good results in practice: running it on 96287 package versions published over the course of one week, we were able to identify 95 previously unknown malware samples, with a manageable number of false positives.

Journal ArticleDOI
TL;DR: Results of DNA extracted from pure cultures indicate high specificity and sensitivity to as little as 2.5 ng/µL E. coli O157 DNA, which demonstrates the capabilities of this GNP biosensor for low-cost and rapid foodborne pathogen detection.

Journal ArticleDOI
TL;DR: Anomaly-based machine learning-enabled intrusion detection systems (AML-IDSs) show low performance and prediction accuracy while detecting intrusions in the Internet of Things (IoT) than that of deep learning-based IDSs as discussed by the authors .
Abstract: Anomaly-based machine learning-enabled intrusion detection systems (AML-IDSs) show low performance and prediction accuracy while detecting intrusions in the Internet of Things (IoT) than that of deep learning-based intrusion detection systems (DL-IDSs). In particular, AML-IDS that employ low complexity models for IoT, such as the principal component machine (PCA) method and the one-class support vector machine (1-SVM) method, are inefficient in detecting intrusions when compared to DL-IDS with the two-class neural network (2-NN) method. PCA and 1-SVM AML-IDS suffer from low detection rates compared to DL-IDS. The size of the data set and the number of features or variants in the data set may influence how well PCA and 1-SVM AML-IDS perform compared to DL-IDS. We attribute the low performance and prediction accuracy of the AML-IDS model to an imbalanced data set, a low similarity index between the training data and testing data, and the use of a single-learner model. The intrinsic limitations of the single-learner model have a direct impact on the accuracy of an intelligent IDS. Also, the dissimilarity between testing data and training data leads to an increasingly high rate of false positives (FPs) in AML-IDS than DL-IDS, which have low false alarms and high predictability. In this article, we examine the use of optimization techniques to enhance the performance of single-learner AML-IDS, such as PCA and 1-SVM AML-IDS models for building efficient, scalable, and distributed intelligent IDS for detecting intrusions in IoT. We evaluate these AML-IDS models by tuning hyperparameters and ensemble learning optimization techniques using the Microsoft Azure ML Studio (AMLS) platform and two data sets containing malicious and benign IoT and industrial IoT (IIoT) network traffic. Furthermore, we present a comparative analysis of AML-IDS models for IoT regarding their performance and predictability.

Journal ArticleDOI
TL;DR: In this paper , an optimized RPA-Cas12a-based platform combined with digital microfluidics (DMF), the RCD platform, was established to achieve the automated, rapid detection of influenza viruses and SARS-CoV-2.
Abstract: Outbreaks of both influenza virus and the novel coronavirus SARS-CoV-2 are serious threats to human health and life. It is very important to establish a rapid, accurate test with large-scale detection potential to prevent the further spread of the epidemic. An optimized RPA-Cas12a-based platform combined with digital microfluidics (DMF), the RCD platform, was established to achieve the automated, rapid detection of influenza viruses and SARS-CoV-2. The probe in the RPA-Cas12a system was optimized to produce maximal fluorescence to increase the amplification signal. The reaction droplets in the platform were all at the microliter level and the detection could be accomplished within 30 min due to the effective mixing of droplets by digital microfluidic technology. The whole process from amplification to recognition is completed in the chip, which reduces the risk of aerosol contamination. One chip can contain multiple detection reaction areas, offering the potential for customized detection. The RCD platform demonstrated a high level of sensitivity, specificity (no false positives or negatives), speed (≤30 min), automation and multiplexing. We also used the RCD platform to detect nucleic acids from influenza patients and COVID-19 patients. The results were consistent with the findings of qPCR. The RCD platform is a one-step, rapid, highly sensitive and specific method with the advantages of digital microfluidic technology, which circumvents the shortcomings of manual operation. The development of the RCD platform provides potential for the isothermal automatic detection of nucleic acids during epidemics.

Journal ArticleDOI
TL;DR: In this paper , a computer aided diagnosis (CAD) system was proposed to detect and classify corona virus using machine learning techniques such as Decision Tree, Support Vector Machine, K-means clustering, and Radial Basis Function.

Proceedings ArticleDOI
12 Feb 2022
TL;DR: The results convey several lessons and provide guidelines for evaluating false alarm detectors and demonstrate limitations in the warning oracle that determines the ground-truth labels, a heuristic comparing warnings in a given revision to a reference revision in the future.
Abstract: Automatic static analysis tools (ASATs), such as Findbugs, have a high false alarm rate. The large number of false alarms produced poses a barrier to adoption. Researchers have proposed the use of machine learning to prune false alarms and present only actionable warnings to developers. The state-of-the-art study has identified a set of “Golden Features” based on metrics computed over the characteristics and history of the file, code, and warning. Recent studies show that machine learning using these features is extremely effective and that they achieve almost perfect performance. We perform a detailed analysis to better understand the strong performance of the “Golden Features”. We found that several studies used an experimental procedure that results in data leakage and data duplication, which are subtle issues with significant implications. Firstly, the ground-truth labels have leaked into features that measure the proportion of actionable warnings in a given context. Secondly, many warnings in the testing dataset appear in the training dataset. Next, we demonstrate limitations in the warning oracle that determines the ground-truth labels, a heuristic comparing warnings in a given revision to a reference revision in the future. We show the choice of reference revision influences the warning distribution. Moreover, the heuristic produces labels that do not agree with human oracles. Hence, the strong performance of these techniques previously seen is overoptimistic of their true performance if adopted in practice. Our results convey several lessons and provide guidelines for evaluating false alarm detectors.

Journal ArticleDOI
TL;DR: RINCE is introduced, a new member in the family of InfoNCE losses that preserves a ranked ordering of positive samples that yields higher classification accuracy, retrieval rates and performs better on out-of-distribution detection than the standardInfoNCE loss.
Abstract: This paper introduces Ranking Info Noise Contrastive Estimation (RINCE), a new member in the family of InfoNCE losses that preserves a ranked ordering of positive samples. In contrast to the standard InfoNCE loss, which requires a strict binary separation of the training pairs into similar and dissimilar samples, RINCE can exploit information about a similarity ranking for learning a corresponding embedding space. We show that the proposed loss function learns favorable embeddings compared to the standard InfoNCE whenever at least noisy ranking information can be obtained or when the definition of positives and negatives is blurry. We demonstrate this for a supervised classification task with additional superclass labels and noisy similarity scores. Furthermore, we show that RINCE can also be applied to unsupervised training with experiments on unsupervised representation learning from videos. In particular, the embedding yields higher classification accuracy, retrieval rates and performs better on out-of-distribution detection than the standard InfoNCE loss.

Journal ArticleDOI
TL;DR: In this paper , the authors proposed a novel glaucoma identification system from fundus images through the deep belief network (DBN) optimized by the elephant-herding optimization (EHO) algorithm.
Abstract: This study proposes a novel glaucoma identification system from fundus images through the deep belief network (DBN) optimized by the elephant-herding optimization (EHO) algorithm. Initially, the input image undergoes the preprocessing steps of noise removal and enhancement processes, followed by optical disc (OD) and optical cup (OC) segmentation and extraction of structural, intensity, and textural features. Most discriminative features are then selected using the ReliefF algorithm and passed to the DBN for classification into glaucomatous or normal. To enhance the classification rate of the DBN, the DBN parameters are fine-tuned by the EHO algorithm. The model has experimented on public and private datasets with 7280 images, which attained a maximum classification rate of 99.4%, 100% specificity, and 99.89% sensitivity. The 10-fold cross validation reduced the misclassification and attained 98.5% accuracy. Investigations proved the efficacy of the proposed method in avoiding bias, dataset variability, and reducing false positives compared to similar works of glaucoma classification. The proposed system can be tested on diverse datasets, aiding in the improved glaucoma diagnosis.