The MinHash algorithm has proven effective for rapidly estimating the resemblance of two genomes or metagenomes. However, this method cannot reliably estimate the containment of a genome within a metagenome. Here, we describe an online algorithm capable of measuring the containment of genomes and proteomes within either assembled or unassembled sequencing read sets. We describe several use cases, including contamination screening and retrospective analysis of metagenomes for novel genome discovery. Using this tool, we provide containment estimates for every NCBI RefSeq genome within every SRA metagenome and demonstrate the identification of a novel polyomavirus species from a public metagenome.

/pdf/mash-screen-high-throughput-sequence-containment-estimation-33mil9nx06.pdf

Mash Screen: high-throughput sequence containment estimation for genome discovery

The plant microbiome is an integral part of the host and increasingly recognized as playing fundamental roles in plant growth and health. Increasing evidence indicates that plant rhizosphere recruits beneficial microbes to the plant to suppress soil-borne pathogens. However, the ecological processes that govern plant microbiome assembly and functions in the below- and aboveground compartments under pathogen invasion are not fully understood. Here, we studied the bacterial and fungal communities associated with 12 compartments (e.g., soils, roots, stems, and fruits) of chili pepper (Capsicum annuum L.) using amplicons (16S and ITS) and metagenomics approaches at the main pepper production sites in China and investigated how Fusarium wilt disease (FWD) affects the assembly, co-occurrence patterns, and ecological functions of plant-associated microbiomes. The amplicon data analyses revealed that FWD affected less on the microbiome of pepper reproductive organs (fruit) than vegetative organs (root and stem), with the strongest impact on the upper stem epidermis. Fungal intra-kingdom networks were less stable and their communities were more sensitive to FWD than the bacterial communities. The analysis of microbial interkingdom network further indicated that FWD destabilized the network and induced the ecological importance of fungal taxa. Although the diseased plants were more susceptible to colonization by other pathogenic fungi, their below- and aboveground compartments can also recruit potential beneficial bacteria. Some of the beneficial bacterial taxa enriched in the diseased plants were also identified as core taxa for plant microbiomes and hub taxa in networks. On the other hand, metagenomic analysis revealed significant enrichment of several functional genes involved in detoxification, biofilm formation, and plant-microbiome signaling pathways (i.e., chemotaxis) in the diseased plants. Together, we demonstrate that a diseased plant could recruit beneficial bacteria and mitigate the changes in reproductive organ microbiome to facilitate host or its offspring survival. The host plants may attract the beneficial microbes through the modulation of plant-microbiome signaling pathways. These findings significantly advance our understanding on plant-microbiome interactions and could provide fundamental and important data for harnessing the plant microbiome in sustainable agriculture.

/pdf/disease-induced-changes-in-plant-microbiome-assembly-and-46ux8yyn8x.pdf

Disease-induced changes in plant microbiome assembly and functional adaptation.

In the past decade, the number of publicly available bacterial genomes has increased dramatically. These genomes have been generated for impactful initiatives, especially in the field of genomic epidemiology (Brown, Dessai, McGarry, & Gerner-Smidt, 2019; Timme et al., 2017). Genomes are sequenced, shared publicly, and subsequently analyzed for phylogenetic relatedness. If two genomes of epidemiological interest are found to be related, further investigation might be prompted. However, comparing the multitudes of genomes for phylogenetic relatedness is computationally expensive and, with large numbers, laborious. Consequently, there are many strategies to reduce the complexity of the data for downstream analysis, especially using nucleotide stretches of length k (kmers).

/pdf/mashtree-a-rapid-comparison-of-whole-genome-sequence-files-qihpx9r7qg.pdf

Mashtree: a rapid comparison of whole genome sequence files

The MinHash algorithm has proven effective for rapidly estimating the resemblance of two genomes or metagenomes. However, this method cannot reliably estimate the containment of a genome within a metagenome. Here we describe an online algorithm capable of measuring the containment of genomes and proteomes within either assembled or unassembled sequencing read sets. We describe several use cases, including contamination screening and retrospective analysis of metagenomes for novel genome discovery. Using this tool, we provide containment estimates for every NCBI RefSeq genome within every SRA metagenome, and demonstrate the identification of a novel polyomavirus species from a public metagenome.

/pdf/mash-screen-high-throughput-sequence-containment-estimation-4j6w5vhxd0.pdf

Mash Screen: High-throughput sequence containment estimation for genome discovery

Background
Accurately identifying single-nucleotide polymorphisms (SNPs) from bacterial sequencing data is an essential requirement for using genomics to track transmission and predict important phenotypes such as antimicrobial resistance. However, most previous performance evaluations of SNP calling have been restricted to eukaryotic (human) data. Additionally, bacterial SNP calling requires choosing an appropriate reference genome to align reads to, which, together with the bioinformatic pipeline, affects the accuracy and completeness of a set of SNP calls obtained. This study evaluates the performance of 209 SNP-calling pipelines using a combination of simulated data from 254 strains of 10 clinically common bacteria and real data from environmentally sourced and genomically diverse isolates within the genera Citrobacter, Enterobacter, Escherichia, and Klebsiella.


Results
We evaluated the performance of 209 SNP-calling pipelines, aligning reads to genomes of the same or a divergent strain. Irrespective of pipeline, a principal determinant of reliable SNP calling was reference genome selection. Across multiple taxa, there was a strong inverse relationship between pipeline sensitivity and precision, and the Mash distance (a proxy for average nucleotide divergence) between reads and reference genome. The effect was especially pronounced for diverse, recombinogenic bacteria such as Escherichia coli but less dominant for clonal species such as Mycobacterium tuberculosis.


Conclusions
The accuracy of SNP calling for a given species is compromised by increasing intra-species diversity. When reads were aligned to the same genome from which they were sequenced, among the highest-performing pipelines was Novoalign/GATK. By contrast, when reads were aligned to particularly divergent genomes, the highest-performing pipelines often used the aligners NextGenMap or SMALT, and/or the variant callers LoFreq, mpileup, or Strelka.

/pdf/genomic-diversity-affects-the-accuracy-of-bacterial-single-39id1yg2bd.pdf

Genomic diversity affects the accuracy of bacterial single-nucleotide polymorphism-calling pipelines.

Dashing is a fast and accurate software tool for estimating similarities of genomes or sequencing datasets. It uses the HyperLogLog sketch together with cardinality estimation methods that are specialized for set unions and intersections. Dashing summarizes genomes more rapidly than previous MinHash-based methods while providing greater accuracy across a wide range of input sizes and sketch sizes. It can sketch and calculate pairwise distances for over 87K genomes in 6 minutes. Dashing is open source and available at https://github.com/dnbaker/dashing.

/pdf/dashing-fast-and-accurate-genomic-distances-with-hyperloglog-3w7sdzi0xs.pdf

Dashing: fast and accurate genomic distances with HyperLogLog

Dashing is a fast and accurate software tool for estimating similarities of genomes or sequencing datasets. It uses the HyperLogLog sketch together with cardinality estimation methods that specialize in set unions and intersections. Dashing sketches genomes more rapidly than previous MinHash-based methods while providing greater accuracy across a wide range of input sizes and sketch sizes. It can sketch and calculate pairwise distances for over 87K genomes in under 6 minutes. Dashing is open source and available at https://github.com/dnbaker/dashing.

/pdf/dashing-fast-and-accurate-genomic-distances-with-hyperloglog-158yxh8y17.pdf

Dashing: Fast and Accurate Genomic Distances with HyperLogLog

Motivation A common way to summarize sequencing datasets is to quantify data lying within genes or other genomic intervals. This can be slow and can require different tools for different input file types. Results Megadepth is a fast tool for quantifying alignments and coverage for BigWig and BAM/CRAM input files, using substantially less memory than the next-fastest competitor. Megadepth can summarize coverage within all disjoint intervals of the Gencode V35 gene annotation for more than 19,000 GTExV8 BigWig files in approximately one hour using 32 threads. Megadepth is available both as a command-line tool and as an R/Bioconductor package providing much faster quantification compared to the rtracklayer package. Availability https://github.com/ChristopherWilks/megadepth,https://bioconductor.org/packages/megadepth.

/pdf/megadepth-efficient-coverage-quantification-for-bigwigs-and-5ajvoj7jph.pdf

Megadepth: efficient coverage quantification for BigWigs and BAMs

Motivation: A common way to summarize sequencing datasets is to quantify data lying within genes or other genomic intervals. This can be slow and can require different tools for different input file types. Results: Megadepth is a fast tool for quantifying alignments and coverage for BigWig and BAM/CRAM input files, using substantially less memory than the next fastest competitor. Megadepth can summarize coverage within all disjoint intervals of the Gencode V35 gene annotation for more than 19,000 GTExV8 BigWig files in approximately one hour using 32 threads. Megadepth is available both as a command-line tool and as an R/Bioconductor package providing much faster quantification compared to the rtracklayer package.

https://www.biorxiv.org/content/biorxiv/early/2020/12/18/2020.12.17.423317.full.pdf

Single-cell RNA-sequencing (scRNA-seq) analyses typically begin by clustering a gene-by-cell expression matrix to empirically define groups of cells with similar expression profiles. We describe new methods and a new open source library, minicore, for efficient k-means++ center finding and k-means clustering of scRNA-seq data. Minicore works with sparse count data, as it emerges from typical scRNA-seq experiments, as well as with dense data from after dimensionality reduction. Minicore's novel vectorized weighted reservoir sampling algorithm allows it to find initial k-means++ centers for a 4-million cell dataset in 1.5 minutes using 20 threads. Minicore can cluster using Euclidean distance, but also supports a wider class of measures like Jensen-Shannon Divergence, Kullback-Leibler Divergence, and the Bhattacharyya distance, which can be directly applied to count data and probability distributions. Further, minicore produces lower-cost centerings more efficiently than scikit-learn for scRNA-seq datasets with millions of cells. With careful handling of priors, minicore implements these distance measures with only minor (k-means++, localsearch++ and mini-batch k-means can cluster a 4-million cell dataset in minutes, using less than 10GiB of RAM. This memory-efficiency enables atlas-scale clustering on laptops and other commodity hardware. Finally, we report findings on which distance measures give clusterings that are most consistent with known cell type labels. Availability: The open source library is at https://github.com/dnbaker/minicore. Code used for experiments is at https://github.com/dnbaker/minicore-experiments.

https://dl.acm.org/doi/pdf/10.1145/3459930.3469523

Daniel N. Baker

Papers

Dashing: fast and accurate genomic distances with HyperLogLog

Dashing: Fast and Accurate Genomic Distances with HyperLogLog

Megadepth: efficient coverage quantification for BigWigs and BAMs

Megadepth: efficient coverage quantification for BigWigs and BAMs

Fast and memory-efficient scRNA-seq k-means clustering with various distances