Motivation Quality control and preprocessing of FASTQ files are essential to providing clean data for downstream analysis. Traditionally, a different tool is used for each operation, such as quality control, adapter trimming and quality filtering. These tools are often insufficiently fast as most are developed using high-level programming languages (e.g. Python and Java) and provide limited multi-threading support. Reading and loading data multiple times also renders preprocessing slow and I/O inefficient. Results We developed fastp as an ultra-fast FASTQ preprocessor with useful quality control and data-filtering features. It can perform quality control, adapter trimming, quality filtering, per-read quality pruning and many other operations with a single scan of the FASTQ data. This tool is developed in C++ and has multi-threading support. Based on our evaluation, fastp is 2-5 times faster than other FASTQ preprocessing tools such as Trimmomatic or Cutadapt despite performing far more operations than similar tools. Availability and implementation The open-source code and corresponding instructions are available at https://github.com/OpenGene/fastp.

/pdf/fastp-an-ultra-fast-all-in-one-fastq-preprocessor-ehzxan9vkg.pdf

fastp: an ultra-fast all-in-one FASTQ preprocessor.

Motivation: Quality control and preprocessing of FASTQ files are essential to providing clean data for downstream analysis. Traditionally, a different tool is used for each operation, such as quality control, adapter trimming, and quality filtering. These tools are often insufficiently fast as most are developed using high level programming languages (e.g., Python and Java) and provide limited multithreading support. Reading and loading data multiple times also renders preprocessing slow and I/O inefficient. Results: We developed fastp as an ultra-fast FASTQ preprocessor with useful quality control and data-filtering features. It can perform quality control, adapter trimming, quality filtering, per read quality cutting, and many other operations with a single scan of the FASTQ data. It also supports unique molecular identifier preprocessing, poly tail trimming, output splitting, and base correction for paired-end data. It can automatically detect adapters for single-end and paired-end FASTQ data. This tool is developed in C++ and has multithreading support. Based on our evaluation, fastp is 2 to 5 times faster than other FASTQ preprocessing tools such as Trimmomatic or Cutadapt despite performing far more operations than similar tools. Availability and Implementation: The open-source code and corresponding instructions are available at https://github.com/OpenGene/fastp

https://www.biorxiv.org/content/biorxiv/early/2018/03/01/274100.full.pdf

fastp: an ultra-fast all-in-one FASTQ preprocessor

Nitrogen-use efficiency of indica varieties of rice is superior to that of japonica varieties. We apply 16S ribosomal RNA gene profiling to characterize root microbiota of 68 indica and 27 japonica varieties grown in the field. We find that indica and japonica recruit distinct root microbiota. Notably, indica-enriched bacterial taxa are more diverse, and contain more genera with nitrogen metabolism functions, than japonica-enriched taxa. Using genetic approaches, we provide evidence that NRT1.1B, a rice nitrate transporter and sensor, is associated with the recruitment of a large proportion of indica-enriched bacteria. Metagenomic sequencing reveals that the ammonification process is less abundant in the root microbiome of the nrt1.1b mutant. We isolated 1,079 pure bacterial isolates from indica and japonica roots and derived synthetic communities (SynComs). Inoculation of IR24, an indica variety, with an indica-enriched SynCom improved rice growth in organic nitrogen conditions compared with a japonica-enriched SynCom. The links between plant genotype and root microbiota membership established in this study will inform breeding strategies to improve nitrogen use in crops.

NRT1.1B is associated with root microbiota composition and nitrogen use in field-grown rice

The order Coleoptera (beetles) is arguably the most speciose group of animals, but the evolutionary history of beetles, including the impacts of plant feeding (herbivory) on beetle diversification, remain poorly understood. We inferred the phylogeny of beetles using 4,818 genes for 146 species, estimated timing and rates of beetle diversification using 89 genes for 521 species representing all major lineages and traced the evolution of beetle genes enabling symbiont-independent digestion of lignocellulose using 154 genomes or transcriptomes. Phylogenomic analyses of these uniquely comprehensive datasets resolved previously controversial beetle relationships, dated the origin of Coleoptera to the Carboniferous, and supported the codiversification of beetles and angiosperms. Moreover, plant cell wall-degrading enzymes (PCWDEs) obtained from bacteria and fungi via horizontal gene transfers may have been key to the Mesozoic diversification of herbivorous beetles—remarkably, both major independent origins of specialized herbivory in beetles coincide with the first appearances of an arsenal of PCWDEs encoded in their genomes. Furthermore, corresponding (Jurassic) diversification rate increases suggest that these novel genes triggered adaptive radiations that resulted in nearly half of all living beetle species. We propose that PCWDEs enabled efficient digestion of plant tissues, including lignocellulose in cell walls, facilitating the evolution of uniquely specialized plant-feeding habits, such as leaf mining and stem and wood boring. Beetle diversity thus appears to have resulted from multiple factors, including low extinction rates over a long evolutionary history, codiversification with angiosperms, and adaptive radiations of specialized herbivorous beetles following convergent horizontal transfers of microbial genes encoding PCWDEs.

https://www.pnas.org/content/pnas/116/49/24729.full.pdf

The evolution and genomic basis of beetle diversity.

Ferns are the closest sister group to all seed plants, yet little is known about their genomes other than that they are generally colossal. Here, we report on the genomes of Azolla filiculoides and Salvinia cucullata (Salviniales) and present evidence for episodic whole-genome duplication in ferns—one at the base of ‘core leptosporangiates’ and one specific to Azolla. One fern-specific gene that we identified, recently shown to confer high insect resistance, seems to have been derived from bacteria through horizontal gene transfer. Azolla coexists in a unique symbiosis with N2-fixing cyanobacteria, and we demonstrate a clear pattern of cospeciation between the two partners. Furthermore, the Azolla genome lacks genes that are common to arbuscular mycorrhizal and root nodule symbioses, and we identify several putative transporter genes specific to Azolla–cyanobacterial symbiosis. These genomic resources will help in exploring the biotechnological potential of Azolla and address fundamental questions in the evolution of plant life.

https://biblio.ugent.be/publication/8575736/file/8575738.pdf

Fern genomes elucidate land plant evolution and cyanobacterial symbioses

Quality control (QC) and preprocessing are essential steps for sequencing data analysis to ensure the accuracy of results. However, existing tools cannot provide a satisfying solution with integrated comprehensive functions, proper architectures, and highly scalable acceleration. In this article, we demonstrate SOAPnuke as a tool with abundant functions for a "QC-Preprocess-QC" workflow and MapReduce acceleration framework. Four modules with different preprocessing functions are designed for processing datasets from genomic, small RNA, Digital Gene Expression, and metagenomic experiments, respectively. As a workflow-like tool, SOAPnuke centralizes processing functions into 1 executable and predefines their order to avoid the necessity of reformatting different files when switching tools. Furthermore, the MapReduce framework enables large scalability to distribute all the processing works to an entire compute cluster.We conducted a benchmarking where SOAPnuke and other tools are used to preprocess a ∼30× NA12878 dataset published by GIAB. The standalone operation of SOAPnuke struck a balance between resource occupancy and performance. When accelerated on 16 working nodes with MapReduce, SOAPnuke achieved ∼5.7 times the fastest speed of other tools.

Zhang Yong

Papers

SOAPnuke: a MapReduce acceleration-supported software for integrated quality control and preprocessing of high-throughput sequencing data.