Abstract: This paper presents a new Unified pre-trained Language Model (UniLM) that can be fine-tuned for both natural language understanding and generation tasks. The model is pre-trained using three types of language modeling tasks: unidirectional, bidirectional, and sequence-to-sequence prediction. The unified modeling is achieved by employing a shared Transformer network and utilizing specific self-attention masks to control what context the prediction conditions on. UniLM compares favorably with BERT on the GLUE benchmark, and the SQuAD 2.0 and CoQA question answering tasks. Moreover, UniLM achieves new state-of-the-art results on five natural language generation datasets, including improving the CNN/DailyMail abstractive summarization ROUGE-L to 40.51 (2.04 absolute improvement), the Gigaword abstractive summarization ROUGE-L to 35.75 (0.86 absolute improvement), the CoQA generative question answering F1 score to 82.5 (37.1 absolute improvement), the SQuAD question generation BLEU-4 to 22.12 (3.75 absolute improvement), and the DSTC7 document-grounded dialog response generation NIST-4 to 2.67 (human performance is 2.65). The code and pre-trained models are available at https://github.com/microsoft/unilm.

Abstract: Ana M Valdes and colleagues discuss strategies for modulating the gut microbiota through diet and probiotics

Abstract: The development of the microbiome from infancy to childhood is dependent on a range of factors, with microbial–immune crosstalk during this time thought to be involved in the pathobiology of later life diseases1–9 such as persistent islet autoimmunity and type 1 diabetes10–12. However, to our knowledge, no studies have performed extensive characterization of the microbiome in early life in a large, multi-centre population. Here we analyse longitudinal stool samples from 903 children between 3 and 46 months of age by 16S rRNA gene sequencing (n = 12,005) and metagenomic sequencing (n = 10,867), as part of the The Environmental Determinants of Diabetes in the Young (TEDDY) study. We show that the developing gut microbiome undergoes three distinct phases of microbiome progression: a developmental phase (months 3–14), a transitional phase (months 15–30), and a stable phase (months 31–46). Receipt of breast milk, either exclusive or partial, was the most significant factor associated with the microbiome structure. Breastfeeding was associated with higher levels of Bifidobacterium species (B. breve and B. bifidum), and the cessation of breast milk resulted in faster maturation of the gut microbiome, as marked by the phylum Firmicutes. Birth mode was also significantly associated with the microbiome during the developmental phase, driven by higher levels of Bacteroides species (particularly B. fragilis) in infants delivered vaginally. Bacteroides was also associated with increased gut diversity and faster maturation, regardless of the birth mode. Environmental factors including geographical location and household exposures (such as siblings and furry pets) also represented important covariates. A nested case–control analysis revealed subtle associations between microbial taxonomy and the development of islet autoimmunity or type 1 diabetes. These data determine the structural and functional assembly of the microbiome in early life and provide a foundation for targeted mechanistic investigation into the consequences of microbial–immune crosstalk for long-term health.

Abstract: Cytotoxic T cells are of central importance in the immune system's response to disease. They recognize defective cells by binding to peptides presented on the cell surface by MHC class I molecules. Peptide binding to MHC molecules is the single most selective step in the Ag-presentation pathway. Therefore, in the quest for T cell epitopes, the prediction of peptide binding to MHC molecules has attracted widespread attention. In the past, predictors of peptide-MHC interactions have primarily been trained on binding affinity data. Recently, an increasing number of MHC-presented peptides identified by mass spectrometry have been reported containing information about peptide-processing steps in the presentation pathway and the length distribution of naturally presented peptides. In this article, we present NetMHCpan-4.0, a method trained on binding affinity and eluted ligand data leveraging the information from both data types. Large-scale benchmarking of the method demonstrates an increase in predictive performance compared with state-of-the-art methods when it comes to identification of naturally processed ligands, cancer neoantigens, and T cell epitopes.

Abstract: In fine art, especially painting, humans have mastered the skill to create unique visual experiences through composing a complex interplay between the content and style of an image. Thus far the algorithmic basis of this process is unknown and there exists no artificial system with similar capabilities. However, in other key areas of visual perception such as object and face recognition near-human performance was recently demonstrated by a class of biologically inspired vision models called Deep Neural Networks. Here we introduce an artificial system based on a Deep Neural Network that creates artistic images of high perceptual quality. The system uses neural representations to separate and recombine content and style of arbitrary images, providing a neural algorithm for the creation of artistic images. Moreover, in light of the striking similarities between performance-optimised artificial neural networks and biological vision, our work offers a path forward to an algorithmic understanding of how humans create and perceive artistic imagery.

Abstract: We suggest a method to construct a homomorphic encryption scheme for approximate arithmetic. It supports an approximate addition and multiplication of encrypted messages, together with a new rescaling procedure for managing the magnitude of plaintext. This procedure truncates a ciphertext into a smaller modulus, which leads to rounding of plaintext. The main idea is to add a noise following significant figures which contain a main message. This noise is originally added to the plaintext for security, but considered to be a part of error occurring during approximate computations that is reduced along with plaintext by rescaling. As a result, our decryption structure outputs an approximate value of plaintext with a predetermined precision.

Abstract: Objective To develop new classification criteria for systemic lupus erythematosus (SLE) jointly supported by the European League Against Rheumatism (EULAR) and the American College of Rheumatology (ACR). Methods This international initiative had four phases. 1) Evaluation of antinuclear antibody (ANA) as an entry criterion through systematic review and meta-regression of the literature and criteria generation through an international Delphi exercise, an early patient cohort, and a patient survey. 2) Criteria reduction by Delphi and nominal group technique exercises. 3) Criteria definition and weighting based on criterion performance and on results of a multi-criteria decision analysis. 4) Refinement of weights and threshold scores in a new derivation cohort of 1,001 subjects and validation compared with previous criteria in a new validation cohort of 1,270 subjects. Results The 2019 EULAR/ACR classification criteria for SLE include positive ANA at least once as obligatory entry criterion; followed by additive weighted criteria grouped in 7 clinical (constitutional, hematologic, neuropsychiatric, mucocutaneous, serosal, musculoskeletal, renal) and 3 immunologic (antiphospholipid antibodies, complement proteins, SLE-specific antibodies) domains, and weighted from 2 to 10. Patients accumulating ≥10 points are classified. In the validation cohort, the new criteria had a sensitivity of 96.1% and specificity of 93.4%, compared with 82.8% sensitivity and 93.4% specificity of the ACR 1997 and 96.7% sensitivity and 83.7% specificity of the Systemic Lupus International Collaborating Clinics 2012 criteria. Conclusion These new classification criteria were developed using rigorous methodology with multidisciplinary and international input, and have excellent sensitivity and specificity. Use of ANA entry criterion, hierarchically clustered, and weighted criteria reflects current thinking about SLE and provides an improved foundation for SLE research.

Abstract: High demand for computation resources severely hinders deployment of large-scale Deep Neural Networks (DNN) in resource constrained devices. In this work, we propose a Structured Sparsity Learning (SSL) method to regularize the structures (i.e., filters, channels, filter shapes, and layer depth) of DNNs. SSL can: (1) learn a compact structure from a bigger DNN to reduce computation cost; (2) obtain a hardware-friendly structured sparsity of DNN to efficiently accelerate the DNN’s evaluation. Experimental results show that SSL achieves on average 5.1X and 3.1X speedups of convolutional layer computation of AlexNet against CPU and GPU, respectively, with off-the-shelf libraries. These speedups are about twice speedups of non-structured sparsity; (3) regularize the DNN structure to improve classification accuracy. The results show that for CIFAR-10, regularization on layer depth reduces a 20-layer Deep Residual Network (ResNet) to 18 layers while improves the accuracy from 91.25% to 92.60%, which is still higher than that of original ResNet with 32 layers. For AlexNet, SSL reduces the error by ~1%.

Abstract: The Global Burden of Disease 2015 study aims to use all available data of sufficient quality to generate reliable and valid prevalence, incidence, and disability-adjusted life year (DALY) estimates of oral conditions for the period of 1990 to 2015. Since death as a direct result of oral diseases is rare, DALY estimates were based on years lived with disability, which are estimated only on those persons with unmet need for dental care. We used our data to assess progress toward the Federation Dental International, World Health Organization, and International Association for Dental Research's oral health goals of reducing the level of oral diseases and minimizing their impact by 2020. Oral health has not improved in the last 25 y, and oral conditions remained a major public health challenge all over the world in 2015. Due to demographic changes, including population growth and aging, the cumulative burden of oral conditions dramatically increased between 1990 and 2015. The number of people with untreated oral conditions rose from 2.5 billion in 1990 to 3.5 billion in 2015, with a 64% increase in DALYs due to oral conditions throughout the world. Clearly, oral diseases are highly prevalent in the globe, posing a very serious public health challenge to policy makers. Greater efforts and potentially different approaches are needed if the oral health goal of reducing the level of oral diseases and minimizing their impact is to be achieved by 2020. Despite some challenges with current measurement methodologies for oral diseases, measurable specific oral health goals should be developed to advance global public health.

Abstract: The aim of this study was to analyze the clinical data, discharge rate, and fatality rate of COVID-19 patients for clinical help. The clinical data of COVID-19 patients from December 2019 to February 2020 were retrieved from four databases. We statistically analyzed the clinical symptoms and laboratory results of COVID-19 patients and explained the discharge rate and fatality rate with a single-arm meta-analysis. The available data of 1994 patients in 10 literatures were included in our study. The main clinical symptoms of COVID-19 patients were fever (88.5%), cough (68.6%), myalgia or fatigue (35.8%), expectoration (28.2%), and dyspnea (21.9%). Minor symptoms include headache or dizziness (12.1%), diarrhea (4.8%), nausea and vomiting (3.9%). The results of the laboratory showed that the lymphocytopenia (64.5%), increase of C-reactive protein (44.3%), increase of lactic dehydrogenase (28.3%), and leukocytopenia (29.4%) were more common. The results of single-arm meta-analysis showed that the male took a larger percentage in the gender distribution of COVID-19 patients 60% (95% CI [0.54, 0.65]), the discharge rate of COVID-19 patients was 52% (95% CI [0.34,0.70]), and the fatality rate was 5% (95% CI [0.01,0.11]).

Abstract: Plastic litter is an ever-increasing global issue and one of this generation’s key environmental challenges. Microplastics have reached oceans via river transport on a global scale. With the exception of two megacities, Paris (France) and Dongguan (China), there is a lack of information on atmospheric microplastic deposition or transport. Here we present the observations of atmospheric microplastic deposition in a remote, pristine mountain catchment (French Pyrenees). We analysed samples, taken over five months, that represent atmospheric wet and dry deposition and identified fibres up to ~750 µm long and fragments ≤300 µm as microplastics. We document relative daily counts of 249 fragments, 73 films and 44 fibres per square metre that deposited on the catchment. An air mass trajectory analysis shows microplastic transport through the atmosphere over a distance of up to 95 km. We suggest that microplastics can reach and affect remote, sparsely inhabited areas through atmospheric transport.

Abstract: BackgroundCombination therapy with the BRAF inhibitor dabrafenib plus the MEK inhibitor trametinib improved survival in patients with advanced melanoma with BRAF V600 mutations. We sought to determine whether adjuvant dabrafenib plus trametinib would improve outcomes in patients with resected, stage III melanoma with BRAF V600 mutations. MethodsIn this double-blind, placebo-controlled, phase 3 trial, we randomly assigned 870 patients with completely resected, stage III melanoma with BRAF V600E or V600K mutations to receive oral dabrafenib at a dose of 150 mg twice daily plus trametinib at a dose of 2 mg once daily (combination therapy, 438 patients) or two matched placebo tablets (432 patients) for 12 months. The primary end point was relapse-free survival. Secondary end points included overall survival, distant metastasis–free survival, freedom from relapse, and safety. ResultsAt a median follow-up of 2.8 years, the estimated 3-year rate of relapse-free survival was 58% in the combination-therapy group a...

Abstract: Background The use of prescription opioid medications has increased greatly in the United States during the past two decades; in 2010, there were 16,651 opioid-related deaths. In response, hundreds of federal, state, and local interventions have been implemented. We describe trends in the diversion and abuse of prescription opioid analgesics using data through 2013. Methods We used five programs from the Researched Abuse, Diversion, and AddictionRelated Surveillance (RADARS) System to describe trends between 2002 and 2013 in the diversion and abuse of all products and formulations of six prescription opioid analgesics: oxycodone, hydrocodone, hydromorphone, fentanyl, morphine, and tramadol. The programs gather data from drug-diversion investigators, poison centers, substance-abuse treatment centers, and college students. Results Prescriptions for opioid analgesics increased substantially from 2002 through 2010 in the United States but then decreased slightly from 2011 through 2013. In general, RADARS System programs reported large increases in the rates of opioid diversion and abuse from 2002 to 2010, but then the rates flattened or decreased from 2011 through 2013. The rate of opioid-related deaths rose and fell in a similar pattern. Reported nonmedical use did not change significantly among college students.

Abstract: The widespread application of next-generation sequencing technologies has revolutionized microbiome research by enabling high-throughput profiling of the genetic contents of microbial communities. How to analyze the resulting large complex datasets remains a key challenge in current microbiome studies. Over the past decade, powerful computational pipelines and robust protocols have been established to enable efficient raw data processing and annotation. The focus has shifted toward downstream statistical analysis and functional interpretation. Here, we introduce MicrobiomeAnalyst, a user-friendly tool that integrates recent progress in statistics and visualization techniques, coupled with novel knowledge bases, to enable comprehensive analysis of common data outputs produced from microbiome studies. MicrobiomeAnalyst contains four modules - the Marker Data Profiling module offers various options for community profiling, comparative analysis and functional prediction based on 16S rRNA marker gene data; the Shotgun Data Profiling module supports exploratory data analysis, functional profiling and metabolic network visualization of shotgun metagenomics or metatranscriptomics data; the Taxon Set Enrichment Analysis module helps interpret taxonomic signatures via enrichment analysis against >300 taxon sets manually curated from literature and public databases; finally, the Projection with Public Data module allows users to visually explore their data with a public reference data for pattern discovery and biological insights. MicrobiomeAnalyst is freely available at http://www.microbiomeanalyst.ca.

Abstract: Objective To systematically review associations between intake of saturated fat and trans unsaturated fat and all cause mortality, cardiovascular disease (CVD) and associated mortality, coronary heart disease (CHD) and associated mortality, ischemic stroke, and type 2 diabetes. Design Systematic review and meta-analysis. Data sources Medline, Embase, Cochrane Central Registry of Controlled Trials, Evidence-Based Medicine Reviews, and CINAHL from inception to 1 May 2015, supplemented by bibliographies of retrieved articles and previous reviews. Eligibility criteria for selecting studies Observational studies reporting associations of saturated fat and/or trans unsaturated fat (total, industrially manufactured, or from ruminant animals) with all cause mortality, CHD/CVD mortality, total CHD, ischemic stroke, or type 2 diabetes. Data extraction and synthesis Two reviewers independently extracted data and assessed study risks of bias. Multivariable relative risks were pooled. Heterogeneity was assessed and quantified. Potential publication bias was assessed and subgroup analyses were undertaken. The GRADE approach was used to evaluate quality of evidence and certainty of conclusions. Results For saturated fat, three to 12 prospective cohort studies for each association were pooled (five to 17 comparisons with 90 501-339 090 participants). Saturated fat intake was not associated with all cause mortality (relative risk 0.99, 95% confidence interval 0.91 to 1.09), CVD mortality (0.97, 0.84 to 1.12), total CHD (1.06, 0.95 to 1.17), ischemic stroke (1.02, 0.90 to 1.15), or type 2 diabetes (0.95, 0.88 to 1.03). There was no convincing lack of association between saturated fat and CHD mortality (1.15, 0.97 to 1.36; P=0.10). For trans fats, one to six prospective cohort studies for each association were pooled (two to seven comparisons with 12 942-230 135 participants). Total trans fat intake was associated with all cause mortality (1.34, 1.16 to 1.56), CHD mortality (1.28, 1.09 to 1.50), and total CHD (1.21, 1.10 to 1.33) but not ischemic stroke (1.07, 0.88 to 1.28) or type 2 diabetes (1.10, 0.95 to 1.27). Industrial, but not ruminant, trans fats were associated with CHD mortality (1.18 (1.04 to 1.33) v 1.01 (0.71 to 1.43)) and CHD (1.42 (1.05 to 1.92) v 0.93 (0.73 to 1.18)). Ruminant trans -palmitoleic acid was inversely associated with type 2 diabetes (0.58, 0.46 to 0.74). The certainty of associations between saturated fat and all outcomes was “very low.” The certainty of associations of trans fat with CHD outcomes was “moderate” and “very low” to “low” for other associations. Conclusions Saturated fats are not associated with all cause mortality, CVD, CHD, ischemic stroke, or type 2 diabetes, but the evidence is heterogeneous with methodological limitations. Trans fats are associated with all cause mortality, total CHD, and CHD mortality, probably because of higher levels of intake of industrial trans fats than ruminant trans fats. Dietary guidelines must carefully consider the health effects of recommendations for alternative macronutrients to replace trans fats and saturated fats.

Abstract: Telomeres are the protective end-complexes at the termini of eukaryotic chromosomes. Telomere attrition can lead to potentially maladaptive cellular changes, block cell division, and interfere with tissue replenishment. Recent advances in the understanding of human disease processes have clarified the roles of telomere biology, especially in diseases of human aging and in some aging-related processes. Greater overall telomere attrition predicts mortality and aging-related diseases in inherited telomere syndrome patients, and also in general human cohorts. However, genetically caused variations in telomere maintenance either raise or lower risks and progression of cancers, in a highly cancer type-specific fashion. Telomere maintenance is determined by genetic factors and is also cumulatively shaped by nongenetic influences throughout human life; both can interact. These and other recent findings highlight both causal and potentiating roles for telomere attrition in human diseases.

Abstract: Recurrent neural network architectures have been shown to efficiently model long term temporal dependencies between acoustic events. However the training time of recurrent networks is higher than feedforward networks due to the sequential nature of the learning algorithm. In this paper we propose a time delay neural network architecture which models long term temporal dependencies with training times comparable to standard feed-forward DNNs. The network uses sub-sampling to reduce computation during training. On the Switchboard task we show a relative improvement of 6% over the baseline DNN model. We present results on several LVCSR tasks with training data ranging from 3 to 1800 hours to show the effectiveness of the TDNN architecture in learning wider temporal dependencies in both small and large data scenarios.

Abstract: Spatial thinning of species occurrence records can help address problems associated with spatial sampling biases. Ideally, thinning removes the fewest records necessary to substantially reduce the effects of sampling bias, while simultaneously retaining the greatest amount of useful information. Spatial thinning can be done manually; however, this is prohibitively time consuming for large datasets. Using a randomization approach, the ‘thin’ function in the spThin R package returns a dataset with the maximum number of records for a given thinning distance, when run for sufficient iterations. We here provide a worked example for the Caribbean spiny pocket mouse, where the results obtained match those of manual thinning.

Abstract: The large pose discrepancy between two face images is one of the key challenges in face recognition. Conventional approaches for pose-invariant face recognition either perform face frontalization on, or learn a pose-invariant representation from, a non-frontal face image. We argue that it is more desirable to perform both tasks jointly to allow them to leverage each other. To this end, this paper proposes Disentangled Representation learning-Generative Adversarial Network (DR-GAN) with three distinct novelties. First, the encoder-decoder structure of the generator allows DR-GAN to learn a generative and discriminative representation, in addition to image synthesis. Second, this representation is explicitly disentangled from other face variations such as pose, through the pose code provided to the decoder and pose estimation in the discriminator. Third, DR-GAN can take one or multiple images as the input, and generate one unified representation along with an arbitrary number of synthetic images. Quantitative and qualitative evaluation on both controlled and in-the-wild databases demonstrate the superiority of DR-GAN over the state of the art.

Abstract: The recent “Text-to-Text Transfer Transformer” (T5) leveraged a unified text-to-text format and scale to attain state-of-the-art results on a wide variety of English-language NLP tasks. In this paper, we introduce mT5, a multilingual variant of T5 that was pre-trained on a new Common Crawl-based dataset covering 101 languages. We detail the design and modified training of mT5 and demonstrate its state-of-the-art performance on many multilingual benchmarks. We also describe a simple technique to prevent “accidental translation” in the zero-shot setting, where a generative model chooses to (partially) translate its prediction into the wrong language. All of the code and model checkpoints used in this work are publicly available.

Abstract: We analyze stochastic gradient descent for optimizing non-convex functions. In many cases for non-convex functions the goal is to find a reasonable local minimum, and the main concern is that gradient updates are trapped in saddle points. In this paper we identify strict saddle property for non-convex problem that allows for efficient optimization. Using this property we show that from an arbitrary starting point, stochastic gradient descent converges to a local minimum in a polynomial number of iterations. To the best of our knowledge this is the first work that gives global convergence guarantees for stochastic gradient descent on non-convex functions with exponentially many local minima and saddle points. Our analysis can be applied to orthogonal tensor decomposition, which is widely used in learning a rich class of latent variable models. We propose a new optimization formulation for the tensor decomposition problem that has strict saddle property. As a result we get the first online algorithm for orthogonal tensor decomposition with global convergence guarantee.

Abstract: INTRODUCTIONThe “sepsis bundle” has been central to the implementation of the Surviving Sepsis Campaign (SSC) from the first publication of its evidence-based guidelines in 2004 through subsequent editions (1–6). Developed separately from the guidelines publication by the SSC, the bundles have been

Abstract: We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. By stacking layers in which nodes are able to attend over their neighborhoods' features, we enable (implicitly) specifying different weights to different nodes in a neighborhood, without requiring any kind of costly matrix operation (such as inversion) or depending on knowing the graph structure upfront. In this way, we address several key challenges of spectral-based graph neural networks simultaneously, and make our model readily applicable to inductive as well as transductive problems. Our GAT models have achieved or matched state-of-the-art results across four established transductive and inductive graph benchmarks: the Cora, Citeseer and Pubmed citation network datasets, as well as a protein-protein interaction dataset (wherein test graphs remain unseen during training).

Abstract: Cancer patients often develop chronic, clinically significant symptoms of depression and anxiety. Previous studies suggest that psilocybin may decrease depression and anxiety in cancer patients. Th...

Abstract: The global emergence of multidrug-resistant Gram-negative bacteria is a growing threat to antibiotic therapy. The chromosomally encoded drug efflux mechanisms that are ubiquitous in these bacteria greatly contribute to antibiotic resistance and present a major challenge for antibiotic development. Multidrug pumps, particularly those represented by the clinically relevant AcrAB-TolC and Mex pumps of the resistance-nodulation-division (RND) superfamily, not only mediate intrinsic and acquired multidrug resistance (MDR) but also are involved in other functions, including the bacterial stress response and pathogenicity. Additionally, efflux pumps interact synergistically with other resistance mechanisms (e.g., with the outer membrane permeability barrier) to increase resistance levels. Since the discovery of RND pumps in the early 1990s, remarkable scientific and technological advances have allowed for an in-depth understanding of the structural and biochemical basis, substrate profiles, molecular regulation, and inhibition of MDR pumps. However, the development of clinically useful efflux pump inhibitors and/or new antibiotics that can bypass pump effects continues to be a challenge. Plasmid-borne efflux pump genes (including those for RND pumps) have increasingly been identified. This article highlights the recent progress obtained for organisms of clinical significance, together with methodological considerations for the characterization of MDR pumps.

Abstract: Perovskite solar cells (PSCs) exceeding a power conversion efficiency (PCE) of 20% have mainly been demonstrated by using mesoporous titanium dioxide (mp-TiO2) as an electron-transporting layer. However, TiO2 can reduce the stability of PSCs under illumination (including ultraviolet light). Lanthanum (La)–doped BaSnO3 (LBSO) perovskite would be an ideal replacement given its electron mobility and electronic structure, but LBSO cannot be synthesized as well-dispersible fine particles or crystallized below 500°C. We report a superoxide colloidal solution route for preparing a LBSO electrode under very mild conditions (below 300°C). The PSCs fabricated with LBSO and methylammonium lead iodide (MAPbI3) show a steady-state power conversion efficiency of 21.2%, versus 19.7% for a mp-TiO2 device. The LBSO-based PSCs could retain 93% of their initial performance after 1000 hours of full-Sun illumination.