Model-based reinforcement learning (MBRL) is widely seen as having the potential to be significantly more sample efficient than model-free RL. However, research in model-based RL has not been very standardized. It is fairly common for authors to experiment with self-designed environments, and there are several separate lines of research, which are sometimes closed-sourced or not reproducible. Accordingly, it is an open question how these various existing MBRL algorithms perform relative to each other. To facilitate research in MBRL, in this paper we gather a wide collection of MBRL algorithms and propose over 18 benchmarking environments specially designed for MBRL. We benchmark these algorithms with unified problem settings, including noisy environments. Beyond cataloguing performance, we explore and unify the underlying algorithmic differences across MBRL algorithms. We characterize three key research challenges for future MBRL research: the dynamics bottleneck, the planning horizon dilemma, and the early-termination dilemma. Finally, to maximally facilitate future research on MBRL, we open-source our benchmark in this http URL.

/pdf/benchmarking-model-based-reinforcement-learning-3o1aw85vqb.pdf

Benchmarking Model-Based Reinforcement Learning

Molecular characterization of cell types using single cell transcriptome sequencing is revolutionizing cell biology and enabling new insights into the physiology of human organs. We created a human reference atlas comprising nearly 500,000 cells from 24 different tissues and organs, many from the same donor. This atlas enabled molecular characterization of more than 400 cell types, their distribution across tissues and tissue specific variation in gene expression. Using multiple tissues from a single donor enabled identification of the clonal distribution of T cells between tissues, the tissue specific mutation rate in B cells, and analysis of the cell cycle state and proliferative potential of shared cell types across tissues. Cell type specific RNA splicing was discovered and analyzed across tissues within an individual.

https://www.science.org/doi/pdf/10.1126/science.abl4896?download=true

The Tabula Sapiens: a multiple organ single cell transcriptomic atlas of humans

Background Large language models such as ChatGPT can produce increasingly realistic text, with unknown information on the accuracy and integrity of using these models in scientific writing. Methods We gathered ten research abstracts from five high impact factor medical journals (n=50) and asked ChatGPT to generate research abstracts based on their titles and journals. We evaluated the abstracts using an artificial intelligence (AI) output detector, plagiarism detector, and had blinded human reviewers try to distinguish whether abstracts were original or generated. Results All ChatGPT-generated abstracts were written clearly but only 8% correctly followed the specific journal’s formatting requirements. Most generated abstracts were detected using the AI output detector, with scores (higher meaning more likely to be generated) of median [interquartile range] of 99.98% [12.73, 99.98] compared with very low probability of AI-generated output in the original abstracts of 0.02% [0.02, 0.09]. The AUROC of the AI output detector was 0.94. Generated abstracts scored very high on originality using the plagiarism detector (100% [100, 100] originality). Generated abstracts had a similar patient cohort size as original abstracts, though the exact numbers were fabricated. When given a mixture of original and general abstracts, blinded human reviewers correctly identified 68% of generated abstracts as being generated by ChatGPT, but incorrectly identified 14% of original abstracts as being generated. Reviewers indicated that it was surprisingly difficult to differentiate between the two, but that the generated abstracts were vaguer and had a formulaic feel to the writing. Conclusion ChatGPT writes believable scientific abstracts, though with completely generated data. These are original without any plagiarism detected but are often identifiable using an AI output detector and skeptical human reviewers. Abstract evaluation for journals and medical conferences must adapt policy and practice to maintain rigorous scientific standards; we suggest inclusion of AI output detectors in the editorial process and clear disclosure if these technologies are used. The boundaries of ethical and acceptable use of large language models to help scientific writing remain to be determined.

/pdf/comparing-scientific-abstracts-generated-by-chatgpt-to-1qt5ziht.pdf

Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers

Nirmatrelvir, an oral antiviral targeting the 3CL protease of SARS-CoV-2, has been demonstrated to be clinically useful against COVID-19 (refs. 1,2). However, because SARS-CoV-2 has evolved to become resistant to other therapeutic modalities3-9, there is a concern that the same could occur for nirmatrelvir. Here we examined this possibility by in vitro passaging of SARS-CoV-2 in nirmatrelvir using two independent approaches, including one on a large scale. Indeed, highly resistant viruses emerged from both and their sequences showed a multitude of 3CL protease mutations. In the experiment peformed with many replicates, 53 independent viral lineages were selected with mutations observed at 23 different residues of the enzyme. Nevertheless, several common mutational pathways to nirmatrelvir resistance were preferred, with a majority of the viruses descending from T21I, P252L or T304I as precursor mutations. Construction and analysis of 13 recombinant SARS-CoV-2 clones showed that these mutations mediated only low-level resistance, whereas greater resistance required accumulation of additional mutations. E166V mutation conferred the strongest resistance (around 100-fold), but this mutation resulted in a loss of viral replicative fitness that was restored by compensatory changes such as L50F and T21I. Our findings indicate that SARS-CoV-2 resistance to nirmatrelvir does readily arise via multiple pathways in vitro, and the specific mutations observed herein form a strong foundation from which to study the mechanism of resistance in detail and to inform the design of next-generation protease inhibitors. 

/pdf/multiple-pathways-for-sars-cov-2-resistance-to-nirmatrelvir-2bxodmwv.pdf

Multiple pathways for SARS-CoV-2 resistance to nirmatrelvir

The nucleoside analog remdesivir (RDV) is a Food and Drug Administration–approved antiviral for treatment of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections. Thus, it is critical to understand factors that promote or prevent RDV resistance. We passaged SARS-CoV-2 in the presence of increasing concentrations of GS-441524, the parent nucleoside of RDV. After 13 passages, we isolated three viral lineages with phenotypic resistance as defined by increases in half-maximal effective concentration from 2.7- to 10.4-fold. Sequence analysis identified nonsynonymous mutations in nonstructural protein 12 RNA-dependent RNA polymerase (nsp12-RdRp): V166A, N198S, S759A, V792I, and C799F/R. Two lineages encoded the S759A substitution at the RdRp Ser759-Asp-Asp active motif. In one lineage, the V792I substitution emerged first and then combined with S759A. Introduction of S759A and V792I substitutions at homologous nsp12 positions in murine hepatitis virus demonstrated transferability across betacoronaviruses; introduction of these substitutions resulted in up to 38-fold RDV resistance and a replication defect. Biochemical analysis of SARS-CoV-2 RdRp encoding S759A demonstrated a roughly 10-fold decreased preference for RDV-triphosphate (RDV-TP) as a substrate, whereas nsp12-V792I diminished the uridine triphosphate concentration needed to overcome template-dependent inhibition associated with RDV. The in vitro–selected substitutions identified in this study were rare or not detected in the greater than 6 million publicly available nsp12-RdRp consensus sequences in the absence of RDV selection. The results define genetic and biochemical pathways to RDV resistance and emphasize the need for additional studies to define the potential for emergence of these or other RDV resistance mutations in clinical settings. Description SARS-CoV-2 develops in vitro resistance to remdesivir by distinct, complementary mutations and mechanisms in the viral RNA-dependent RNA polymerase. Remdesivir resistance The nucleoside analog remdesivir (RDV) is a mainstay of treatment for SARS-CoV-2 infection; however, it is not clear how SARS-CoV-2 could develop resistance to RDV. Here, Stevens et al. serially passaged SARS-CoV-2 in the presence of increasing concentrations of GS-441524, the parent nucleoside of RDV. They found that two mutations in the SARS-CoV-2 RNA-dependent nonstructural protein 12 RNA polymerase (nsp12-RdRp), S759A and V792I, mediated RDV resistance. The authors validated a role for these mutations in mediating RDV resistance by inserting in the nsp12 gene of another coronavirus, murine hepatitis virus. Analysis of more than 6 million SARS-CoV-2 nsp12 sequences demonstrated that these mutations were rare or not detected. Together, these data demonstrate how RDV resistance may arise.

https://www.science.org/cms/asset/7acca585-1a26-4178-b99a-0447c4b07981/scitranslmed.abo0718.v1.pdf

Mutations in the SARS-CoV-2 RNA dependent RNA polymerase confer resistance to remdesivir by distinct mechanisms

seaborn is a library for making statistical graphics in Python. It provides a high-level interface to matplotlib and integrates closely with pandas data structures. Functions in the seaborn library expose a declarative, dataset-oriented API that makes it easy to translate questions about data into graphics that can answer them. When given a dataset and a specification of the plot to make, seaborn automatically maps the data values to visual attributes such as color, size, or style, internally computes statistical transformations, and decorates the plot with informative axis labels and a legend. Many seaborn functions can generate figures with multiple panels that elicit comparisons between conditional subsets of data or across different pairings of variables in a dataset. seaborn is designed to be useful throughout the lifecycle of a scientific project. By producing complete graphics from a single function call with minimal arguments, seaborn facilitates rapid prototyping and exploratory data analysis. And by offering extensive options for customization, along with exposing the underlying matplotlib objects, it can be used to create polished, publication-quality figures.

Bachant, Pete Quintero, Eric

Papers

seaborn: Statistical data visualization