Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

Although genomewide RNA expression analysis has become a routine tool in biomedical research, extracting biological insight from such information remains a major challenge. Here, we describe a powerful analytical method called Gene Set Enrichment Analysis (GSEA) for interpreting gene expression data. The method derives its power by focusing on gene sets, that is, groups of genes that share common biological function, chromosomal location, or regulation. We demonstrate how GSEA yields insights into several cancer-related data sets, including leukemia and lung cancer. Notably, where single-gene analysis finds little similarity between two independent studies of patient survival in lung cancer, GSEA reveals many biological pathways in common. The GSEA method is embodied in a freely available software package, together with an initial database of 1,325 biologically defined gene sets.

/pdf/gene-set-enrichment-analysis-a-knowledge-based-approach-for-mbpti6qljw.pdf

Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

抗原变异可使得多种致病微生物易于逃避宿主免疫应答。表达在感染红细胞表面的恶性疟原虫红细胞表面蛋白1（PfPMP1）与感染红细胞、内皮细胞、树突状细胞以及胎盘的单个或多个受体作用，在黏附及免疫逃避中起关键的作用。每个单倍体基因组var基因家族编码约60种成员，通过启动转录不同的var基因变异体为抗原变异提供了分子基础。

疟原虫var基因转换速率变化导致抗原变异[英]／Paul H, Robert P, Christodoulou Z, et al//Proc Natl Acad Sci U S A

http://bartellab.wi.mit.edu/publication_reprints/Bartel_Cell09.pdf

MicroRNAs: Target Recognition and Regulatory Functions

Advances in self-supervised learning have drawn attention to developing techniques to extract effective visual representations from unlabeled images. Contrastive learning (CL) trains a model to extract consistent features by generating different views. Recent success of Masked Autoencoders (MAE) highlights the benefit of generative modeling in self-supervised learning. The generative approaches encode the input into a compact embedding and empower the model’s ability of recovering the original input. However, in our experiments, we found vanilla MAE mainly recovers coarse high level semantic information and is inadequate in recovering detailed low level information. We show that in dense downstream prediction tasks like multi-organ segmentation, directly applying MAE is not ideal. Here, we propose RepRec, a hybrid visual representation learning framework for self-supervised pre-training on large-scale unlabelled medical datasets, which takes advantage of both contrastive and generative modeling. To solve the aforementioned dilemma that MAE encounters, a convolutional encoder is pre-trained to provide low-level feature information, in a contrastive way; and a transformer encoder is pre-trained to produce high level semantic dependency, in a generative way – by recovering masked representations from the convolutional encoder. Extensive experiments on three multi-organ segmentation datasets demonstrate that our method outperforms current state-of-the-art methods.

Representation Recovering for Self-Supervised Pre-training on Medical Images

Background: Cellular senescence has recently been considered a new cancer hallmark. However, the factors regulating cellular senescence have not been well characterized. The aim of this study is to identify long non-coding RNAs (lncRNAs) associated with senescence and prognosis in patients with lung adenocarcinoma (LUAD). Methods: Using RNA sequence data from the Cancer Genome Atlas Lung Adenocarcinoma (TCGA-LUAD) and senescence genes from the CellAge database, a subset of senescence-related lncRNAs was first identified. Then, using univariate and multivariate Cox regression analyses, a senescence lncRNA signature (LUADSenLncSig) associated with LUAD prognosis was developed. Based on the median LUADSenLncSig risk score, LUAD patients were divided into high-risk and low-risk groups. Kaplan-Meier analysis was used to compare the overall survival (OS) in the high- and low-risk score subgroups. Differences in Gene Set Enrichment Analysis (GSEA), immune infiltration, tumor mutation burden (TMB), tumor immune dysfunction and exclusion (TIDE) module score, chemotherapy, and targeted therapy selection were also compared between the high-risk and low-risk groups. Results: A prognostic risk model was obtained consisting of the following nine senescence-related lncRNAs: LINC01116, AC005838.2, SH3PXD2A-AS1, VIMS-AS1, SH3BP5-AS1, AC092279.1, AC026355.1, AC027020.2, and LINC00996. The LUADSenLncSig high-risk group was associated with poor OS (hazard ratio = 1.17, 95% confidence interval = 1.102–1.242; p < 0.001). The accuracy of the model was further supported based on receiver operating characteristic (ROC), principal component analysis (PCA), and internal validation cohorts. In addition, a nomogram was developed consisting of LUADSenLncSig for LUAD prognosis, which is consistent with the actual probability of OS. Furthermore, immune infiltration analysis showed the low-risk group had a stronger anti-tumor immune response in the tumor microenvironment. Notably, the levels of immune checkpoint genes such as CTLA-4, PDCD-1, and CD274, and the TIDE scores were significantly higher in the low-risk subgroups than in high-risk subgroups (p < 0.001). This finding indicates the LUADSenLncSig can potentially predict immunotherapy efficacy. Conclusion: In this study, a lncRNA signature, LUADSenLncSig, that has dual functions of senescence phenotype identification and prognostic prediction as well as the potential to predict the LUAD response to immunotherapy was developed.

A novel senescence-related lncRNA signature that predicts prognosis and the tumor microenvironment in patients with lung adenocarcinoma

Risk of Urinary Tract Infection in Patients with Type 2 Diabetes Mellitus Treated with Dapagliflozin: A Systematic Review and Meta-Analysis of Randomized Controlled Trials

Objective . To explore the role and mechanism of epithelial-mesenchymal transition (EMT) mediated by inflammatory stress-induced TGF- β 1 in promoting arteriovenous fistula stenosis. Methods . The inflammatory cells HK-2 were cultured by adding TGF- β 1. The optimal stimulation time was determined after TGF- β 1 was added. HK-2 cells were divided into two groups, DMEM/F12 medium was added to one group (the control group), and the other group was treated with TGF- β 1 (10ng/ml) in serum-free DMEM/F12 medium to stimulate cell differentiation to mesenchymal. Results . TGF- β 1 was stably expressed after being transfected into EMT. The expression of TGF- β 1 in the experimental group was higher than that in the control group ( P < 0 . 05) 7 days after transfection. Western blot showed that TGF- β 1 protein expression was higher in the experimental group 7 days after transfection, and no TGF- β 1 protein expression was detected in the control group. The smooth muscle cells showed α -SMA expression in the control group, but no cells with expression of SMA and CD31/vWF were found at the same time; α -SMA expression was shown in smooth muscle cells and proliferative myofibroblasts, but no cells with expressions of SMA and CD31/ vWF were found at the same time. The observation group showed that the expression of α -SMA was detected in smooth muscle cells and proliferative myofibroblasts, CD31/vWF was also expressed in endothelial cells, and α -SMA and vWF were also observed in endothelial cells, but no CD31 expression was found. Conclusion . The inflammatory stress-induced TGF- β 1 could act on epithelial-mesenchymal transition and promote the degree of arteriovenous fistula stenosis

Role and Mechanism of Epithelial-Mesenchymal Transition Mediated by Inflammatory Stress-Induced TGF- β 1 in Promoting Arteriovenous Fistula Stenosis

Asymmetric lateral connections are one possible mechanism that can account for the direction selectivity of cortical neurons We present a mathematical analysis for a class of these models Contrasting with earlier theoretical work that has relied on methods from linear systems theory, we study the network's nonlinear dynamic properties that arise when the threshold nonlinearity of the neurons is taken into account We show that such networks have stimulus-locked traveling pulse solutions that are appropriate for modeling the responses of direction selective cortical neurons In addition, our analysis shows that outside a certain regime of stimulus speeds the stability of this solutions breaks down giving rise to another class of solutions that are characterized by specific spatio-temporal periodicity This predicts that if direction selectivity in the cortex is mainly achieved by asymmetric lateral connections lurching activity waves might be observable in ensembles of direction selective cortical neurons within appropriate regimes of the stimulus speed

/pdf/generating-velocity-tuning-by-asymmetric-recurrent-3ajkv28jf0.pdf

Xiaohui Xie

Papers

Representation Recovering for Self-Supervised Pre-training on Medical Images

A novel senescence-related lncRNA signature that predicts prognosis and the tumor microenvironment in patients with lung adenocarcinoma

Risk of Urinary Tract Infection in Patients with Type 2 Diabetes Mellitus Treated with Dapagliflozin: A Systematic Review and Meta-Analysis of Randomized Controlled Trials

Role and Mechanism of Epithelial-Mesenchymal Transition Mediated by Inflammatory Stress-Induced TGF- β 1 in Promoting Arteriovenous Fistula Stenosis

Generating velocity tuning by asymmetric recurrent connections