Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

Although genomewide RNA expression analysis has become a routine tool in biomedical research, extracting biological insight from such information remains a major challenge. Here, we describe a powerful analytical method called Gene Set Enrichment Analysis (GSEA) for interpreting gene expression data. The method derives its power by focusing on gene sets, that is, groups of genes that share common biological function, chromosomal location, or regulation. We demonstrate how GSEA yields insights into several cancer-related data sets, including leukemia and lung cancer. Notably, where single-gene analysis finds little similarity between two independent studies of patient survival in lung cancer, GSEA reveals many biological pathways in common. The GSEA method is embodied in a freely available software package, together with an initial database of 1,325 biologically defined gene sets.

/pdf/gene-set-enrichment-analysis-a-knowledge-based-approach-for-mbpti6qljw.pdf

Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

抗原变异可使得多种致病微生物易于逃避宿主免疫应答。表达在感染红细胞表面的恶性疟原虫红细胞表面蛋白1（PfPMP1）与感染红细胞、内皮细胞、树突状细胞以及胎盘的单个或多个受体作用，在黏附及免疫逃避中起关键的作用。每个单倍体基因组var基因家族编码约60种成员，通过启动转录不同的var基因变异体为抗原变异提供了分子基础。

疟原虫var基因转换速率变化导致抗原变异[英]／Paul H, Robert P, Christodoulou Z, et al//Proc Natl Acad Sci U S A

http://bartellab.wi.mit.edu/publication_reprints/Bartel_Cell09.pdf

MicroRNAs: Target Recognition and Regulatory Functions

Current state-of-the-art nonparametric Bayesian text clustering methods model documents through multinomial distribution on bags of words. Although these methods can effectively utilize the word burstiness representation of documents and achieve decent performance, they do not explore the sequential information of text and relationships among synonyms. In this paper, the documents are modeled as the joint of bags of words, sequential features and word embeddings. We proposed Sequential Embedding induced Dirichlet Process Mixture Model (SiDPMM) to effectively exploit this joint document representation in text clustering. The sequential features are extracted by the encoder-decoder component. Word embeddings produced by the continuous-bag-of-words (CBOW) model are introduced to handle synonyms. Experimental results demonstrate the benefits of our model in two major aspects: (1) improved performance across multiple diverse text datasets in terms of the normalized mutual information (NMI); (2) more accurate inference of ground truth cluster numbers with regularization effect on tiny outlier clusters.

Sequential Embedding Induced Text Clustering, a Non-parametric Bayesian Approach

Recently, pre-trained language models (PTM) have achieved great success on ad hoc search. However, the performance decline in low-resource scenarios demonstrates the capability of PTM has not been inspired fully. As a novel paradigm to apply PTM to downstream tasks, prompt learning is a feasible scheme to boost PTM’s performance by aligning the pre-training task and downstream task. This paper investigates the effectiveness of the standard prompt learning paradigm on the ad hoc search task. Based on various PTMs, two types of prompts are tailored for the ad hoc search task. Overall experimental results on the MS Marco dataset show the credible better performance of our prompt learning method than fine-tuning based methods and another previous prompt learning based model. Experiments conducted in various resource scenarios show the stability of prompt learning. RoBERTa and T5 deliver better results compared to BM25 using 100 training queries utilizing prompt learning, while fine-tuning based methods need more data. Further analysis shows the significance of the uniformity of tasks’ format and adding continuous tokens into training in our prompt learning method. 

Enhance Performance of Ad-hoc Search via Prompt Learning

Enhanced near-bottom circulation and mixing driven by the Surface Eddies over Abyssal Seamounts

Identifying Unanticipated Genes and Mechanisms in Serious Mental Illness

Integration in the head-direction system is a computation by which horizontal angular head velocity signals from the vestibular nuclei are integrated to yield a neural representation of head direction. In the thala-mus, the postsubiculum and the mammillary nuclei, the head-direction representation has the form of a place code: neurons have a preferred head direction in which their firing is maximal [Blair and Sharp, 1995, Blair et al., 1998, ?].

Integration is a difficult computation, given that head-velocities can vary over a large range. Previous models of the head-direction system relied on the assumption that the integration is achieved in a firing-rate-based attractor network with a ring structure. In order to correctly integrate head-velocity signals during high-speed head rotations, very fast synaptic dynamics had to be assumed.

Here we address the question whether integration in the head-direction system is possible with slow synapses, for example excitatory NMDA and inhibitory GABA(B) type synapses. For neural networks with such slow synapses, rate-based dynamics are a good approximation of spiking neurons [Ermentrout, 1994]. We find that correct integration during high-speed head rotations imposes strong constraints on possible network architectures.

/pdf/a-theory-of-neural-integration-in-the-head-direction-system-2mzxwwnl8q.pdf

Xiaohui Xie

Papers

Sequential Embedding Induced Text Clustering, a Non-parametric Bayesian Approach

Enhance Performance of Ad-hoc Search via Prompt Learning

Enhanced near-bottom circulation and mixing driven by the Surface Eddies over Abyssal Seamounts

Identifying Unanticipated Genes and Mechanisms in Serious Mental Illness

A theory of neural integration in the head-direction system