Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

Although genomewide RNA expression analysis has become a routine tool in biomedical research, extracting biological insight from such information remains a major challenge. Here, we describe a powerful analytical method called Gene Set Enrichment Analysis (GSEA) for interpreting gene expression data. The method derives its power by focusing on gene sets, that is, groups of genes that share common biological function, chromosomal location, or regulation. We demonstrate how GSEA yields insights into several cancer-related data sets, including leukemia and lung cancer. Notably, where single-gene analysis finds little similarity between two independent studies of patient survival in lung cancer, GSEA reveals many biological pathways in common. The GSEA method is embodied in a freely available software package, together with an initial database of 1,325 biologically defined gene sets.

/pdf/gene-set-enrichment-analysis-a-knowledge-based-approach-for-mbpti6qljw.pdf

Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

抗原变异可使得多种致病微生物易于逃避宿主免疫应答。表达在感染红细胞表面的恶性疟原虫红细胞表面蛋白1（PfPMP1）与感染红细胞、内皮细胞、树突状细胞以及胎盘的单个或多个受体作用，在黏附及免疫逃避中起关键的作用。每个单倍体基因组var基因家族编码约60种成员，通过启动转录不同的var基因变异体为抗原变异提供了分子基础。

疟原虫var基因转换速率变化导致抗原变异[英]／Paul H, Robert P, Christodoulou Z, et al//Proc Natl Acad Sci U S A

http://bartellab.wi.mit.edu/publication_reprints/Bartel_Cell09.pdf

MicroRNAs: Target Recognition and Regulatory Functions

The formation of passivation films by interfacial reactions, though critical for applications ranging from advanced alloys to electrochemical energy storage, is often poorly understood. In this work, we explore the formation of an exemplar passivation film, the solid electrolyte interphase (SEI), which is responsible for stabilizing lithium-ion batteries. Using stochastic simulations based on quantum chemical calculations and data-driven chemical reaction networks, we directly model competition between SEI products at a mechanistic level for the first time. Our results recover the Peled-like separation of the SEI into inorganic and organic domains resulting from rich reactive competition without fitting parameters to experimental inputs. By conducting accelerated simulations at elevated temperature, we track SEI evolution, confirming the postulated reduction of lithium ethylene monocarbonate to dilithium ethylene monocarbonate and H2. These findings furnish fundamental insights into the dynamics of SEI formation and illustrate a path forward towards a predictive understanding of electrochemical passivation.

/pdf/towards-a-mechanistic-model-of-solid-electrolyte-interphase-1g1feswc.pdf

Towards a Mechanistic Model of Solid-Electrolyte Interphase Formation and Evolution in Lithium-ion Batteries

Recent mooring observations on the continental slope on the east side of the Dongsha Island in the northeastern South China Sea (SCS) showed that an along-slope bottom current can be generated when internal tides obliquely incident to the slope are dissipated near the seafloor. In this study, new mooring data collected on the south side of the Dongsha Island are used to explore the universality of internal wave driven the bottom currents and test the ability of the previous theory in estimating the along-slope current. The data show strong near-bottom energy dissipation due to the critical reflection of diurnal internal tides on the continental slope, with a time-mean depth-integrated dissipation rate of ~4.8×10-3 W/m2. Because of the obliquely incident of diurnal internal tides to the slope, near-bottom dissipation of internal tides generates a southwestward along-slope current, with the maximum velocity exceeding 6 cm/s. By comparing the observations, the previous theory for internal wave induced mean flows developed by Thorpe (1999) shows a good ability to estimate the along-slope bottom current velocity. Based on the theory, as well as modelled energy dissipation on the entire continental slope in the northeastern SCS, a map is obtained to quantitatively describe the along-slope bottom flow caused by internal tide breaking on the slope.

Along-slope bottom currents driven by dissipation of internal tides in the northeastern South China Sea

An Overview of Systematic Reviews of Traditional Chinese Medicine for Pneumonia in Children

Land market concentration, developers’ pricing decisions, and class monopoly rent in urban China

AbstractNowadays, the traffic data information explodes. Variety of malicious traffic is used to frequently evade the traffic identification systems. However, the extraction of massive traffic features and identification of malicious traffic using a single machine learning approach is ineffective in real-time recognition. To this end, this paper proposes the real-time traffic recognition method of Cluster-TRnet that focuses on the temporal and spatial behavior features of the application layer packets. K-means is used to filter disruptive traffic, lessen the learning burden and gather the malicious traffic features. Furthermore, the parameters of jointed convolutional neural network (CNN) and Long Short-Term Memory (LSTM) are adjusted, while the data inputted is augmented and grouped. The malicious traffic features precisely and consequently improve the accuracy of the recognition. Based on Spirent’s traffic data-set and the public traffic data-set, the proposed model is eventually validated. The experimental results show that the proposed method strictly identifies different types of traffic with an accuracy of 99.98%. Besides, it improves the efficiency of malicious traffic identification through filtering 83.51% of invalid traffic, and this dramatically saves the detection time by 78.18%.KeywordsK-meansCNNLSTMTraffic identification 

Xiaohui Xie

Papers

Towards a Mechanistic Model of Solid-Electrolyte Interphase Formation and Evolution in Lithium-ion Batteries

Along-slope bottom currents driven by dissipation of internal tides in the northeastern South China Sea

An Overview of Systematic Reviews of Traditional Chinese Medicine for Pneumonia in Children

Land market concentration, developers’ pricing decisions, and class monopoly rent in urban China

Cluster-TRnet: Jointed Model for Real-Time Traffic Identification with High Accuracy