Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

Although genomewide RNA expression analysis has become a routine tool in biomedical research, extracting biological insight from such information remains a major challenge. Here, we describe a powerful analytical method called Gene Set Enrichment Analysis (GSEA) for interpreting gene expression data. The method derives its power by focusing on gene sets, that is, groups of genes that share common biological function, chromosomal location, or regulation. We demonstrate how GSEA yields insights into several cancer-related data sets, including leukemia and lung cancer. Notably, where single-gene analysis finds little similarity between two independent studies of patient survival in lung cancer, GSEA reveals many biological pathways in common. The GSEA method is embodied in a freely available software package, together with an initial database of 1,325 biologically defined gene sets.

/pdf/gene-set-enrichment-analysis-a-knowledge-based-approach-for-mbpti6qljw.pdf

Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

抗原变异可使得多种致病微生物易于逃避宿主免疫应答。表达在感染红细胞表面的恶性疟原虫红细胞表面蛋白1（PfPMP1）与感染红细胞、内皮细胞、树突状细胞以及胎盘的单个或多个受体作用，在黏附及免疫逃避中起关键的作用。每个单倍体基因组var基因家族编码约60种成员，通过启动转录不同的var基因变异体为抗原变异提供了分子基础。

疟原虫var基因转换速率变化导致抗原变异[英]／Paul H, Robert P, Christodoulou Z, et al//Proc Natl Acad Sci U S A

http://bartellab.wi.mit.edu/publication_reprints/Bartel_Cell09.pdf

MicroRNAs: Target Recognition and Regulatory Functions

With the growth of information on the Web, most users heavily rely on information access systems (e.g., search engines, recommender systems, etc.) in their daily lives. During this procedure, modeling users' satisfaction status plays an essential part in improving their experiences with the systems. In this paper, we aim to explore the benefits of using Electroencephalography (EEG) signals for satisfaction modeling in interactive information access system design. Different from existing EEG classification tasks, the arisen of satisfaction involves multiple brain functions, such as arousal, prototypicality, and appraisals, which are related to different brain topographical areas. Thus modeling user satisfaction raises great challenges to existing solutions. To address this challenge, we propose BTA, a Brain Topography Adaptive network with a multi-centrality encoding module and a spatial attention mechanism module to capture cognitive connectives in different spatial distances. We explore the effectiveness of BTA for satisfaction modeling in two popular information access scenarios, i.e., search and recommendation. Extensive experiments on two real-world datasets verify the effectiveness of introducing brain topography adaptive strategy in satisfaction modeling. Furthermore, we also conduct search result re-ranking task and video rating prediction task based on the satisfaction inferred from brain signals on search and recommendation scenarios, respectively. Experimental results show that brain signals extracted with BTA help improve the performance of interactive information access systems significantly.

/pdf/brain-topography-adaptive-network-for-satisfaction-modeling-22agduxe.pdf

Brain Topography Adaptive Network for Satisfaction Modeling in Interactive Information Access System

Aims The study was designed to develop a measurement for the motivation for and against change in methamphetamine users in the compulsory detoxification setting. Design This is a cross-sectional study. Setting The study was carried out in a compulsory detoxification center for male drug users in China. Participants A total of 228 male methamphetamine users who had undergone the program for at least 30 days. Measurements The motivation for/against change relating to compulsory detoxification was carried out using the Likert scale. A series of questionnaires were filled out by the participants, including the Egna Minnen Beträffande Uppfostran for rearing style, the Barratt Impulsiveness Scale-11, the adult ADHD self-report scale, and the Pittsburgh sleep quality index. Participants were also asked to recall the withdrawal symptoms before the program and to rate their current craving levels. Findings Motivations were grouped into three factors, namely, the expectation to use drugs upon the completion of the program (factor 1), the disagreement with the compulsory setting (factor 2), and the motivation to quit drug use (factor 3). Cronbach's alpha values were 0.8037, 0.8049, and 0.6292, respectively. The structural equation model showed that the overall motivation was characterized by motivation against change rather than that for change. The overall motivation was also directly affected by the current craving level and indirectly affected by the severity of addiction, paternal authoritarian upbringing style, and ADHD traits. Conclusion This study provided a measurement of motivation for and against change in subjects with drug misconduct and suggested that the motivation against change may disclose more psychological barriers than the motivation for change.

/pdf/the-motivation-against-change-in-male-methamphetamine-users-1xjczwt8.pdf

The motivation against change in male methamphetamine users in the compulsory detoxification setting

Most reinforced concrete structures serve under windy environments, and the carbonation resistance under that circumstance exhibits significant difference from that under the steady (no wind) environment. In this study, a windy environment was simulated using one self-developed wind tunnel, and alkali-activated slag/fly ash paste specimens were adopted for the carbonation under variant windy environments. Meanwhile, to reveal the effect of inner humidity on the carbonation, sliced alkali-activated materials (AAM) were mass-balanced first to variant humidity, and were then carbonated under a 2.5 m/s windy environment. With the assistance of computed tomography (CT), the structure of AAM at variant carbonation ages was rendered. The experimental result showed that wind is capable of promoting the exchange of moisture between the sample inside and the outer atmosphere, leading to faster carbonation as compared to that under no wind environment. When preconditioned to lower inner humidity, the carbonation rate of AAM was faster because the larger gaseous space benefited the intrusion of both CO2 and moisture. Furthermore, when preconditioned to lower humidity, the cracking extent of AAM was severer, which also contributed to the faster carbonation. Moreover, compared with ordinary Portland cement (OPC), the carbonation front on each instant 1D gray-scale value profile was broader, which suggested that the carbonation progress of AAM under windy environments was no longer controlled solely by diffusion. In addition, the gray-scale value on instant 1D profile fluctuated drastically, which verified cracking in AAM carbonated under windy environments. The current work not only deepens the understanding of the carbonation mechanism in-site (mostly under windy environments), but also helps to develop more environment-friendly construction material, with better durability performance.

/pdf/investigation-on-the-carbonation-behavior-of-alkali-1pgmhpi0.pdf

Investigation on the Carbonation Behavior of Alkali-Activated Pastes Served under Windy Environments

Modified parameterization for near-inertial waves

Organs-at-risk (OAR) delineation in computed tomography (CT) is an important step in Radiation Therapy (RT) planning. Recently, deep learning based methods for OAR delineation have been proposed and applied in clinical practice for separate regions of the human body (head and neck, thorax, and abdomen). However, there are few researches regarding the end-to-end whole-body OARs delineation because the existing datasets are mostly partially or incompletely annotated for such task. In this paper, our proposed end-to-end convolutional neural network model, called \textbf{AttentionAnatomy}, can be jointly trained with three partially annotated datasets, segmenting OARs from whole body. Our main contributions are: 1) an attention module implicitly guided by body region label to modulate the segmentation branch output; 2) a prediction re-calibration operation, exploiting prior information of the input images, to handle partial-annotation(HPA) problem; 3) a new hybrid loss function combining batch Dice loss and spatially balanced focal loss to alleviate the organ size imbalance problem. Experimental results of our proposed framework presented significant improvements in both Sorensen-Dice coefficient (DSC) and 95\% Hausdorff distance compared to the baseline model.

Xiaohui Xie

Papers

Brain Topography Adaptive Network for Satisfaction Modeling in Interactive Information Access System

The motivation against change in male methamphetamine users in the compulsory detoxification setting

Investigation on the Carbonation Behavior of Alkali-Activated Pastes Served under Windy Environments

Modified parameterization for near-inertial waves

AttentionAnatomy: A unified framework for whole-body organs at risk segmentation using multiple partially annotated datasets