Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

Although genomewide RNA expression analysis has become a routine tool in biomedical research, extracting biological insight from such information remains a major challenge. Here, we describe a powerful analytical method called Gene Set Enrichment Analysis (GSEA) for interpreting gene expression data. The method derives its power by focusing on gene sets, that is, groups of genes that share common biological function, chromosomal location, or regulation. We demonstrate how GSEA yields insights into several cancer-related data sets, including leukemia and lung cancer. Notably, where single-gene analysis finds little similarity between two independent studies of patient survival in lung cancer, GSEA reveals many biological pathways in common. The GSEA method is embodied in a freely available software package, together with an initial database of 1,325 biologically defined gene sets.

/pdf/gene-set-enrichment-analysis-a-knowledge-based-approach-for-mbpti6qljw.pdf

Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

抗原变异可使得多种致病微生物易于逃避宿主免疫应答。表达在感染红细胞表面的恶性疟原虫红细胞表面蛋白1（PfPMP1）与感染红细胞、内皮细胞、树突状细胞以及胎盘的单个或多个受体作用，在黏附及免疫逃避中起关键的作用。每个单倍体基因组var基因家族编码约60种成员，通过启动转录不同的var基因变异体为抗原变异提供了分子基础。

疟原虫var基因转换速率变化导致抗原变异[英]／Paul H, Robert P, Christodoulou Z, et al//Proc Natl Acad Sci U S A

http://bartellab.wi.mit.edu/publication_reprints/Bartel_Cell09.pdf

MicroRNAs: Target Recognition and Regulatory Functions

Recent advance in Dense Retrieval (DR) techniques has significantly improved the effectiveness of first-stage retrieval. Trained with large-scale supervised data, DR models can encode queries and documents into a low-dimensional dense space and conduct effective semantic matching. However, previous studies have shown that the effectiveness of DR models would drop by a large margin when the trained DR models are adopted in a target domain that is different from the domain of the labeled data. One of the possible reasons is that the DR model has never seen the target corpus and thus might be incapable of mitigating the difference between the training and target domains. In practice, unfortunately, training a DR model for each target domain to avoid domain shift is often a difficult task as it requires additional time, storage, and domain-specific data labeling, which are not always available. To address this problem, in this paper, we propose a novel DR framework named Disentangled Dense Retrieval (DDR) to support effective and flexible domain adaptation for DR models. DDR consists of a Relevance Estimation Module (REM) for modeling domain-invariant matching patterns and several Domain Adaption Modules (DAMs) for modeling domain-specific features of multiple target corpora. By making the REM and DAMs disentangled, DDR enables a flexible training paradigm in which REM is trained with supervision once and DAMs are trained with unsupervised data. Comprehensive experiments in different domains and languages show that DDR significantly improves ranking performance compared to strong DR baselines and substantially outperforms traditional retrieval methods in most scenarios.

Disentangled Modeling of Domain and Relevance for Adaptable Dense Retrieval

Physicochemical properties and aroma release of gelatin-stabilized rapeseed oil-in-water emulsions as affected by pH

Purpose To observe and analyze the clinical and multi-mode imaging features of eyes with PHOMS, and to introduce two cases of PHOMS which underwent multi-mode imaging. Methods Retrospective clinical observational study. A total of 26 patients (37 eyes) with hyperreflective structures surrounded by hyporeflective edges around the optic discs who were examined and diagnosed at Shandong Eye Hospital between January 2019 and June 2021 were included in the study. Among these patients, 12 were male and 14 were female. Fifteen were monocular. The average age was 39 years. All patients underwent the following examinations: Best-corrected visual acuity (BCVA), intraocular pressure examinations, slit-lamp anterior segment examinations, indirect ophthalmoscopy, visual field examinations, fundus color photography, fundus autofluorescence (FAF), optical coherence tomography (OCT), and optical coherence tomography angiography (OCTA). Some of the patients were examined with fundus fluorescein angiography (FFA). Clinical data and imaging characteristics from the OCT, OCTA, and FFA were analyzed retrospectively. Results We found the hyperreflective structures surrounded by hyporeflective edges around the optic discs in 37 eyes. EDI-OCT results revealed hyperreflective structures surrounded by hyporeflective edges around the optic discs in all eyes. Typical hyperreflexia lesions occurred around the optic disc, located subretinally and above Bruch's membrane. OCTA revealed that the highly reflective perioptic material also had vascular structures. Conclusion EDI-OCT of PHOMS showed hyperreflective structures surrounded by hyporeflective edges around all of the optic discs. Infra-red photography showed temporal hyperreflexia. These characteristics can be seen in a variety of diseases and may be a relatively common feature revealed by EDI-OCT scanning. These characteristics may also be seen in elderly patients as well as children. PHOMS may be found in optic disc drusen (ODD), tilted disc syndrome (TDS), optic neuritis, ischemic optic neuropathy, and in white dot syndromes. Few patients may be developed into macular neovascularization (MNV). In order to improve the accuracy and robustness of the conclusions and provide better clinical guidance, we need to conduct more comprehensive research in the subsequent clinical work.

/pdf/clinical-and-multi-mode-imaging-features-of-eyes-with-19wfdl9v.pdf

Clinical and Multi-Mode Imaging Features of Eyes With Peripapillary Hyperreflective Ovoid Mass-Like Structures

Web-based image search engines differ from Web search engines greatly. The intents or goals behind human interactions with image search engines are different. In image search, users mainly search images instead of Web pages or online services. It is essential to know why people search for images because user satisfaction may vary as intent varies. Furthermore, image search engines show results differently. For example, grid-based placement is used in image search instead of the linear result list, so that users can browse result list both vertically and horizontally. Different user intents and system UIs lead to different user behavior. Thus, it is hard to apply standard user behavior models developed for general Web search to image search. To better understand user intent and behavior in image search scenarios, we plan to conduct the lab-based user study, field study and commercial search log analysis. We then propose user behavior models based on the observation from data analysis to improve the performance of Web image search engines.

User Behavior Modeling for Web Image Search

Many biological processes are governed by protein-ligand interactions. One such example is the recognition of self and nonself cells by the immune system. This immune response process is regulated by the major histocompatibility complex (MHC) protein which is encoded by the human leukocyte antigen (HLA) complex. Understanding the binding potential between MHC and peptides can lead to the design of more potent, peptide-based vaccines and immunotherapies for infectious autoimmune diseases.

We apply machine learning techniques from the natural language processing (NLP) domain to address the task of MHC-peptide binding prediction. More specifically, we introduce a new distributed representation of amino acids, name HLA-Vec, that can be used for a variety of downstream proteomic machine learning tasks. We then propose a deep convolutional neural network architecture, name HLA-CNN, for the task of HLA class I-peptide binding prediction. Experimental results show combining the new distributed representation with our HLA-CNN architecture acheives state-of-the-art results in the majority of the latest two Immune Epitope Database (IEDB) weekly automated benchmark datasets. We further apply our model to predict binding on the human genome and identify 15 genes with potential for self binding. Codes are available at https://github.com/uci-cbcl/HLA-bind.

/pdf/hla-class-i-binding-prediction-via-convolutional-neural-3tvizstvu6.pdf

Xiaohui Xie

Papers

Disentangled Modeling of Domain and Relevance for Adaptable Dense Retrieval

Physicochemical properties and aroma release of gelatin-stabilized rapeseed oil-in-water emulsions as affected by pH

Clinical and Multi-Mode Imaging Features of Eyes With Peripapillary Hyperreflective Ovoid Mass-Like Structures

User Behavior Modeling for Web Image Search

HLA class I binding prediction via convolutional neural networks