Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

Although genomewide RNA expression analysis has become a routine tool in biomedical research, extracting biological insight from such information remains a major challenge. Here, we describe a powerful analytical method called Gene Set Enrichment Analysis (GSEA) for interpreting gene expression data. The method derives its power by focusing on gene sets, that is, groups of genes that share common biological function, chromosomal location, or regulation. We demonstrate how GSEA yields insights into several cancer-related data sets, including leukemia and lung cancer. Notably, where single-gene analysis finds little similarity between two independent studies of patient survival in lung cancer, GSEA reveals many biological pathways in common. The GSEA method is embodied in a freely available software package, together with an initial database of 1,325 biologically defined gene sets.

/pdf/gene-set-enrichment-analysis-a-knowledge-based-approach-for-mbpti6qljw.pdf

Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

抗原变异可使得多种致病微生物易于逃避宿主免疫应答。表达在感染红细胞表面的恶性疟原虫红细胞表面蛋白1（PfPMP1）与感染红细胞、内皮细胞、树突状细胞以及胎盘的单个或多个受体作用，在黏附及免疫逃避中起关键的作用。每个单倍体基因组var基因家族编码约60种成员，通过启动转录不同的var基因变异体为抗原变异提供了分子基础。

疟原虫var基因转换速率变化导致抗原变异[英]／Paul H, Robert P, Christodoulou Z, et al//Proc Natl Acad Sci U S A

http://bartellab.wi.mit.edu/publication_reprints/Bartel_Cell09.pdf

MicroRNAs: Target Recognition and Regulatory Functions

High resveratrol-loaded microcapsules with trehalose and OSA starch as the wall materials: Fabrication, characterization, and evaluation.

Summary Forward modeling and AVO analysis on wedge models are integrated in this paper to analyze the influence of thickness of a single thin bed on AVO responses. Study shows that tuning effect similar with post-stack amplitude can also be observed on pre-stack AVO intercept (P) and gradient (G) as well as their product P*G, which all reach their maximum at 1/4 wavelength and then gradually decrease to stable values with the increase of layer thickness. Quantitative analysis is subsequently conducted on AVO tuning effect through detailed comparison of AVO forward modeling on gas and brine saturated wedge models. It’s indicated that the AVO responses of both gas and brine saturated models are obviously strengthened at the tuning thickness without changing the original AVO types, namely the algebraic sign of intercept and gradient. That is to say, the differences of AVO responses between gas and brine saturated thin beds are strengthened due to tuning effect, thus making gas and brine saturated thin beds more distinguishable, which has great significances for fluid identification in thin beds. The method is then applied to hydrocarbon detection in Bohai Oilfield, and the effectiveness of the study is proved by drilling outcomes. 

Pre-Stack Avo Tuning Effect Analysis and the Application on Hydrocarbon Detection

Mesh deformation is a core task for 3D mesh reconstruction, but defining an efficient discrepancy between predicted and target meshes remains an open problem. A prevalent approach in current deep learning is the set-based approach which measures the discrepancy between two surfaces by comparing two randomly sampled point-clouds from the two meshes with Chamfer pseudo-distance. Nevertheless, the set-based approach still has limitations such as lacking a theoretical guarantee for choosing the number of points in sampled point-clouds, and the pseudo-metricity and the quadratic complexity of the Chamfer divergence. To address these issues, we propose a novel metric for learning mesh deformation. The metric is defined by sliced Wasserstein distance on meshes represented as probability measures that generalize the set-based approach. By leveraging probability measure space, we gain flexibility in encoding meshes using diverse forms of probability measures, such as continuous, empirical, and discrete measures via \textit{varifold} representation. After having encoded probability measures, we can compare meshes by using the sliced Wasserstein distance which is an effective optimal transport distance with linear computational complexity and can provide a fast statistical rate for approximating the surface of meshes. Furthermore, we employ a neural ordinary differential equation (ODE) to deform the input surface into the target shape by modeling the trajectories of the points on the surface. Our experiments on cortical surface reconstruction demonstrate that our approach surpasses other competing methods in multiple datasets and metrics.

Diffeomorphic Deformation via Sliced Wasserstein Distance Optimization for Cortical Surface Reconstruction

It is widely used PLGA in synthesizing active drug nanoparticles due to its high biocompatibility, biodegradability, and pelletization. In this study with PLGA as the carrier, the florfenicol nanoparticles (FF-PLGA NPs) were investigated to improve water solubility and controlled drug release for achieving higher antibacterial activity. The nanoparticles were characterized by a particle analyzer and a transmission electron microscope. The size and morphology of the nanoparticles were all <200 nm. The way of drugs binding to the nanoparticles was determined by Fourier to transform infrared spectroscopy (FTIR), X-ray diffraction (XRD), and differential scanning calorimetry (DSC). The nanoparticles with a high encapsulation rate (>80%) can achieve sustainable drug release in vitro. The addition of hydrophilic materials (2-HP-β-CD or PEG4000) to the FF-PLGA NPs improved drug solubility by about 3 and 2.5 times, respectively, and slowed down sudden release. The freeze-dried nanoparticles had high stability when stored at 4 °C. In vitro antibacterial test showed the antibacterial effects of the nanoparticle formulations on different strains were significantly improved compared with raw drugs. Hence, it is proved that PLGA as the carrier of the florfenicol nanoparticle delivery system is a new drug preparation with promising development prospects. 

Based on functional materials and PLGA for the florfenicol controlled release system and its antibacterial properties

Background Adrenal tumours are common in urology and endocrinology, and the diagnosis of adrenal tumours were mainly depends on imaging diagnosis. Howerver, misdiagnosis can still occur for some adrenal space-occupying lesions without specific manifestations or abnormal biochemical indexes. Methods We report the case of a 55-year-old patient with a soft-tissue mass in the left adrenal region, and have no specific manifestations or abnormalities in biochemical indexes. The patient had undergone open splenectomy 20 years ago for splenic rupture caused by traffic-accident trauma, and had a 10-year special history of hypertension. Because of the uncertain nature of the mass, surgical treatment was recommended. Results The surgeon managed to remove the left adrenal region mass. During the surgery, the adrenal source was excluded. In the histological examination, the splenic corpuscle and splenic medullary structure were seen under the microscope, and an accessory spleen was diagnosed. Conclusions The accessory spleen was located in the adrenal region rarely, and can easily be misdiagnosed as an adrenal tumour. When the cases show abnormal adrenal space-occupying lesions in imaging examinations, non-adrenal diseases should be considered. we need to combine different imaging techniques for analysis, and think more about it, avoid misdiagnosis leading to unnecessary surgery.

https://www.frontiersin.org/articles/10.3389/fsurg.2022.1017603/pdf

Xiaohui Xie

Papers

High resveratrol-loaded microcapsules with trehalose and OSA starch as the wall materials: Fabrication, characterization, and evaluation.

Pre-Stack Avo Tuning Effect Analysis and the Application on Hydrocarbon Detection

Diffeomorphic Deformation via Sliced Wasserstein Distance Optimization for Cortical Surface Reconstruction

Based on functional materials and PLGA for the florfenicol controlled release system and its antibacterial properties

Case report: Misdiagnosis of accessory spleen in the left adrenal region as an adrenal tumour after splenectomy