Matthew D. Taylor

Bio: Matthew D. Taylor is an academic researcher from New South Wales Department of Primary Industries. The author has contributed to research in topics: Reinforcement learning & Transfer of learning. The author has an hindex of 58, co-authored 484 publications receiving 15024 citations. Previous affiliations of Matthew D. Taylor include Newcastle University & University of Alberta.

Transfer Learning for Reinforcement Learning Domains: A Survey

Abstract: The reinforcement learning paradigm is a popular way to address problems that have only limited environmental feedback, rather than correctly labeled examples, as is common in other machine learning contexts. While significant progress has been made to improve learning in a single task, the idea of transfer learning has only recently been applied to reinforcement learning tasks. The core idea of transfer is that experience gained in learning to perform one task can help improve learning performance in a related, but different, task. In this article we present a framework that classifies transfer learning methods in terms of their capabilities and goals, and then use it to survey the existing literature, as well as to suggest future directions for transfer learning work.

Helminth parasites – masters of regulation

Abstract: Summary: Immune regulation by parasites is a global concept that includes suppression, diversion, and conversion of the host immune response to the benefit of the pathogen. While many microparasites escape immune attack by antigenic variation or sequestration in specialized niches, helminths appear to thrive in exposed extracellular locations, such as the lymphatics, bloodstream, or gastrointestinal tract. We review here the multiple layers of immunoregulation that have now been discovered in helminth infection and discuss both the cellular and the molecular interactions involved. Key events among the host cell population are dominance of the T-helper 2 cell (Th2) phenotype and the selective loss of effector activity, against a background of regulatory T cells, alternatively activated macrophages, and Th2-inducing dendritic cells. Increasingly, there is evidence of important effects on other innate cell types, particularly mast cells and eosinophils. The sum effect of these changes to host reactivity is to create an anti-inflammatory environment, which is most favorable to parasite survival. We hypothesize therefore that parasites have evolved specific molecular strategies to induce this conducive landscape, and we review the foremost candidate immunomodulators released by helminths, including cytokine homologs, protease inhibitors, and an intriguing set of novel products implicated in immune suppression.

Suppression of allergic airway inflammation by helminth-induced regulatory T cells

Abstract: Allergic diseases mediated by T helper type (Th) 2 cell immune responses are rising dramatically in most developed countries. Exaggerated Th2 cell reactivity could result, for example, from diminished exposure to Th1 cell‐inducing microbial infections. Epidemiological studies, however, indicate that Th2 cell‐stimulating helminth parasites may also counteract allergies, possibly by generating regulatory T cells which suppress both Th1 and Th2 arms of immunity. We therefore tested the ability of the Th2 cell‐inducing gastrointestinal nematode Heligmosomoides polygyrus to influence experimentally induced airway allergy to ovalbumin and the house dust mite allergen Der p 1. Inflammatory cell infiltrates in the lung were suppressed in infected mice compared with uninfected controls. Suppression was reversed in mice treated with antibodies to CD25. Most notably, suppression was transferable with mesenteric lymph node cells (MLNC) from infected animals to uninfected sensitized mice, demonstrating that the effector phase was targeted. MLNC from infected animals contained elevated numbers of CD4 � CD25 � Foxp3 � T cells, higher TGF- � expression, and produced strong interleukin (IL)-10 responses to parasite antigen. However, MLNC from IL-10‐deficient animals transferred suppression to sensitized hosts, indicating that IL-10 is not the primary modulator of the allergic response. Suppression was associated with CD4 � T cells from MLNC, with the CD4 � CD25 � marker defining the most active population. These data support the contention

Continental-scale animal tracking reveals functional movement classes across marine taxa.

Abstract: Acoustic telemetry is a principle tool for observing aquatic animals, but coverage over large spatial scales remains a challenge. To resolve this, Australia has implemented the Integrated Marine Observing System’s Animal Tracking Facility which comprises a continental-scale hydrophone array and coordinated data repository. This national acoustic network connects localized projects, enabling simultaneous monitoring of multiple species over scales ranging from 100 s of meters to 1000 s of kilometers. There is a need to evaluate the utility of this national network in monitoring animal movement ecology, and to identify the spatial scales that the network effectively operates over. Cluster analyses assessed movements and residency of 2181 individuals from 92 species, and identified four functional movement classes apparent only through aggregating data across the entire national network. These functional movement classes described movement metrics of individuals rather than species, and highlighted the plasticity of movement patterns across and within populations and species. Network analyses assessed the utility and redundancy of each component of the national network, revealing multiple spatial scales of connectivity influenced by the geographic positioning of acoustic receivers. We demonstrate the significance of this nationally coordinated network of receivers to better reveal intra-specific differences in movement profiles and discuss implications for effective management.

A survey and critique of multiagent deep reinforcement learning

Abstract: Deep reinforcement learning (RL) has achieved outstanding results in recent years. This has led to a dramatic increase in the number of applications and methods. Recent works have explored learning beyond single-agent scenarios and have considered multiagent learning (MAL) scenarios. Initial results report successes in complex multiagent domains, although there are several challenges to be addressed. The primary goal of this article is to provide a clear overview of current multiagent deep reinforcement learning (MDRL) literature. Additionally, we complement the overview with a broader analysis: (i) we revisit previous key components, originally presented in MAL and RL, and highlight how they have been adapted to multiagent deep reinforcement learning settings. (ii) We provide general guidelines to new practitioners in the area: describing lessons learned from MDRL works, pointing to recent benchmarks, and outlining open avenues of research. (iii) We take a more critical tone raising practical challenges of MDRL (e.g., implementation and computational demands). We expect this article will help unify and motivate future research to take advantage of the abundant literature that exists (e.g., RL and MAL) in a joint effort to promote fruitful research in the multiagent community.

A Survey on Transfer Learning

Abstract: A major assumption in many machine learning and data mining algorithms is that the training and future data must be in the same feature space and have the same distribution. However, in many real-world applications, this assumption may not hold. For example, we sometimes have a classification task in one domain of interest, but we only have sufficient training data in another domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In such cases, knowledge transfer, if done successfully, would greatly improve the performance of learning by avoiding much expensive data-labeling efforts. In recent years, transfer learning has emerged as a new learning framework to address this problem. This survey focuses on categorizing and reviewing the current progress on transfer learning for classification, regression, and clustering problems. In this survey, we discuss the relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift. We also explore some potential future issues in transfer learning research.

Machine learning

Abstract: Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Phd by thesis

Abstract: Deposits of clastic carbonate-dominated (calciclastic) sedimentary slope systems in the rock record have been identified mostly as linearly-consistent carbonate apron deposits, even though most ancient clastic carbonate slope deposits fit the submarine fan systems better. Calciclastic submarine fans are consequently rarely described and are poorly understood. Subsequently, very little is known especially in mud-dominated calciclastic submarine fan systems. Presented in this study are a sedimentological core and petrographic characterisation of samples from eleven boreholes from the Lower Carboniferous of Bowland Basin (Northwest England) that reveals a >250 m thick calciturbidite complex deposited in a calciclastic submarine fan setting. Seven facies are recognised from core and thin section characterisation and are grouped into three carbonate turbidite sequences. They include: 1) Calciturbidites, comprising mostly of highto low-density, wavy-laminated bioclast-rich facies; 2) low-density densite mudstones which are characterised by planar laminated and unlaminated muddominated facies; and 3) Calcidebrites which are muddy or hyper-concentrated debrisflow deposits occurring as poorly-sorted, chaotic, mud-supported floatstones. These

Influence of Diet On the Distribtion of Nitrogen Isotopes in Animals

Abstract: The influence of diet on the distribution of nitrogen isotopes in animals was investigated by analyzing animals grown in the laboratory on diets of constant nitrogen isotopic composition. The isotopic composition of the nitrogen in an animal reflects the nitrogen isotopic composition of its diet. The δ^(15)N values of the whole bodies of animals are usually more positive than those of their diets. Different individuals of a species raised on the same diet can have significantly different δ^(15)N values. The variability of the relationship between the δ^(15)N values of animals and their diets is greater for different species raised on the same diet than for the same species raised on different diets. Different tissues of mice are also enriched in ^(15)N relative to the diet, with the difference between the δ^(15)N values of a tissue and the diet depending on both the kind of tissue and the diet involved. The δ^(15)N values of collagen and chitin, biochemical components that are often preserved in fossil animal remains, are also related to the δ^(15)N value of the diet. The dependence of the δ^(15)N values of whole animals and their tissues and biochemical components on the δ^(15)N value of diet indicates that the isotopic composition of animal nitrogen can be used to obtain information about an animal's diet if its potential food sources had different δ^(15)N values. The nitrogen isotopic method of dietary analysis probably can be used to estimate the relative use of legumes vs non-legumes or of aquatic vs terrestrial organisms as food sources for extant and fossil animals. However, the method probably will not be applicable in those modern ecosystems in which the use of chemical fertilizers has influenced the distribution of nitrogen isotopes in food sources. The isotopic method of dietary analysis was used to reconstruct changes in the diet of the human population that occupied the Tehuacan Valley of Mexico over a 7000 yr span. Variations in the δ^(15)C and δ^(15)N values of bone collagen suggest that C_4 and/or CAM plants (presumably mostly corn) and legumes (presumably mostly beans) were introduced into the diet much earlier than suggested by conventional archaeological analysis.