Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

The concept of an agent has become important in both Artificial Intelligence (AI) and mainstream computer science. Our aim in this paper is to point the reader at what we perceive to be the most important theoretical and practical issues associated with the design and construction of intelligent agents. For convenience, we divide these issues into three areas (though as the reader will see, the divisions are at times somewhat arbitrary). Agent theory is concerned with the question of what an agent is, and the use of mathematical formalisms for representing and reasoning about the properties of agents. Agent architectures can be thought of as software engineering models of agents;researchers in this area are primarily concerned with the problem of designing software or hardware systems that will satisfy the properties specified by agent theorists. Finally, agent languages are software systems for programming and experimenting with agents; these languages may embody principles proposed by theorists. The paper is not intended to serve as a tutorial introduction to all the issues mentioned; we hope instead simply to identify the most important issues, and point to work that elaborates on them. The article includes a short review of current and potential applications of agent technology.

/pdf/intelligent-agents-theory-and-practice-5dchj7p8pc.pdf

Intelligent Agents: Theory and Practice

An Introduction to MultiAgent Systems.

Model-based Bayesian Reinforcement Learning (BRL) provides a principled solution to dealing with the exploration-exploitation trade-off, but such methods typically assume a fully observable environments. The few Bayesian RL methods that are applicable in partially observable domains, such as the Bayes-Adaptive POMDP (BA-POMDP), scale poorly. To address this issue, we introduce the Factored BA-POMDP model (FBA-POMDP), a framework that is able to learn a compact model of the dynamics by exploiting the underlying structure of a POMDP. The FBA-POMDP framework casts the problem as a planning task, for which we adapt the Monte-Carlo Tree Search planning algorithm and develop a belief tracking method to approximate the joint posterior over the state and model variables. Our empirical results show that this method outperforms a number of BRL baselines and is able to learn efficiently when the factorization is known, as well as learn both the factorization and the model parameters simultaneously.

/pdf/bayesian-reinforcement-learning-in-factored-pomdps-xpra8a8w97.pdf

Bayesian Reinforcement Learning in Factored POMDPs

Agent software is a rapidly developing area of research. However, the overuse of the word ‘agent’ has tended to mask the fact that, in reality, there is a truly heterogeneous body of research being carried out under this banner. This overview paper presents a typology of agents. Next, it places agents in context, defines them and then goes on, inter alia, to overview critically the rationales, hypotheses, goals, challenges and state-of-the-art demonstrators of the various agent types in our typology. Hence, it attempts to make explicit much of what is usually implicit in the agents literature. It also proceeds to overview some other general issues which pertain to all the types of agents in the typology. This paper largely reviews software agents, and it also contains some strong opinions that are not necessarily widely accepted by the agent community.

/pdf/software-agents-an-overview-3wmclhpj1p.pdf

Software agents: an overview

A multiagent approach to designing the transportation domain is presented The MARS system is described which models cooperative order scheduling within a society of shipping companies We argue why Distributed Artificial Intelligence (DAI) offers suitable tools to deal with the hard problems in this domain We present three important instances for DAI techniques that proved useful in the transportation application: cooperation among the agents, task decomposition and task allocation, and decentralised planning An extension of the contract net protocol for task decomposition and task allocation is presented; we show that it can be used to obtain good initial solutions for complex resource allocation problems By introducing global information based upon auction protocols, this initial solution can be improved significantly We demonstrate that the auction mechanism used for schedule optimisation can also be used for implementing dynamic replanning Experimental results are provided evaluating the performance of different scheduling strategies

/pdf/cooperative-transportation-scheduling-an-application-domain-8pthb2peyz.pdf

Cooperative transportation scheduling : an application domain for dai

Municipal wastewaters contain a multitude of organic trace pollutants. Often, their biodegradability by activated sludge microorganisms is decisive for their elimination during wastewater treatment. Since the amounts of micropollutants seem too low to serve as growth substrate, cometabolism is supposed to be the dominating biodegradation process. Nevertheless, as many biodegradation studies were performed without the intention to discriminate between metabolic and cometabolic processes, the specific contribution of the latter to substance transformations is often not clarified. This minireview summarizes current knowledge about the cometabolic degradation of organic trace pollutants by activated sludge and sludge-inherent microorganisms. Due to their relevance for communal wastewater contamination, the focus is laid on pharmaceuticals, personal care products, antibiotics, estrogens, and nonylphenols. Wherever possible, reference is made to the molecular process level, i.e., cometabolic pathways, involved enzymes, and formed transformation products. Particular cometabolic capabilities of different activated sludge consortia and various microbial species are highlighted. Process conditions favoring cometabolic activities are emphasized. Finally, knowledge gaps are identified, and research perspectives are outlined.

Cometabolic degradation of organic wastewater micropollutants by activated sludge and sludge-inherent microorganisms

For sometime now agent-based and multi-agent systems (MASs) have attracted the interest of researchers far beyond traditional computer science and artificial intelligence (AI). In this article we try to identify focal points of interest for researchers working in the area of distributed AI (DAI) and MAS as well as application-oriented researchers coming from related disciplines, e.g. electrical and mechanical engineering. We do this by presenting key research topics in DAI and MAS research and by identifying application domains in which the DAI and MAS technologies are most suitable. The research topics we discuss are separated into agent architectures and organisations, negotiation among agents, and self-adaptation of MAS using learning techniques. Regarding the application domains for these techniques we distinguish the application domains according to whether the agents control a physical or virtual body (Gestalt) or not. This separation of the application domains is not strict; it represents two ends of a continuum. On the one end of this continuum we have autonomous robot systems which act in a physical environment (sometimes referred to as hardware agents), and on the other end, we have abstract environments, such as in workflow systems, which rarely display the geometrical and physical aspects of the environment we are used to living in.

Multi-agent systems: which research for which applications

The MARS system is described which models cooperative scheduling witldn a society of shipping companies as a multiagent system. Emphasis is placed on the functionality of the system as a whole -- the solution of the global scheduling problem emerges from local decision-making and problem-solving strategies. An extension of the contract net protocol is presented; we show that it can be used to obtain good initial solutiom for complex resource allocation problems. By introducing global information based upon auction protocols, this initial solution can be improved significantly. Experimental results are provided evaluating the performance of different cooperative scheduling strategies. Although the concepts for resource scheduling are presented solely for the transportation domain, their abstraction is useful for a broad variety of resource allocation problems. The MARS system solves the dynmnic scheduling problem where no complete specification of the problem is available a priori; thus, it is designed as an on-line system based upon anytime algorithms.

/pdf/a-model-for-cooperative-transportation-scheduling-54chsc1vk0.pdf

A model for cooperative transportation scheduling

Low molecular weight organic substances (LMWOS) in soil and soil solution include mainly amino acids, carboxylic acids, and carbohydrates. Due to their high bioavailability they play a crucial role in the cycles of C and nutrients in soils. The variety of soil processes that involve LMWOS requires identifying their composition to elucidate reactions and transformations. In most studies, LMWOS are extracted under artificial conditions, e.g. batch experiments, which may overestimate the actual concentrations. This study measures the composition of carbohydrates and amino acids in solution of a Haplic Luvisol leached in a column experiment. A combined system for simultaneous leaching and blowout of CO2 was used to estimate LMWOS decomposition. 14 C-labeled glucose was added as a highly sensitive tracer to control the efficiency of the LMWOS extraction by leaching and to estimate LMWOS decomposition during leaching. High performance liquid chromatography (HPLC), optimized for soil extracts, was used to analyze LMWOS composition. For HPLC optimization, different preparations of leached solutions (filtration vs. centrifugation, and drying vs. no-drying) were compared. For sugar determination, drying had no influence on the solution concentrations. In contrast, amino acid concentrations significantly decreased by drying LMWOS eluted substances. Combining the HPLC identification of eluted substances with 14 C tracer application revealed that about 5% of the glucose could be leached unchanged within 786 min (13.1 h), whereas about 84% remained in the soil, 9% were decomposed to CO2, and 2% were transformed to other LMWOS and recovered in the soil solution. The total amino acid concentration (TAC) in soil solution was about 8.2mmol l � 1 , dominated by alanine (14.4% of TAC), glycine (13.4%), glutamic acid (9.9%), serine (9.4%), and leucine (9.3%). The total carbohydrate concentration was about 2.4mM, dominated by glucose (29.9%), glucuronic acid (26.8%), and galacturonic acid (17.3%). Ratios of hexoses to pentoses, amino sugars glucosamine to galactosamine, and neutral sugars to uronic acids were determined. All three parameters pointed to the dominant influence of plants as the source of LMWOS in the leached soil solution. Within the small contribution of microorganisms, bacteria dominated over fungi. These used biomarker ratios as well as LMWOS concentrations differed widely from the ones obtained with conventional batch extraction. More research is necessary to evaluate the application of these biomarkers to soil solutions. r 2007 Elsevier Ltd. All rights reserved.

http://wwwuser.gwdg.de/%7Ekuzyakov/Holger_SBB_2007_Sugars-AminoAcids-Leached-DOM.pdf

Klaus Fischer

Papers

Cooperative transportation scheduling : an application domain for dai

Cometabolic degradation of organic wastewater micropollutants by activated sludge and sludge-inherent microorganisms

Multi-agent systems: which research for which applications

A model for cooperative transportation scheduling

Carbohydrate and amino acid composition of dissolved organic matter leached from soil