https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Basics of qualitative research: Grounded theory procedures and techniques

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

How to Do Things With Words

http://www-anw.cs.umass.edu/~barto/courses/cs687/Sutton-Precup-Singh-AIJ99.pdf

Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning

A key element in the solution of reinforcement learning problems is the value function. The purpose of this function is to measure the long-term utility or value of any given state and it is important because an agent can use it to decide what to do next. A common problem in reinforcement learning when applied to systems having continuous states and action spaces is that the value function must operate with a domain consisting of real-valued variables, which means that it should be able to represent the value of infinitely many state and action pairs. For this reason, function approximators are used to represent the value function when a close-form solution of the optimal policy is not available. In this paper, we extend a previously proposed reinforcement learning algorithm so that it can be used with function approximators that generalize the value of individual experiences across both, state and action spaces. In particular, we discuss the benefits of using sparse coarse-coded function approximators to represent value functions and describe in detail three implementations: CMAC, instance-based, and case-based. Additionally, we discuss how function approximators having different degrees of resolution in different regions of the state and action spaces may influence the performance and learning efficiency of the agent. We propose a simple and modular technique that can be used to implement function approximators with non-uniform degrees of resolution so that it can represent the value function with higher accuracy in important regions of the state and action spaces. We performed extensive experiments in the double-integrator and pendulum swing up systems to demonstrate the proposed ideas.

Experiments with reinforcement learning in problems with continuous state and action spaces

Conversational agents are exploding in popularity. However, much work remains in the area of social conversation as well as free-form conversation over a broad range of domains and topics. To advance the state of the art in conversational AI, Amazon launched the Alexa Prize, a 2.5-million-dollar university competition where sixteen selected university teams were challenged to build conversational agents, known as socialbots, to converse coherently and engagingly with humans on popular topics such as Sports, Politics, Entertainment, Fashion and Technology for 20 minutes. The Alexa Prize offers the academic community a unique opportunity to perform research with a live system used by millions of users. The competition provided university teams with real user conversational data at scale, along with the user-provided ratings and feedback augmented with annotations by the Alexa team. This enabled teams to effectively iterate and make improvements throughout the competition while being evaluated in real-time through live user interactions. To build their socialbots, university teams combined state-of-the-art techniques with novel strategies in the areas of Natural Language Understanding, Context Modeling, Dialog Management, Response Generation, and Knowledge Acquisition. To support the efforts of participating teams, the Alexa Prize team made significant scientific and engineering investments to build and improve Conversational Speech Recognition, Topic Tracking, Dialog Evaluation, Voice User Experience, and tools for traffic management and scalability. This paper outlines the advances created by the university teams as well as the Alexa Prize team to achieve the common goal of solving the problem of Conversational AI.

/pdf/conversational-ai-the-science-behind-the-alexa-prize-59jplxuzxi.pdf

Conversational AI: The Science Behind the Alexa Prize.

Artificial Intelligence techniques have been successfully applied to several computer games. However in some kinds of computer games, like real-time strategy (RTS) games, traditional artificial intelligence techniques fail to play at a human level because of the vast search spaces that they entail. In this paper we present a real-time case based planning and execution approach designed to deal with RTS games. We propose to extract behavioral knowledge from expert demonstrations in form of individual cases. This knowledge can be reused via a case based behavior generator that proposes behaviors to achieve the specific open goals in the current plan. Specifically, we applied our technique to the W ARGUS domain with promising results.

https://www.cc.gatech.edu/faculty/ashwin/papers/er-07-11.pdf

Case-Based Planning and Execution for Real-Time Strategy Games

Learning, goals, and learning goals, Ashwin Ram and David B. Leake. Part 1 Current state of the field: planning to learn, Lawrence Hunter quantitative results concerning the utility of explanation-based learning, Steven Minton the use of explicit goals for knowledge to guide inference and learning, Ashwin Ram and Lawrence Hunter deriving categories to achieve goals, Lawrence W. Barsalou harpoons and long sticks - the interaction of theory and similarity in rule induction, Edward J. Wisniewski and Douglas L. Medlin introspective reasoning using meta-explanations for multistrategy learning, Ashwin Ram and Michael T. Cox goal-directed learning - a decision-theoretic model for deciding what to learn next, Marie desJardins goal-based explanation evaluation, David B. Leake planning to perceive, Louise Pryor and Gregg Collins planning and learning in PRODIGY - overview of an integrated architecture, Jaime Carbonell et al a learning model for the selection of problem-solving strategies in continuous physical systems, Xiaodong Xia and Dit-Yan Yeung explicitly biased generalization, Diana Gordon and Donald Perlis three levels of goal orientation in learning, Evelyn Ng and Carl Bereiter characterizing the application of computer simulations in education - instructional criteria, Jos J.A. van Berkum et al. Part 2 Current research and recent directions: goal-driven learning - fundamental issues and symposium report, David B. Leake and Ashwin Ram storage side effects - studying processing to understand learning, Lawrence W. Barsalou goal-driven learning in multistrategy reasoning and learning systems, Ashwin Ram et al inference to the best plan - a coherence theory of decision, Paul Thagard and Elijah Millgram toward goal-driven integration of explanation and action, David B. Leake learning as goal-driven inference, Ryszard Michalski and Ashwin Ram.

Goal-driven learning

The goal of transfer learning is to use the knowledge acquired in a set of source tasks to improve performance in a related but previously unseen target task. In this paper, we present a multilayered architecture named CAse-Based Reinforcement Learner (CARL). It uses a novel combination of Case-Based Reasoning (CBR) and Reinforcement Learning (RL) to achieve transfer while playing against the Game AI across a variety of scenarios in MadRTSTM, a commercial Real Time Strategy game. Our experiments demonstrate that CARL not only performs well on individual tasks but also exhibits significant performance gains when allowed to transfer knowledge from previous tasks.

/pdf/transfer-learning-in-real-time-strategy-games-using-hybrid-22hyzmf7no.pdf

Ashwin Ram

Papers

Experiments with reinforcement learning in problems with continuous state and action spaces

Conversational AI: The Science Behind the Alexa Prize.

Case-Based Planning and Execution for Real-Time Strategy Games

Goal-driven learning

Transfer learning in real-time strategy games using hybrid CBR/RL