Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

The Oxford Dictionary of Philosophy

Deep reinforcement learning (RL) has achieved outstanding results in recent years. This has led to a dramatic increase in the number of applications and methods. Recent works have explored learning beyond single-agent scenarios and have considered multiagent learning (MAL) scenarios. Initial results report successes in complex multiagent domains, although there are several challenges to be addressed. The primary goal of this article is to provide a clear overview of current multiagent deep reinforcement learning (MDRL) literature. Additionally, we complement the overview with a broader analysis: (i) we revisit previous key components, originally presented in MAL and RL, and highlight how they have been adapted to multiagent deep reinforcement learning settings. (ii) We provide general guidelines to new practitioners in the area: describing lessons learned from MDRL works, pointing to recent benchmarks, and outlining open avenues of research. (iii) We take a more critical tone raising practical challenges of MDRL (e.g., implementation and computational demands). We expect this article will help unify and motivate future research to take advantage of the abundant literature that exists (e.g., RL and MAL) in a joint effort to promote fruitful research in the multiagent community.

A survey and critique of multiagent deep reinforcement learning

Artificial intelligence (AI) is becoming increasingly integrated in user-facing technology, but public understanding of these technologies is often limited. There is a need for additional HCI research investigating a) what competencies users need in order to effectively interact with and critically evaluate AI and b) how to design learner-centered AI technologies that foster increased user understanding of AI. This paper takes a step towards realizing both of these goals by providing a concrete definition of AI literacy based on existing research. We synthesize a variety of interdisciplinary literature into a set of core competencies of AI literacy and suggest several design considerations to support AI developers and educators in creating learner-centered AI. These competencies and design considerations are organized in a conceptual framework thematically derived from the literature. This paper's contributions can be used to start a conversation about and guide future research on AI literacy within the HCI community.

What is AI Literacy? Competencies and Design Considerations

2012 [AS13b, Wal12a]. 2013 [Alv13, Bar13, Cam13, Car13, CCS13, Cut13, Edi14, Swe13]. 2014 [CDS13, DN14, ND14, Rod14, Tym14a]. 2015 [DeL14a, DeL14b, She15b]. 2016 [AT16a, CCV16, CCV17, MR15, SDCT16a, SDCT16c, She16a]. 2017 [DG17, EC17, Fit18b, MJ18, TCSM17, TCM17]. 2018 [BG18b, DD17, Dec19b, Gol18, JI18, MKMP18, MMK18, PQH17, She18]. 2019 [Cut19, FI19, HPQ19, MP19a, MPRM19, MP19b, MS19, Rod18, She19a]. 2020 [DH19].

/pdf/a-complete-bibliography-of-publications-in-sigcse-bulletin-1aolnsl5m4.pdf

A Complete Bibliography of Publications in SIGCSE Bulletin: 2000{2009

The object of the jeopardy dice game Pig is to be the first player to reach 100 points Each player's turn consists of repeatedly rolling a die After each roll, the player is faced with two choices: roll again, or hold (decline to roll again) • If the player rolls a 1, the player scores nothing and it becomes the opponent's turn • If the player rolls a number other than 1, the number is added to the player's turn total and the player's turn continues • If the player holds, the turn total, the sum of the rolls during the turn, is added to the player's score, and it becomes the opponent's turn For such a simple dice game, one might expect a simple optimal strategy, such as in Blackjack (eg, "stand on 17" under certain circumstances, etc) As we shall see, this simple dice game yields a much more complex and intriguing optimal policy, described here for the first time The reader should be familiar with basic concepts and notation of probability and linear algebra

/pdf/optimal-play-of-the-dice-game-pig-21o26nnbp4.pdf

Optimal Play of the Dice Game Pig

The field of AI has changed significantly in the past couple of years and will likely continue to do so. Driven by a desire to expose our students to relevant and modern materials, we conducted two surveys, one of AI instructors and one of AI practitioners. The surveys were aimed at gathering information about the current state of the art of introducing AI as well as gathering input from practitioners in the field on techniques used in practice. In this paper, we present and briefly discuss the responses to those two surveys.

/pdf/a-survey-of-current-practice-and-teaching-of-ai-5etfd68g4a.pdf

A survey of current practice and teaching of AI

In this paper, we provide a practical framework for characterizing, evaluating and selecting reformulation techniques for reasoning about physical systems, with the long-term goal of automating the selection and application of these techniques. We view reformulation as a mapping from one encoding of a problem to another. A problem solving task is in turn accomplished by the application of a sequence of reformulations to an initial problem encoding to produce a final encoding that addresses the task. Our framework provides the terminology to specify the conditions under which a particular reformulation technique is applicable, the cost associated with performing the reformulation, and the effects of the reformulation with respect to the problem encoding. As such it provides the vocabulary to characterize the selection of a sequence of reformulation techniques as a planning problem. Our framework is sufficiently flexible to accommodate previously proposed properties and metrics for reformulation. We have used the framework to characterize a variety of reformulation techniques, three of which are presented in this paper.

/pdf/preliminary-thoughts-towards-a-practical-theory-of-176x66bcel.pdf

Preliminary Thoughts Towards a Practical Theory of Reformulation for Reasoning about Physical Systems

An Introduction to k-Means Clustering

In 2000, Hart and Mas-Colell introduced the important game-theoretic algorithm of regret matching. Players reach equilibrium play by tracking regrets for past plays, making future plays proportional to positive regrets. The technique is not only simple and intuitive; it has sparked a revolution in computer game play of some of the most difficult bluffing games, including clear domination of annual computer poker competitions. Since the algorithm is relatively recent, there are few curricular materials available to introduce regret-based algorithms to the next generation of researchers and practitioners in this area. These materials represent a modest first step towards making recent innovations more accessible to advanced Computer Science undergraduates, graduate students, interested researchers, and ambitious practitioners. In Section 2, we introduce the concept of player regret, describe the regret-matching algorithm, present a rock-paper-scissors worked example in the literate programming style, and suggest related exercises. Counterfactual Regret Minimization (CFR) is introduced in Section 3 with a worked example solving Kuhn Poker. Supporting code is provided for a substantive CFR exercise computing optimal play for 1-die-versus-1-die Dudo. In Section 4, we briefly mention means of “cleaning” approximately optimal computed policies, which can in many cases improve results. Section 5 covers an advanced application of CFR to games with repeated states (e.g. through imperfect recall abstraction) that can reduce computational complexity of a CFR training iteration from exponential to linear. Here, we use our independently devised game of Liar Die to demonstrate application of the algorithm. We then suggest that the reader apply the technique to 1-die-versus-1-die Dudo with a memory of 3 claims. In Section 6, we briefly discuss an open research problem: Among possible equilibrium strategies, how do we compute one that optimally exploits opponent errors? The reader is invited to modify our Liar Die example code to so as to gain insight to this interesting problem. Finally, in Section 7, we suggest further challenge problems and paths for continued learning.

/pdf/an-introduction-to-counterfactual-regret-minimization-28ud638k2g.pdf

Todd W. Neller

Papers

Optimal Play of the Dice Game Pig

A survey of current practice and teaching of AI

Preliminary Thoughts Towards a Practical Theory of Reformulation for Reasoning about Physical Systems

An Introduction to k-Means Clustering

An Introduction to Counterfactual Regret Minimization