Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

The rise of multi-million-item dataset initiatives has enabled data-hungry machine learning algorithms to reach near-human semantic classification performance at tasks such as visual object and scene recognition. Here we describe the Places Database, a repository of 10 million scene photographs, labeled with scene semantic categories, comprising a large and diverse list of the types of environments encountered in the world. Using the state-of-the-art Convolutional Neural Networks (CNNs), we provide scene classification CNNs (Places-CNNs) as baselines, that significantly outperform the previous approaches. Visualization of the CNNs trained on Places shows that object detectors emerge as an intermediate representation of scene classification. With its high-coverage and high-diversity of exemplars, the Places Database along with the Places-CNNs offer a novel resource to guide future progress on scene recognition problems.

Places: A 10 Million Image Database for Scene Recognition

ConceptNet is a freely available commonsense knowledge base and natural-language-processing tool-kit which supports many practical textual-reasoning tasks over real-world documents including topic-gisting, analogy-making, and other context oriented inferences. The knowledge base is a semantic network presently consisting of over 1.6 million assertions of commonsense knowledge encompassing the spatial, physical, social, temporal, and psychological aspects of everyday life. ConceptNet is generated automatically from the 700 000 sentences of the Open Mind Common Sense Project — a World Wide Web based collaboration with over 14 000 authors.

/pdf/conceptnet-a-practical-commonsense-reasoning-tool-kit-3723gsymn1.pdf

ConceptNet — A Practical Commonsense Reasoning Tool-Kit

Experimental Psychology Its Scope and Method. IV. Learning and Memory. By Jean-Francois Le Ny, G. De Montpellier, G. Oleron and Cesar Flores. Translated by Louise Elkington. Edited by P. Fraisse and Jean Piaget. Pp. viii + 376. (Routledge and Kegan Paul: London, April 1970.) 80s.

Learning and Memory

When I started out as a newly hatched PhD student, back in the day, one of the first articles I read and understood (or at least thought that I understood) was Ray Reiter’s classic article on default logic (Reiter, 1980).This was some years after the famous ‘non-monotonic logic’ issue of Artificial Intelligence in which that article appeared, but default logic was still one of the leading approaches, a tribute to the simplicity and power of the theory. As a result of reading the article, I became fascinated by both default logic and, more generally, non-monotonic logics. However, despite my fascination, these approaches never seemed terribly useful for the kinds of problem that I was supposed to be studying—problems like those in medical decision making—and so I eventually lost interest. In fact non-monotonic logics seemed to me, and to many people at the time I think, not to be terribly useful for anything. They were interesting, and clearly relevant to the long-term goals of Artificial Intelligence as a discipline, but not of any immediate practical importance. This verdict, delivered at the end of the 1980s, continued, I think, to be true for the next few years while researchers working in non-monotonic logics studied problems that to outsiders seemed to be ever more obscure. However, by the end of the 1990s, it was becoming clear, even to folk as short-sighted as I, that non-monotonic logics were getting to the point at which they could be used to solve practical problems. Knowledge in action shows quite how far these techniques have come. The reason that non-monotonic logics were invented was, of course, in order to use logic to reason about the world. Our knowledge of the world is typically incomplete, and so, in order to reason about it, one has to make assumptions about things one does not know. This, in turn, requires mechanisms for both making assumptions and then retracting them if and when they turn out not to be true. Non-monotonic logics are intended to handle this kind of assumption making and retracting, providing a mechanism that has the clean semantics of logic, but which has a non-monotonic set of conclusions. Much of the early work on non-monotonic logics was concerned with theoretical reasoning, that is reasoning about the beliefs of an agent—what the agent believes to be true. Theoretical reasoning is the domain of all those famous examples like ‘Typically birds fly. Tweety is a bird, so does Tweety fly?’, and the fact that so much of non-monotonic reasoning seemed to focus on theoretical reasoning was why I lost interest in it. I became much more concerned with practical reasoning—that is reasoning about what an agent should do—and non-monotonic reasoning seemed to me to have nothing interesting to say about practical reasoning. Of course I was wrong. When one tries to formulate any kind of description of the world as the basis for planning, one immediately runs into applications of non-monotonic logics, for example in keeping track of the state of a changing world. It is this use of non-monotonic logic that is at the heart of Knowledge in action. Building on the McCarthy’s situation calculus, Knowledge in action constructs a theory of action that encompasses a very large part of what an agent requires to reason about the world. As Reiter says in the final chapter,

Knowledge in Action: Logical Foundations for Specifying and Implementing Dynamical Systems

Open Mind Common Sense is a knowledge acquisition system designed to acquire commonsense knowledge from the general public over the web. We describe and evaluate our first fielded system, which enabled the construction of a 450,000 assertion commonsense knowledge base. We then discuss how our second-generation system addresses weaknesses discovered in the first. The new system acquires facts, descriptions, and stories by allowing participants to construct and fill in natural language templates. It employs word-sense disambiguation and methods of clarifying entered knowledge, analogical inference to provide feedback, and allows participants to validate knowledge and in turn each other.

/pdf/open-mind-common-sense-knowledge-acquisition-from-the-1zssg2ka8k.pdf

Open Mind Common Sense: Knowledge Acquisition from the General Public

https://scalar.usc.edu/works/meet-my-friend-watson-1/media/Beyond%20Jeopardy.pdf

Watson: beyond jeopardy!

To endow computers with common sense is one of the major long-term goals of artificial intelligence research. One approach to this problem is to formalize commonsense reasoning using mathematical logic. Commonsense Reasoning: An Event Calculus Based Approach is a detailed, high-level reference on logic-based commonsense reasoning. It uses the event calculus, a highly powerful and usable tool for commonsense reasoning, which Erik Mueller demonstrates as the most effective tool for the broadest range of applications. He provides an up-to-date work promoting the use of the event calculus for commonsense reasoning, and bringing into one place information scattered across many books and papers. Mueller shares the knowledge gained in using the event calculus and extends the literature with detailed event calculus solutions that span many areas of the commonsense world.

The Second Edition features new chapters on commonsense reasoning using unstructured information including the Watson system, commonsense reasoning using answer set programming, and techniques for acquisition of commonsense knowledge including crowdsourcing.



Drawing upon years of practical experience and using numerous examples and illustrative applications Erik Mueller shows you the keys to mastering commonsense reasoning. You?ll be able to:

Understand techniques for automated commonsense reasoning
Incorporate commonsense reasoning into software solutions
Acquire a broad understanding of the field of commonsense reasoning.
Gain comprehensive knowledge of the human capacity for commonsense reasoning



Table of Contents


Part I: Foundations

Part II: Commonsense Phenomena 

Part III: Commonsense Domains

Part IV: Default Reasoning

Part V: Programs and Applications

Part VI: Logical and Non-logical Methods

Part VII: Knowledge Acquisition

Part VIII: Conclusion

Part IX: Appendices

Commonsense Reasoning: An Event Calculus Based Approach

A system and method for reasoning about concepts, relations and rules having a semantic network comprising at least one node from a predetermined set of node types, at least one link from a predetermined set of link types, and zero or more rules from a predetermined set of rule types, a subset of the rule types being matching rule types, each node and each link being associated with a set of zero or more rules; a network reasoning data structure having a reasoning type database having at least one regular expression, each of the regular expressions being a class of sequences having at least three node types and two link types, wherein the network reasoning data structure further has a context being a set of rules; and a reasoning engine having an activator for activating one or more activated paths in the semantic network, the set of activated paths having a common starting node in the semantic network, wherein the reasoning engine further has a validator for selecting a subset of the activated paths being valid paths, each rule from the set of rule matching types that is associated with one or more path elements on each valid path being matched by one or more rules in the context and wherein the reasoning engine further has a legal inferencer for selecting a subset of the set of valid paths being legal and valid paths, the legal and valid paths matching at least one of the regular expressions.

Method and knowledge structures for reasoning about concepts, relations, and rules

We present an implemented method for encoding reasoning problems of a discrete version of the classical logic event calculus in propositional conjunctive normal form, enabling the problems to be solved efficiently by off-the-shelf complete satisfiability (SAT) solvers. We build on the previous encoding method of Shanahan and Witkowski, extending it to support causal constraints, concurrent events, determining fluents, effect axioms with conditions, events triggered by conditions, gradual change, incompletely specified initial situations, state constraints, and release from the commonsense law of inertia. We present an alternative classical logic axiomatization of the event calculus and prove its equivalence to a standard axiomatization for integer time. We describe our encoding method based on the alternative axiomatization and prove its correctness. We evaluate the method on 14 benchmark reasoning problems for the event calculus and compare performance with the causal calculator on eight problems in the zoo world domain.

/pdf/event-calculus-reasoning-through-satisfiability-2sn825xkxy.pdf

Erik T. Mueller

Papers

Open Mind Common Sense: Knowledge Acquisition from the General Public

Watson: beyond jeopardy!

Commonsense Reasoning: An Event Calculus Based Approach

Method and knowledge structures for reasoning about concepts, relations, and rules

Event Calculus Reasoning Through Satisfiability