The ImageNet Large Scale Visual Recognition Challenge is a benchmark in object category classification and detection on hundreds of object categories and millions of images. The challenge has been run annually from 2010 to present, attracting participation from more than fifty institutions. This paper describes the creation of this benchmark dataset and the advances in object recognition that have been possible as a result. We discuss the challenges of collecting large-scale ground truth annotation, highlight key breakthroughs in categorical object recognition, provide a detailed analysis of the current state of the field of large-scale image classification and object detection, and compare the state-of-the-art computer vision accuracy with human accuracy. We conclude with lessons learned in the 5 years of the challenge, and propose future directions and improvements.

/pdf/imagenet-large-scale-visual-recognition-challenge-caky5nmmix.pdf

ImageNet Large Scale Visual Recognition Challenge

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.

Machine Learning : A Probabilistic Perspective

We introduce a system for sensing complex social systems with data collected from 100 mobile phones over the course of 9 months. We demonstrate the ability to use standard Bluetooth-enabled mobile telephones to measure information access and use in different contexts, recognize social patterns in daily user activity, infer relationships, identify socially significant locations, and model organizational rhythms.

/pdf/reality-mining-sensing-complex-social-systems-2gf3rdnw3x.pdf

Reality mining: sensing complex social systems

Recent research has shown the benefit of framing problems of imitation learning as solutions to Markov Decision Problems. This approach reduces learning to the problem of recovering a utility function that makes the behavior induced by a near-optimal policy closely mimic demonstrated behavior. In this work, we develop a probabilistic approach based on the principle of maximum entropy. Our approach provides a well-defined, globally normalized distribution over decision sequences, while providing the same performance guarantees as existing methods.

We develop our technique in the context of modeling real-world navigation and driving behaviors where collected data is inherently noisy and imperfect. Our probabilistic approach enables modeling of route preferences as well as a powerful new approach to inferring destinations and routes based on partial trajectories.

/pdf/maximum-entropy-inverse-reinforcement-learning-51o9kfg5yc.pdf

Maximum entropy inverse reinforcement learning

We describe the design of the NomaticBubbles, a visualization that provides cues of communal whereabouts. Unlike most location displays showing whereabouts on a geographical map, the NomaticBubbles depicts historical and aggregate traces of participants' whereabouts in an abstract and ambiguous manner. We describe the design of the NomaticBubbles, and discuss some early experiences and feedback we got, as well as future work.

NomaticBubbles: visualizing communal whereabouts

The increased availability of inexpensive sensors, tremendous processing capabilities (even in mobile devices), high-bandwidth wireless networks, and vast quantities of data storage have made it much more practical to continuously collect streams of low-level data about people and their environments. This approach enables researchers to compile detailed records of various contextual factors surrounding people’s interactions with their world. Their locations, physiological states, contact with other people, situated uses of devices, and other digital traces can potentially be recorded and analyzed. Each of these kinds of data can be collected at virtually any frequency, with or without participant knowledge or intervention, and for extended periods of time. Due to the automated nature of the method, a large number of samples can be gathered quickly and with relatively low overhead by the researchers during sessions in the fi eld. However, the degree of automation involved in this method requires a number of pragmatic and analytic considerations, beginning with careful experimental design to ensure appropriate sensor design, and including how the study is deployed, participant training, privacy safeguards, and data storage requirements. Sensor Data Streams

A Short Description of the Method

As AI systems proliferate, their greenhouse gas emissions are an increasingly important concern for human societies. We analyze the emissions of several AI systems (ChatGPT, BLOOM, DALL-E2, Midjourney) relative to those of humans completing the same tasks. We find that an AI writing a page of text emits 130 to 1500 times less CO2e than a human doing so. Similarly, an AI creating an image emits 310 to 2900 times less. Emissions analysis do not account for social impacts such as professional displacement, legality, and rebound effects. In addition, AI is not a substitute for all human tasks. Nevertheless, at present, the use of AI holds the potential to carry out several major activities at much lower emission levels than can humans.

The Carbon Emissions of Writing and Illustrating Are Lower for AI than for Humans

The Proof-of-Work algorithm that underlies Bitcoin and many other cryptocurrencies is well known for its energy-intensive requirements. The Proof-of-Stake algorithm that underlies Ethereum2 and various other cryptocurrencies is less impactful environmentally, but it has a second, looming issue: the problem of wealth inequality. We have developed an alternative to Proof-of-Work and Proof-of-Stake, called Proof-by-Location, that has the potential to address both of these issues. This paper describes Proof-by-Location and a financial platform called Xylem that is based on it. This platform seeks to distribute transaction fees to billions of cryptocurrency "Notaries" around the world (essentially, anyone with a smartphone), who work together to establish a distributed consensus about financial transactions. Using Xylem as a global financial infrastructure could lead to significantly better social and environmental outcomes than existing financial platforms.

Proof-by-Location as a Socially Responsible Financial Infrastructure

Intelligent assistants that support human communication need to respect the difficulty of understanding the context surrounding the interchange. Rather than attempting to directly communicate for a user, intelligent assistants should support decision making on the part of the involved parties so that complex social negotiations are preserved. We describe an intelligent assistant that does this for instant messaging called Nomatic*Gaim.

https://www.aaai.org/Papers/Symposia/Spring/2007/SS-07-04/SS07-04-018.pdf

Donald J. Patterson

Papers

NomaticBubbles: visualizing communal whereabouts

A Short Description of the Method

The Carbon Emissions of Writing and Illustrating Are Lower for AI than for Humans

Proof-by-Location as a Socially Responsible Financial Infrastructure

Involving Intelligent Assistants in Active Human Communication.