Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

/pdf/deep-residual-learning-for-image-recognition-b75jg61qrq.pdf

Deep Residual Learning for Image Recognition

In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

Very Deep Convolutional Networks for Large-Scale Image Recognition

Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers---8x deeper than VGG nets but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. 
The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

We present hybrid crowd-machine learning classifiers: classification models that start with a written description of a learning goal, use the crowd to suggest predictive features and label data, and then weigh these features using machine learning to produce models that are accurate and use human-understandable features. These hybrid classifiers enable fast prototyping of machine learning models that can improve on both algorithm performance and human judgment, and accomplish tasks where automated feature extraction is not yet feasible. Flock, an interactive machine learning platform, instantiates this approach. To generate informative features, Flock asks the crowd to compare paired examples, an approach inspired by analogical encoding. The crowd's efforts can be focused on specific subsets of the input space where machine-extracted features are not predictive, or instead used to partition the input space and improve algorithm performance in subregions of the space. An evaluation on six prediction tasks, ranging from detecting deception to differentiating impressionist artists, demonstrated that aggregating crowd features improves upon both asking the crowd for a direct prediction and off-the-shelf machine learning features by over 10%. Further, hybrid systems that use both crowd-nominated and machine-extracted features can outperform those that use either in isolation.

/pdf/flock-hybrid-crowd-machine-learning-classifiers-2rfqlu19za.pdf

Flock: Hybrid Crowd-Machine Learning Classifiers

This paper introduces architectural and interaction patterns for integrating crowdsourced human contributions directly into user interfaces. We focus on writing and editing, complex endeavors that span many levels of conceptual and pragmatic activity. Authoring tools offer help with pragmatics, but for higher-level help, writers commonly turn to other people. We thus present Soylent, a word processing interface that enables writers to call on Mechanical Turk workers to shorten, proofread, and otherwise edit parts of their documents on demand. To improve worker quality, we introduce the Find-Fix-Verify crowd programming pattern, which splits tasks into a series of generation and review stages. Evaluation studies demonstrate the feasibility of crowdsourced editing and investigate questions of reliability, cost, wait time, and work time for edits.

Soylent: a word processor with a crowd inside

Intelligence does not arise only in individual brains; it also arises in groups of individuals. This is collective intelligence: groups of individuals acting collectively in ways that seem intelligent. In recent years, a new kind of collective intelligence has emerged: interconnected groups of people and computers, collectively doing intelligent things. Today these groups are engaged in tasks that range from writing software to predicting the results of presidential elections. This volume reports on the latest research in the study of collective intelligence, laying out a shared set of research challenges from a variety of disciplinary and methodological perspectives. Taken together, these essays -- by leading researchers from such fields as computer science, biology, economics, and psychology -- lay the foundation for a new multidisciplinary field. Each essay describes the work on collective intelligence in a particular discipline -- for example, economics and the study of markets; biology and research on emergent behavior in ant colonies; human-computer interaction and artificial intelligence; and cognitive psychology and the "wisdom of crowds" effect. Other areas in social science covered include social psychology, organizational theory, law, and communications. Contributors Eytan Adar, Ishani Aggarwal, Yochai Benkler, Michael S. Bernstein, Jeffrey P. Bigham, Jonathan Bragg, Deborah M. Gordon, Benjamin Mako Hill, Christopher H. Lin, Andrew W. Lo, Thomas W. Malone, Mausam, Brent Miller, Aaron Shaw, Mark Steyvers, Daniel S. Weld, Anita Williams Woolley

Handbook of Collective Intelligence

A large, seemingly overwhelming task can sometimes be transformed into a set of smaller, more manageable microtasks that can each be accomplished independently. For example, it may be hard to subjectively rank a large set of photographs, but easy to sort them in spare moments by making many pairwise comparisons. In crowdsourcing systems, microtasking enables unskilled workers with limited commitment to work together to complete tasks they would not be able to do individually. We explore the costs and benefits of decomposing macrotasks into microtasks for three task categories: arithmetic, sorting, and transcription. We find that breaking these tasks into microtasks results in longer overall task completion times, but higher quality outcomes and a better experience that may be more resilient to interruptions. These results suggest that microtasks can help people complete high quality work in interruption-driven environments.

/pdf/break-it-down-a-comparison-of-macro-and-microtasks-21fgoc08su.pdf

Break It Down: A Comparison of Macro- and Microtasks

Visualization design can benefit from careful consideration of perception, as different assignments of visual encoding variables such as color, shape and size affect how viewers interpret data. In this work, we introduce perceptual kernels: distance ma- trices derived from aggregate perceptual judgments. Perceptual kernels represent perceptual differences between and within visual variables in a reusable form that is directly applicable to visualization evaluation and automated design. We report results from crowd- sourced experiments to estimate kernels for color, shape, size and combinations thereof. We analyze kernels estimated using five different judgment types—including Likert ratings among pairs, ordinal triplet comparisons, and manual spatial arrangement—and compare them to existing perceptual models. We derive recommendations for collecting perceptual similarities, and then demonstrate how the resulting kernels can be applied to automate visualization design decisions. Visual encoding decisions are central to visualization design. As view- ers' interpretation of data may shift across encodings, it is important to understand how choices of visual encoding variables such as color, shape, size—and their combinations—affect graphical perception. One way to evaluate these effects is to measure the perceived simi- larities(orconversely, distances) between visual variables. Webroadly refer to subjective measures of judged similarity as perceptual dis- tances. In this context, a perceptual kernel is the distance matrix of aggregated pairwise perceptual distances. These measures quantify the effects of alternative encodings and thereby help create visualiza- tions that better reflect structures in data. Figure 1a shows a perceptual kernel for a set of symbols; distances are visualized using grayscale values, with darker cells indicating higher similarity. The prominent clusters suggest that users will perceive similarities among shapes that may or may not mirror encoded data values. Perceptual kernels can also benefit automated visualization design. Typically, automated design methods (27) leverage an effectiveness ranking of visual encoding variables with respect to data types (nom- inal, ordinal, quantitative). Once a visual variable is chosen, these methods provide little guidance on how to best pair data values with visual elements, instead relying on default palettes for variables such as color and shape. Perceptual kernels provide a means for computing optimized assignments to visual variables whose perceived differences are congruent with underlying distances among data points. In short, perceptual kernels enable the direct application of empirical percep- tion data within visualization tools. In this work, we contribute the results of crowdsourced experiments to estimate perceptual kernels for visual encoding variables of shape, size, color and combinations thereof. There are alternative ways of eliciting judged similarities among visual variables. We compare a variety of judgment types: Likert ratings among pairs, ordinal triplet comparisons, and manual spatial arrangement. We also assess the re- sulting kernels via comparisons to existing perceptual models. Wefind that ordinal triplet matching judgments provide the most consistent re- sults, albeit with higher time and money costs than pairwise ratings or spatial arrangement. We then demonstrate how perceptual kernels can be applied to improve visualization design through automatic palette optimization and by providing distances for visual embedding (8) of data points into visual spaces.

http://idl.cs.washington.edu/files/2014-PerceptualKernels-InfoVis.pdf

Michael S. Bernstein

Papers

Flock: Hybrid Crowd-Machine Learning Classifiers

Soylent: a word processor with a crowd inside

Handbook of Collective Intelligence

Break It Down: A Comparison of Macro- and Microtasks

Learning Perceptual Kernels for Visualization Design.