Michael S. Bernstein

Bio: Michael S. Bernstein is an academic researcher from Stanford University. The author has contributed to research in topics: Crowdsourcing & Computer science. The author has an hindex of 52, co-authored 191 publications receiving 42744 citations. Previous affiliations of Michael S. Bernstein include Association for Computing Machinery & Massachusetts Institute of Technology.

Breaking Out of the Ivory Tower: A Large-scale Analysis of Patent Citations to HCI Research

Abstract: What is the impact of human-computer interaction research on industry? While it is impossible to track all research impact pathways, the growing literature on translational research impact measurement offers patent citations as one measure of how industry recognizes and draws on research in its inventions. In this paper, we perform a large-scale measurement study primarily of 70, 000 patent citations to premier HCI research venues, tracing how HCI research are cited in United States patents over the last 30 years. We observe that 20.1% of papers from these venues, including 60–80% of papers at UIST and 13% of papers in a broader dataset of SIGCHI-sponsored venues overall, are cited by patents—far greater than premier venues in science overall (9.7%) and NLP (11%). However, the time lag between a patent and its paper citations is long (10.5 years) and getting longer, suggesting that HCI research and practice may not be efficiently connected.

Text Mining Emergent Human Behaviors for Interactive Systems

Abstract: People engage with thousands of situations, activities, and objects on a daily basis. Hand-coding this knowledge into interactive systems is prohibitively labor-intensive, but fiction captures a vast number of human lives in moment to moment detail. In this paper, we bootstrap a knowledge graph of human activities by text mining a large dataset of modern fiction on the web. Our knowledge graph, Augur, describes human actions over time as conditioned by nearby locations, people, and objects. Applications can use this graph to react to human behavior in a data-driven way. We demonstrate an Augur-enhanced video game world in which non-player characters follow realistic patterns of behavior, interact with their environment and each other, and respond to the user's behavior.

Shared Autonomy for an Interactive AI System

Abstract: Across many domains, interactive systems either make decisions for us autonomously or yield decision-making authority to us and play a supporting role. However, many settings, such as those in education or the workplace, benefit from sharing this autonomy between the user and the system, and thus from a system that adapts to them over time. In this paper, we pursue two primary research questions: (1) How do we design interfaces to share autonomy between the user and the system? (2) How does shared autonomy alter a user"s perception of a system? We present SharedKeys, an interactive shared autonomy system for piano instruction that plays different video segments of a piece for students to emulate and practice. Underlying our approach to shared autonomy is a mixed-observability Markov decision process that estimates a user"s desired autonomy level based on her performance and attentiveness. Pilot studies revealed that students sharing autonomy with the system learned more quickly and perceived the system as more intelligent.

Understanding Undergraduate Course Consideration

Abstract: In elective curriculums, undergraduates are encouraged to consider a range of academic courses for possible enrollment each term, yet course consideration has not been explicitly theorized and is difficult to observe. We develop a conceptual framework and a novel web-based mechanism for observing consideration, and use them to study consideration among an entire cohort of students at a private university between 2016-2018. Our findings reveal (1) substantial winnowing from available to considered courses; (2) homogeneous consideration set sizes regardless of students’ subsequent majors; and (3) heterogeneous consideration set compositions correlated with subsequent majors. Our work demonstrates that course consideration is an empirically demonstrable component of course selection and suggests mechanisms for intervening in consideration to support informed choice and efficient academic progress.

Information scraps : understanding and design

Abstract: In this thesis I investigate information scraps – personal information whose content has been scribbled on Post-it notes, scrawled on the corners of sheets of paper, stuck in our pockets, sent in e-mail messages to ourselves, and stashed into miscellaneous digital text files. Information scraps encode information ranging from ideas and sketches to notes, reminders, shipment tracking numbers, driving directions, and even poetry. I proceed by performing an in-depth ethnographic investigation of the nature and use of information scraps, and by designing and building two research systems designed for information scrap management. The first system, Jourknow, lowers the capture barrier for unstructured notes and structured information such as calendar items and to-dos, captures contextual information surrounding note creation such as location, documents viewed, and people corresponded with, and manages uncommon user-generated personal information such as restaurant reviews or this week’s shopping list. The follow-up system, Pinky, further explores the lightweight capture space by providing a command line interface that is tolerant to re-ordering and GUI affordances for quick and accurate entry. Reflecting on these tools’ successes and failures, I characterize the design process challenges inherent in designing and building information scrap tools. Thesis Supervisor: David R. Karger Title: Professor of Electrical Engineering and Computer Science

Deep Residual Learning for Image Recognition

Abstract: Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

Very Deep Convolutional Networks for Large-Scale Image Recognition

Abstract: In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

Very Deep Convolutional Networks for Large-Scale Image Recognition

Abstract: In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

Deep Residual Learning for Image Recognition

Abstract: Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers---8x deeper than VGG nets but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

Deep Learning

Abstract: Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.