Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

计量经济分析 = Econometric analysis

An important part of our information-gathering behavior has always been to find out what other people think. With the growing availability and popularity of opinion-rich resources such as online review sites and personal blogs, new opportunities and challenges arise as people now can, and do, actively use information technologies to seek out and understand the opinions of others. The sudden eruption of activity in the area of opinion mining and sentiment analysis, which deals with the computational treatment of opinion, sentiment, and subjectivity in text, has thus occurred at least in part as a direct response to the surge of interest in new systems that deal directly with opinions as a first-class object.

This survey covers techniques and approaches that promise to directly enable opinion-oriented information-seeking systems. Our focus is on methods that seek to address the new challenges raised by sentiment-aware applications, as compared to those that are already present in more traditional fact-based analysis. We include material on summarization of evaluative text and on broader issues regarding privacy, manipulation, and economic impact that the development of opinion-oriented information-access services gives rise to. To facilitate future work, a discussion of available resources, benchmark datasets, and evaluation campaigns is also provided.

/pdf/opinion-mining-and-sentiment-analysis-3v5ya3qqiw.pdf

Opinion Mining and Sentiment Analysis

Sentiment analysis and opinion mining is the field of study that analyzes people's opinions, sentiments, evaluations, attitudes, and emotions from written language. It is one of the most active research areas in natural language processing and is also widely studied in data mining, Web mining, and text mining. In fact, this research has spread outside of computer science to the management sciences and social sciences due to its importance to business and society as a whole. The growing importance of sentiment analysis coincides with the growth of social media such as reviews, forum discussions, blogs, micro-blogs, Twitter, and social networks. For the first time in human history, we now have a huge volume of opinionated data recorded in digital form for analysis. Sentiment analysis systems are being applied in almost every business and social domain because opinions are central to almost all human activities and are key influencers of our behaviors. Our beliefs and perceptions of reality, and the choices we make, are largely conditioned on how others see and evaluate the world. For this reason, when we need to make a decision we often seek out the opinions of others. This is true not only for individuals but also for organizations. This book is a comprehensive introductory and survey text. It covers all important topics and the latest developments in the field with over 400 references. It is suitable for students, researchers and practitioners who are interested in social media analysis in general and sentiment analysis in particular. Lecturers can readily use it in class for courses on natural language processing, social media analysis, text mining, and data mining. Lecture slides are also available online.

/pdf/sentiment-analysis-and-opinion-mining-8314pykwrf.pdf

Sentiment Analysis and Opinion Mining

Comunicacio presentada a la 2016 Conference of the North American Chapter of the Association for Computational Linguistics, celebrada a San Diego (CA, EUA) els dies 12 a 17 de juny 2016.

/pdf/neural-architectures-for-named-entity-recognition-4xswt97nv2.pdf

Neural Architectures for Named Entity Recognition

The automation of design decisions can be seen as a problem of generating mappings between elements in an abstract specification of the object to be designed and the concrete parts of the object itself (Puerta and Eisenstein 1999). In some cases, it is difficult to discover a formalism that takes all relevant variables into account; human designers proceed by Ointuition.O Individual designers may have stylistic preferences that are purely idiosyncratic or are common only to one particular Oschool.O By ignoring such preferences, automatic design forfeits the flexibility, creativity, and vitality of human design. In short, automatic design algorithms suffer from a lack of flexibility. Adaptation is offered as a solution to this problem. By making automatic design algorithms adaptive, we can begin to do automation without a complete knowledge baseNit can be developed and refined along the way. Stylistic preferences can likewise be accommodated if an automatic design algorithm can adapt to the user. We add adaptation to an existing piece of automatic design software: the TIMM module of the MOBI-D user- interface design environment (Puerta 1997). MOBI-D maintains explicit, formal representations of the abstract and concrete sides of the interface. TIMM automates the mappings between the abstract domain objects and concrete presentation elements, using a decision tree. Although there exists a body of work on using decision trees to automate selection of interactors (Vanderdonckt and Bodart 1996), most theory-refinement algorithms involve neural nets (Maclin and Shavlik 1996). A theory- refinement algorithm for decision trees is presented below. An error occurs when the user disagrees with one of TIMMOs design decisions. The system then searches for a set of operations that can be performed on the decision tree so as to yield the least number of errors, when applied over the entire history of user interactions. When multiple solutions produce the same number of errors, operations that are reversible and do not affect the structure of the decision tree are preferred. The most preferred operation is to change the output of the leaf that produced the error. Suppose that the decision tree dictates that boolean domain elements are best treated by a checkbox, but the user selects radio buttons instead. This stylistic preference can be satisfied by changing the output of the leaf for boolean domain variables to radio buttons. An alteration of the boundary conditions is the next most preferred operation, because it can be reversed by altering the boundary conditions again. Suppose that the decision- tree selects radio buttons for variables with less than four allowed values, and list boxes when there are four or more allowed values. If the user rejects the selection of list boxes in cases where there are four allowed values, then a shift in the boundary condition will produce more agreeable output. There is no facility for removing branches. Thus, the addition of a new branch is the least preferred operation, as it is irreversible. It is necessary to add a branch when there is a relevant piece of information that the decision tree did not consider. The designerOs selection of certain interactors might be influenced by the size of their parent dialog window. Adding a new discriminant to take that information into account is the only way to properly correct the decision tree. Preliminary testing has shown that all of these operations are useful. Local minima have not appeared, so the fact that our search is a greedy hill-climb does not pose a problem. Earlier, three advantages of adaptation were cited: acquisition of new design knowledge, accommodation of user preferences, and update of the automation algorithm in response to changing technology. Testing has shown that the theory-refinement algorithm described here can deliver all three of those advantages.

/pdf/learning-design-guidelines-by-theory-refinement-4dm17be6ir.pdf

Learning design guidelines by theory refinement

We demonstrate the effectiveness of multilingual learning for unsupervised part-of-speech tagging. The central assumption of our work is that by combining cues from multiple languages, the structure of each becomes more apparent. We consider two ways of applying this intuition to the problem of unsupervised part-of-speech tagging: a model that directly merges tag structures for a pair of languages into a single sequence and a second model which instead incorporates multilingual context using latent variables. Both approaches are formulated as hierarchical Bayesian models, using Markov Chain Monte Carlo sampling techniques for inference. Our results demonstrate that by incorporating multilingual evidence we can achieve impressive performance gains across a range of scenarios. We also found that performance improves steadily as the number of available languages increases.

/pdf/multilingual-part-of-speech-tagging-two-unsupervised-50be7yaev7.pdf

Multilingual Part-of-Speech Tagging: Two Unsupervised Approaches

Speakers of non-English languages often adopt loanwords from English to express new or unusual concepts. While these loanwords may be borrowed unchanged, speakers may also integrate the words to fit the constraints of their native language, e.g. creating Spanish "tuitear" from English "tweet." Linguists have often considered the process of loanword integration to be more dependent on language-internal constraints, but sociolinguistic constraints such as speaker background remain only qualitatively understood. We investigate the role of social context and speaker background in Spanish speakers' use of integrated loanwords on social media. We find first that newspaper authors use the integrated forms of loanwords and native words more often than social media authors, showing that integration is associated with formal domains. In social media, we find that speaker background and expectations of formality explain loanword and native word integration, such that authors who use more Spanish and who write to a wider audience tend to use integrated verb forms more often. This study shows that loanword integration reflects not only language-internal constraints but also social expectations that vary by conversation and speaker.

Tuiteamos o pongamos un tuit? Investigating the Social Constraints of Loanword Integration in Spanish Social Media

The incorporation of plastic and adaptive user-interfaces into the model-based paradigm requires a new, more flexible modeling formalism. Rather than modeling the user-interface as a set of static structures and mappings, the UI should be modeled as a set of design preferences. Preferences are frequently many-toone or many-to-many relationships that elude conventional UI modeling, which has largely focused on oneto-one mappings. In this paper, a spectrum of preference relations is described, and a new syntax for modeling preference is proposed. This spectrum extends from simple one-to-one bindings to complex design guidelines that can be structured together to implement decision trees. This new representation allows decision trees to be tightly integrated into the user-interface model itself, enhancing their flexibility and power.

Modeling preference for adaptive user-interfaces.

Social media datasets make it possible to rapidly quantify collective attention to emerging topics and breaking news, such as crisis events. Collective attention is typically measured by aggregate counts, such as the number of posts that mention a name or hashtag. But according to rationalist models of natural language communication, the collective salience of each entity will be expressed not only in how often it is mentioned, but in the form that those mentions take. This is because natural language communication is premised on (and customized to) the expectations that speakers and writers have about how their messages will be interpreted by the intended audience. We test this idea by conducting a large-scale analysis of public online discussions of breaking news events on Facebook and Twitter, focusing on five recent crisis events. We examine how people refer to locations, focusing specifically on contextual descriptors, such as "San Juan" versus "San Juan, Puerto Rico." Rationalist accounts of natural language communication predict that such descriptors will be unnecessary (and therefore omitted) when the named entity is expected to have high prior salience to the reader. We find that the use of contextual descriptors is indeed associated with proxies for social and informational expectations, including macro-level factors like the location's global salience and micro-level factors like audience engagement. We also find a consistent decrease in descriptor context use over the lifespan of each crisis event. These findings provide evidence about how social media users communicate with their audiences, and point towards more fine-grained models of collective attention that may help researchers and crisis response organizations to better understand public perception of unfolding crisis events.

Jacob Eisenstein

Papers

Learning design guidelines by theory refinement

Multilingual Part-of-Speech Tagging: Two Unsupervised Approaches

Tuiteamos o pongamos un tuit? Investigating the Social Constraints of Loanword Integration in Spanish Social Media

Modeling preference for adaptive user-interfaces.

Characterizing Collective Attention via Descriptor Context: A Case Study of Public Discussions of Crisis Events