Open AccessProceedings Article
What to do about bad language on the internet
Jacob Eisenstein
- pp 359-369
Reads0
Chats0
TLDR
A critical review of the NLP community's response to the landscape of bad language is offered, and a quantitative analysis of the lexical diversity of social media text, and its relationship to other corpora is presented.Abstract:
The rise of social media has brought computational linguistics in ever-closer contact with bad language: text that defies our expectations about vocabulary, spelling, and syntax. This paper surveys the landscape of bad language, and offers a critical review of the NLP community’s response, which has largely followed two paths: normalization and domain adaptation. Each approach is evaluated in the context of theoretical and empirical work on computer-mediated communication. In addition, the paper presents a quantitative analysis of the lexical diversity of social media text, and its relationship to other corpora.read more
Citations
More filters
Proceedings Article
Predicting Author Age from Weibo Microblog Posts.
TL;DR: An author profiling study based on Chinese social media texts gleaned from Sina Weibo in which the author’s age group is predicted based on various linguistic text features mainly relating to non-standard orthography: classical Chinese characters, hashtags, emoticons and kaomoji, homogeneous punctuation and Latin character sequences, and poetic format.
unimelb: Spanish Text Normalisation.
Bo Han,Paul Cook,Timothy Baldwin +2 more
TL;DR: Experimental results indicate that the corpus-derived lexicon complements existing lexicons, but that the approach could be improved through better handling of certain word types, such as named entities.
Proceedings ArticleDOI
Parsing transcripts of speech
TL;DR: It is found that parser performance tends to deteriorate with increasing length of string, more so for spoken than for written texts, and an alternative parsing model with added speech data is trained and improvements in accuracy on speech-units are demonstrated.
Posted Content
MTNT: A Testbed for Machine Translation of Noisy Text
Paul Michel,Graham Neubig +1 more
TL;DR: The authors proposed a benchmark dataset for machine translation of noisy text (MTNT), consisting of noisy comments on Reddit and professionally sourced translations, and qualitatively and quantitatively examine the types of noise included in this dataset, then demonstrate that existing MT models fail badly on a number of noise-related phenomena, even after performing adaptation on a small training set of in-domain data.
Proceedings ArticleDOI
#Anorexia, #anarexia, #anarexyia: Characterizing online community practices with orthographic variation
TL;DR: The utility of orthographic variation as a new lens to study sociolinguistic change in online communities, particularly when the change results from an exogenous force such as a content ban is demonstrated.
References
More filters
Proceedings ArticleDOI
Earthquake shakes Twitter users: real-time event detection by social sensors
TL;DR: This paper investigates the real-time interaction of events such as earthquakes in Twitter and proposes an algorithm to monitor tweets and to detect a target event and produces a probabilistic spatiotemporal model for the target event that can find the center and the trajectory of the event location.
Journal ArticleDOI
Critical questions for big data
danah boyd,Kate Crawford +1 more
TL;DR: The era of Big Data has begun as discussed by the authors, where diverse groups argue about the potential benefits and costs of analyzing genetic sequences, social media interactions, health records, phone logs, government records, and other digital traces left by people.
Proceedings ArticleDOI
Feature-rich part-of-speech tagging with a cyclic dependency network
TL;DR: A new part-of-speech tagger is presented that demonstrates the following ideas: explicit use of both preceding and following tag contexts via a dependency network representation, broad use of lexical features, and effective use of priors in conditional loglinear models.
Book
Natural Language Processing with Python
TL;DR: This book offers a highly accessible introduction to natural language processing, the field that supports a variety of language technologies, from predictive text and email filtering to automatic summarization and translation.
Proceedings ArticleDOI
Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling
TL;DR: By using simulated annealing in place of Viterbi decoding in sequence models such as HMMs, CMMs, and CRFs, it is possible to incorporate non-local structure while preserving tractable inference.