What is Twitter

The impact of polices on government social media usage: Issues, challenges, and recommendations

What does reliability mean for building a grounded theory? What about when writing an auto-ethnography? When is it appropriate to use measures like inter-rater reliability (IRR)? Reliability is a familiar concept in traditional scientific practice, but how, and even whether to establish reliability in qualitative research is an oft-debated question. For researchers in highly interdisciplinary fields like computer-supported cooperative work (CSCW) and human-computer interaction (HCI), the question is particularly complex as collaborators bring diverse epistemologies and training to their research. In this article, we use two approaches to understand reliability in qualitative research. We first investigate and describe local norms in the CSCW and HCI literature, then we combine examples from these findings with guidelines from methods literature to help researchers answer questions like: "should I calculate IRR?" Drawing on a meta-analysis of a representative sample of CSCW and HCI papers from 2016-2018, we find that authors use a variety of approaches to communicate reliability; notably, IRR is rare, occurring in around 1/9 of qualitative papers. We reflect on current practices and propose guidelines for reporting on reliability in qualitative research using IRR as a central example of a form of agreement. The guidelines are designed to generate discussion and orient new CSCW and HCI scholars and reviewers to reliability in qualitative research.

/pdf/reliability-and-inter-rater-reliability-in-qualitative-2aubgdtns7.pdf

Reliability and Inter-rater Reliability in Qualitative Research: Norms and Guidelines for CSCW and HCI Practice

We survey 146 papers analyzing "bias" in NLP systems, finding that their motivations are often vague, inconsistent, and lacking in normative reasoning, despite the fact that analyzing "bias" is an inherently normative process. We further find that these papers' proposed quantitative techniques for measuring or mitigating "bias" are poorly matched to their motivations and do not engage with the relevant literature outside of NLP. Based on these findings, we describe the beginnings of a path forward by proposing three recommendations that should guide work analyzing "bias" in NLP systems. These recommendations rest on a greater recognition of the relationships between language and social hierarchies, encouraging researchers and practitioners to articulate their conceptualizations of "bias"---i.e., what kinds of system behaviors are harmful, in what ways, to whom, and why, as well as the normative reasoning underlying these statements---and to center work around the lived experiences of members of communities affected by NLP systems, while interrogating and reimagining the power relations between technologists and such communities.

Language (Technology) is Power: A Critical Survey of "Bias" in NLP

Social media has been established in many larger emergencies and crises. This process has not started just a few years ago, but already 15 years ago in 2001 after the terrorist attacks of 9/11. In the following years, especially in the last 10, sometimes summarized under the term crisis informatics, a variety of studies focusing on the use of ICT and social media before, during or after nearly every crisis and emergency has arisen. This article aims to recapitulate 15 years of social media in emergencies and its research with a special emphasis on use patterns, role patterns and perception patterns that can be found across different cases in order to point out what has been achieved so far, and what future potentials exist.

https://onlinelibrary.wiley.com/doi/pdf/10.1111/1468-5973.12196

Fifteen years of social media in emergencies: A retrospective review and future directions for crisis Informatics

Research to date aimed at the fairness, accountability, and transparency of algorithmic systems has largely focused on topics such as identifying failures of current systems and on technical interventions intended to reduce bias in computational processes. Researchers have given less attention to methods that account for the social and political contexts of specific, situated technical systems at their points of use. Co-developing algorithmic accountability interventions in communities supports outcomes that are more likely to address problems in their situated context and re-center power with those most disparately affected by the harms of algorithmic systems. In this paper we report on our experiences using participatory and co-design methods for algorithmic accountability in a project called the Algorithmic Equity Toolkit. The main insights we gleaned from our experiences were: (i) many meaningful interventions toward equitable algorithmic systems are non-technical; (ii) community organizations derive the most value from localized materials as opposed to what is "scalable" beyond a particular policy context; (iii) framing harms around algorithmic bias suggests that more accurate data is the solution, at the risk of missing deeper questions about whether some technologies should be used at all. More broadly, we found that community-based methods are important inroads to addressing algorithmic harms in their situated contexts.

https://dl.acm.org/doi/pdf/10.1145/3351095.3372874

Toward situated interventions for algorithmic equity: lessons from the field

This research examines how information about an oil spill, its impacts, and the use of dispersants to treat the oil, moved through social media and the surrounding Internet during the 2010 BP Deepwater Horizon oil spill. Using a collection of tweets captured during the spill, we employ a mixed-method approach including an in-depth qualitative analysis to examine the content of Twitter posts, the connections that Twitter users made with each other, and the links between Twitter content and the surrounding Internet. This article offers a range of findings to help practitioners and others understand how social media is used by a variety of different actors during a slow-moving, long-term, environmental disaster. We enumerate some of the most salient themes in the Twitter data, noting that concerns about health impacts were more likely to be communicated in tweets about dispersant use, than in the larger conversation. We describe the accounts and behaviors of highly retweeted Twitter users, noting how...

Social Media, Public Participation, and the 2010 BP Deepwater Horizon Oil Spill

Widespread adoption of new information communication technologies (ICTs) is disrupting traditional models of news production and distribution. In this rapidly changing media landscape, the role of the journalist is evolving. Our research examines how professional journalists within a rural community impacted by Hurricane Irene successfully negotiated a new role for themselves, transforming their journalistic practice to serve in a new capacity as leaders of an online volunteer community. We describe an emergent organization of media professionals, citizen journalists, online volunteers, and collaborating journalistic institutions that provided real-time event coverage. In this rural context, where communications infrastructure is relatively uneven, this ad hoc effort bridged gaps in ICT infrastructure to unite its audience. In this paper, we introduce a new perspective for characterizing these information-sharing activities: the "human powered mesh network" extends the concept of a mesh network to include human actors in the movement of information. Our analysis shows how journalists played a key role in this network, and facilitated the movement of information to those who needed it. These findings also note a contrast between how HCI researchers are designing crowdsourcing platforms for news production and how crowdsourcing efforts are forming during disaster events, suggesting an alternative approach to designing for emergent collaborations in this context.

Journalists as Crowdsourcerers: Responding to Crisis by Reporting with a Crowd

The social function of the Internet has changed dramatically in recent years. What was, until recently, a supplement to other channels of information and communication has become increasingly a basic requirement of social and economic inclusion. Educational systems, employers, and government agencies at all levels have shifted services online—and are pushing rapidly to do more. Price remains only one factor shaping the fragile equilibrium of home broadband adoption, and library and community organizations fill the gap by providing critical training and support services while under severe economic pressures. Commissioned by the Federal Communications Commission (FCC) to analyze the factors shaping low rates of adoption of home broadband services in low-income and other marginalized communities, this SSRC study is one of the only large-scale qualitative investigations of barriers to adoption in the US and complements FCC survey research on adoption designed to inform the 2010 National Broadband Plan. The study draws on some 170 interviews of non-adopters, community access providers, and other intermediaries conducted across the US in late 2009 and early 2010 and identifies a range of factors that make broadband services hard to acquire and even harder to maintain in such communities.

Broadband adoption in low-income communities

As the Internet, and broadband in particular, becomes a platform for social and political engagement, researchers investigate more carefully both the factors that drive broadband adoption and the barriers that constrain it. This paper reports on one of the only large-scale qualitative studies of the barriers to broadband adoption in the United States, where 30% of the population lack broadband access. The primary research question asks: how can we qualitatively understand barriers to broadband adoption among low-income communities? The study’s community-based approach, undertaken in four regions of the country, reveals the complex equilibrium of broadband adoption. Drawing from 170 interviews with broadband non-adopters as well as community access providers and other intermediaries, this study finds that price is only one factor shaping home broadband adoption, and that libraries and other community organizations fill the gap between low home adoption and high demand for broadband. These intermediaries compensate for shortages in digital skills that also constitute barriers to adoption in a context where broadband is essential for gaining access to jobs, education, and e-government. These three main findings suggest that low-income people like our research participants are playing roles as actors in an ecology of broadband access games (Dutton et al. 2004). In particular, they are overcoming barriers to being online in order to participate in accessing services and gaining education. This is part of the process of defining broadband as an infrastructure for e-democracy. The paper recommends a renewed focus on factors that sustain home access rather than drive demand, as well as support for community intermediaries in provisioning public broadband access within a context of skill shortages. It recommends further qualitative research to

Dharma Dailey

Papers

Toward situated interventions for algorithmic equity: lessons from the field

Social Media, Public Participation, and the 2010 BP Deepwater Horizon Oil Spill

Journalists as Crowdsourcerers: Responding to Crisis by Reporting with a Crowd

Broadband adoption in low-income communities

The Essential Internet: Digital Exclusion in Low-Income American Communities