Data Mining Practical Machine Learning Tools and Techniques

User Centered Design

Although rapid advances in machine learning have made it increasingly applicable to expert decision-making, the delivery of accurate algorithmic predictions alone is insufficient for effective human-AI collaboration. In this work, we investigate the key types of information medical experts desire when they are first introduced to a diagnostic AI assistant. In a qualitative lab study, we interviewed 21 pathologists before, during, and after being presented deep neural network (DNN) predictions for prostate cancer diagnosis, to learn the types of information that they desired about the AI assistant. Our findings reveal that, far beyond understanding the local, case-specific reasoning behind any model decision, clinicians desired upfront information about basic, global properties of the model, such as its known strengths and limitations, its subjective point-of-view, and its overall design objective--what it's designed to be optimized for. Participants compared these information needs to the collaborative mental models they develop of their medical colleagues when seeking a second opinion: the medical perspectives and standards that those colleagues embody, and the compatibility of those perspectives with their own diagnostic patterns. These findings broaden and enrich discussions surrounding AI transparency for collaborative decision-making, providing a richer understanding of what experts find important in their introduction to AI assistants before integrating them into routine practice.

https://dl.acm.org/doi/pdf/10.1145/3359206

"Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making

Data-driven decision-making consequential to individuals raises important questions of accountability and justice. Indeed, European law provides individuals limited rights to 'meaningful information about the logic' behind significant, autonomous decisions such as loan approvals, insurance quotes, and CV filtering. We undertake three experimental studies examining people's perceptions of justice in algorithmic decision-making under different scenarios and explanation styles. Dimensions of justice previously observed in response to human decision-making appear similarly engaged in response to algorithmic decisions. Qualitative analysis identified several concerns and heuristics involved in justice perceptions including arbitrariness, generalisation, and (in)dignity. Quantitative analysis indicates that explanation styles primarily matter to justice perceptions only when subjects are exposed to multiple different styles --- under repeated exposure of one style, scenario effects obscure any explanation effects. Our results suggests there may be no 'best' approach to explaining algorithmic decisions, and that reflection on their automated nature both implicates and mitigates justice dimensions.

'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions

Designing interfaces or applications that move beyond the bounds of a single device screen enables new ways to engage with digital content. Research addressing the opportunities and challenges of interactions with multiple devices in concert is of continued focus in HCI research. To inform the future research agenda of this field, we contribute an analysis and taxonomy of a corpus of 510 papers in the cross-device computing domain. For both new and experienced researchers in the field we provide: an overview, historic trends and unified terminology of cross-device research; discussion of major and under-explored application areas; mapping of enabling technologies; synthesis of key interaction techniques spanning across multiple devices; and review of common evaluation strategies. We close with a discussion of open issues. Our taxonomy aims to create a unified terminology and common understanding for researchers in order to facilitate and stimulate future cross-device research.

/pdf/cross-device-taxonomy-survey-opportunities-and-challenges-of-2rubom7yex.pdf

Cross-Device Taxonomy: Survey, Opportunities and Challenges of Interactions Spanning Across Multiple Devices

While humans possess a well-developed sense of direction and can easily walk to a visible target, that ability is drastically reduced when lacking visual cues. In situations where people cannot depend on sight, orientation might become a critical issue, as when escaping a room filled with smoke, swimming in open waters, hiking in the fog or crossing the woods at night. In this paper, we present the design and implementation of VOS - a Visual Orientation System for providing an augmented sense of direction. Our system uses LEDs to offer cues on how to correct the current heading. Our findings consist of demonstrating the viability of such system, as well as its usability. We discuss the implications for designing technology that enables people to orient themselves and navigate places with little or no visual cues.

VOS -- Designing a Visual Orientation System

Assistive systems supporting users with cognitive impairments are commonplace in sheltered living facilities. Assistive systems may collect data or analyze user behavior to tailor services for the users’ type and level of impairment. However, ways to integrate ethical standards into the design and deployment of assistive systems for users with cognitive impairments are not yet established. We conducted a qualitative inquiry inspired by literature in ethics to address this. We interviewed caretakers and tenants with cognitive impairments in a sheltered living facility. We present four themes that describe the lived practice of ethics when using assistive systems: Autonomy & Independence, Confidence in Technology, Motivation, and Communal Living. Combining the themes with ethics theory, we derived five design implications for the ethical design of assistive systems. Our work proposes boundaries in which new assistive systems can be designed ethically and guide future assistive systems for marginalized populations.

Engagement, not Dependence: Ethically Designing Assistive Systems for Users with Cognitive Impairments

Personal tracking devices are equipped with more and more sensors and offer an ever-increasing level of accuracy. Yet, this comes at the cost of increased complexity. To deal with that problem, fitness trackers use derived metrics---scores calculated based on sensor data, e.g. a stress score. This means that part of the agency in interpreting health data is transferred from the user to the tracker. In this paper, we investigate the consequences of that transition and study how derived metrics can be designed to offer an optimal personal informatics experience. We conducted an online survey and a series of interviews which examined a health score (a hypothetical derived metric) at three levels of abstraction. We found that the medium abstraction level led to the highest level of reflection. Further, we determined that presenting the metric without contextual information led to decreased transparency and meaning. Our work contributes guidelines for designing effective derived metrics.

Designing Reflective Derived Metrics for Fitness Trackers

Humans are inherently skilled at using subtle physiological cues from other persons, for example gaze direction in a conversation. Personal computers have yet to explore this implicit input modality. In a study with 14 participants, we investigate how a user's gaze can be leveraged in adaptive computer systems. In particular, we examine the impact of different languages on eye movements by presenting simple questions in multiple languages to our participants. We found that fixation duration is sufficient to ascertain if a user is highly proficient in a given language. We propose how these findings could be used to implement adaptive visualizations that react implicitly on the user's gaze.

Towards Using Gaze Properties to Detect Language Proficiency

Abstract We present TomoTable—a research prototype of a position sensing device hidden inside an ordinary table. While the Human-Computer Interaction (HCI) field has extensively explored possibilities for spatially-aware multi-device interactions, the sensing methods that would enable such systems are still complex and hard to deploy in the wild. TomoTable presents a possibility of easily deployable invisible positional sensing that uses electrical capacitance tomography (ECT). Electrodes are embedded inside the table structure and provide accurate imaging of what is placed on the table. The entire system is invisible to the user. Objects can be identified based on their electrical properties. Our work creates opportunities for in-the-wild studies using multi-device systems. In this paper, we share the technical concept of TomoTable, preliminary insights on its use and perspectives for future studies.

Paweł W. Woźniak

Papers

VOS -- Designing a Visual Orientation System

Engagement, not Dependence: Ethically Designing Assistive Systems for Users with Cognitive Impairments

Designing Reflective Derived Metrics for Fitness Trackers

Towards Using Gaze Properties to Detect Language Proficiency

Sensing Mobile Device Orientation Through ECT Reconstructed Image Processing