Data Mining Practical Machine Learning Tools and Techniques

User Centered Design

Although rapid advances in machine learning have made it increasingly applicable to expert decision-making, the delivery of accurate algorithmic predictions alone is insufficient for effective human-AI collaboration. In this work, we investigate the key types of information medical experts desire when they are first introduced to a diagnostic AI assistant. In a qualitative lab study, we interviewed 21 pathologists before, during, and after being presented deep neural network (DNN) predictions for prostate cancer diagnosis, to learn the types of information that they desired about the AI assistant. Our findings reveal that, far beyond understanding the local, case-specific reasoning behind any model decision, clinicians desired upfront information about basic, global properties of the model, such as its known strengths and limitations, its subjective point-of-view, and its overall design objective--what it's designed to be optimized for. Participants compared these information needs to the collaborative mental models they develop of their medical colleagues when seeking a second opinion: the medical perspectives and standards that those colleagues embody, and the compatibility of those perspectives with their own diagnostic patterns. These findings broaden and enrich discussions surrounding AI transparency for collaborative decision-making, providing a richer understanding of what experts find important in their introduction to AI assistants before integrating them into routine practice.

https://dl.acm.org/doi/pdf/10.1145/3359206

"Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making

Data-driven decision-making consequential to individuals raises important questions of accountability and justice. Indeed, European law provides individuals limited rights to 'meaningful information about the logic' behind significant, autonomous decisions such as loan approvals, insurance quotes, and CV filtering. We undertake three experimental studies examining people's perceptions of justice in algorithmic decision-making under different scenarios and explanation styles. Dimensions of justice previously observed in response to human decision-making appear similarly engaged in response to algorithmic decisions. Qualitative analysis identified several concerns and heuristics involved in justice perceptions including arbitrariness, generalisation, and (in)dignity. Quantitative analysis indicates that explanation styles primarily matter to justice perceptions only when subjects are exposed to multiple different styles --- under repeated exposure of one style, scenario effects obscure any explanation effects. Our results suggests there may be no 'best' approach to explaining algorithmic decisions, and that reflection on their automated nature both implicates and mitigates justice dimensions.

'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions

Designing interfaces or applications that move beyond the bounds of a single device screen enables new ways to engage with digital content. Research addressing the opportunities and challenges of interactions with multiple devices in concert is of continued focus in HCI research. To inform the future research agenda of this field, we contribute an analysis and taxonomy of a corpus of 510 papers in the cross-device computing domain. For both new and experienced researchers in the field we provide: an overview, historic trends and unified terminology of cross-device research; discussion of major and under-explored application areas; mapping of enabling technologies; synthesis of key interaction techniques spanning across multiple devices; and review of common evaluation strategies. We close with a discussion of open issues. Our taxonomy aims to create a unified terminology and common understanding for researchers in order to facilitate and stimulate future cross-device research.

/pdf/cross-device-taxonomy-survey-opportunities-and-challenges-of-2rubom7yex.pdf

Cross-Device Taxonomy: Survey, Opportunities and Challenges of Interactions Spanning Across Multiple Devices

While the number of users sporting fitness trackers is constantly increasing, little is understood about how tracking goals can evolve over time. As recent studies have shown that the long-term health effects of trackers are limited, we need to readdress how trackers engage users. We conducted semi-structured interviews and an online survey to explore how users change their tracking goals. Based on our results, we created the Tracker Goal Evolution Model. The model describes how tracker goals can evolve from internal user needs through qualitative goals to quantitative goals that can be used with trackers. It also includes trust and reflection as key contextual factors contributing to meaningful transitions between goals. We postulate showing how tracker goals relate to other personal fitness goals as key for long-term engagement with trackers. Our model is useful for designers of future trackers as a tool to create evolving and meaningful tracking goals.

Supporting Meaningful Personal Fitness: the Tracker Goal Evolution Model

Smart glasses are increasingly being used in professional contexts. Having key applications such as short messaging and newsreader, they enable continuous access to textual information. In particular, smart glasses allow reading while performing other activities as they do not occlude the user's world view. For efficient reading, it is necessary to understand how a text should be presented on them. We, therefore, conducted a study with 24 participants using a Microsoft HoloLens to investigate how to display text on smart glasses while walking and sitting. We compared text presentation in the top-right, center, and bottom-center positions with Rapid Serial Visual Presentation (RSVP) and line-by-line scrolling. We found that text displayed in the top-right of smart glasses increases subjective workload and reduces comprehension. RSVP yields higher comprehension while sitting. Conversely, reading with scrolling yields higher comprehension while walking. Insights from our study inform the design of reading interfaces for smart glasses.

Reading on Smart Glasses: The Effect of Text Position, Presentation Type and Walking

This paper explores remote cheering during amateur races through a formative design inquiry. Friends and family of advanced amateur runners are part of their running experience. Runners rely on support during the race day and it is usually provided in the form of co-located cheers. RUFUS -- a prototype remote ambient runner support system -- was developed. The system enables supporters to send three types of signals to runners during a race and runners can send signals back to supporters. Input from supporters is sent through a webpage and received by runners through a device designed to lower distraction. An in situ study was conducted to evaluate the prototype during an organized race. Results show that two-way communication between runners and supporters was achieved. We also found that our system reflected varying user needs correctly. Runners and supporters reported increased motivation and enhanced race experience through feeling connected.

RUFUS: Remote Supporter Feedback for Long-Distance Runners

Collocated interaction has received growing interest in both academic research and the design of information and communication technological applications. An emergent research topic within this area relates to technological enhancement of social interaction. Various envisioned systems aim beyond simply enabling interaction, to actively enhance—i.e., improve the quality or extent of—social interaction between collocated people. However, there is little understanding of the optimal design solutions and roles of technology considering this goal. This literature review outlines the landscape of design explorations in this emergent research topic. We contribute an in-depth study of 92 publications that present relevant solutions or prototypes, analyzing their focus areas, design objectives, and design and evaluation approaches. To contribute with a new theoretical perspective, we identify various roles of technology relevant for enhancement, representing three abstract categories: facilitating, inviting and encouraging. This review helps researchers to describe, analyze, and position relevant prior research and identify gaps in scientific knowledge.

/pdf/technologies-for-enhancing-collocated-social-interaction-3h3juvsckm.pdf

Technologies for Enhancing Collocated Social Interaction: Review of Design Solutions and Approaches

A common objective for context-aware computing systems is to predict how user interfaces impact user performance regarding their cognitive capabilities. Existing approaches such as questionnaires or pupil dilation measurements either only allow for subjective assessments or are susceptible to environmental influences and user physiology. We address these challenges by exploiting the fact that cognitive workload influences smooth pursuit eye movements. We compared three trajectories and two speeds under different levels of cognitive workload within a user study (N=20). We found higher deviations of gaze points during smooth pursuit eye movements for specific trajectory types at higher cognitive workload levels. Using an SVM classifier, we predict cognitive workload through smooth pursuit with an accuracy of 99.5% for distinguishing between low and high workload as well as an accuracy of 88.1% for estimating workload between three levels of difficulty. We discuss implications and present use cases of how cognition-aware systems benefit from inferring cognitive workload in real-time by smooth pursuit eye movements.

Paweł W. Woźniak

Papers

Supporting Meaningful Personal Fitness: the Tracker Goal Evolution Model

Reading on Smart Glasses: The Effect of Text Position, Presentation Type and Walking

RUFUS: Remote Supporter Feedback for Long-Distance Runners

Technologies for Enhancing Collocated Social Interaction: Review of Design Solutions and Approaches

Your Eyes Tell: Leveraging Smooth Pursuit for Assessing Cognitive Workload