The Ecological Approach to Visual Perception

In some arrangements, product packaging is digitally watermarked over most of its extent to facilitate high-throughput item identification at retail checkouts. Imagery captured by conventional or plenoptic cameras can be processed (e.g., by GPUs) to derive several different perspective-transformed views—further minimizing the need to manually reposition items for identification. Crinkles and other deformations in product packaging can be optically sensed, allowing such surfaces to be virtually flattened to aid identification. Piles of items can be 3D-modelled and virtually segmented into geometric primitives to aid identification, and to discover locations of obscured items. Other data (e.g., including data from sensors in aisles, shelves and carts, and gaze tracking for clues about visual saliency) can be used in assessing identification hypotheses about an item. A great variety of other features and arrangements are also detailed.

Methods and arrangements for identifying objects

While smartphones and related mobile technologies are recognized as flexible and powerful tools that, when used prudently, can augment human cognition, there is also a growing perception that habitual involvement with these devices may have a negative and lasting impact on users’ ability to think, remember, pay attention, and regulate emotion. The present review considers an intensifying, though still limited, area of research exploring the potential cognitive impacts of smartphone-related habits, and seeks to determine in which domains of functioning there is accruing evidence of a significant relationship between smartphone technology and cognitive performance, and in which domains the scientific literature is not yet mature enough to endorse any firm conclusions. We focus our review primarily on three facets of cognition that are clearly implicated in public discourse regarding the impacts of mobile technology – attention, memory, and delay of gratification – and then consider evidence regarding the broader relationships between smartphone habits and everyday cognitive functioning. Along the way, we highlight compelling findings, discuss limitations with respect to empirical methodology and interpretation, and offer suggestions for how the field might progress toward a more coherent and robust area of scientific inquiry.​

/pdf/smartphones-and-cognition-a-review-of-research-exploring-the-1dcnqqvpmq.pdf

Smartphones and Cognition: A Review of Research Exploring the Links between Mobile Technology Habits and Cognitive Functioning.

Flattening the Earth, Two Thousand Years of Map Projections

Notifications are a core feature of mobile phones. They inform users about a variety of events. Users may take immediate action or ignore them depending on the importance of a notification as well as their current context. The nature of notifications is manifold, applications use them both sparsely and frequently. In this paper we present the first large-scale analysis of mobile notifications with a focus on users' subjective perceptions. We derive a holistic picture of notifications on mobile phones by collecting close to 200 million notifications from more than 40,000 users. Using a data-driven approach, we break down what users like and dislike about notifications. Our results reveal differences in importance of notifications and how users value notifications from messaging apps as well as notifications that include information about people and events. Based on these results we derive a number of findings about the nature of notifications and guidelines to effectively use them.

/pdf/large-scale-assessment-of-mobile-notifications-3ayexh71go.pdf

Large-scale assessment of mobile notifications

Smartphone users might be interrupted while interacting with an application, either by intended or unintended circumstances. In this paper, we report on a large-scale observational study that investigated mobile application interruptions in two scenarios: (1) intended back and forth switching between applications and (2) unintended interruptions caused by incoming phone calls. Our findings reveal that these interruptions rarely happen (at most 10% of the daily application usage), but when they do, they may introduce a significant overhead (can delay completion of a task by up to 4 times). We conclude with a discussion of the results, their limitations, and a series of implications for the design of mobile phones.

https://interruptions.net/literature/Leiva-MobileHCI12.pdf

Back to the app: the costs of mobile application interruptions

The increasing number of media facades in urban spaces offers great potential for new forms of interaction especially for collaborative multi-user scenarios. In this paper, we present a way to directly interact with them through live video on mobile devices. We extend the Touch Projector interface to accommodate multiple users by showing individual content on the mobile display that would otherwise clutter the facade's canvas or distract other users. To demonstrate our concept, we built two collaborative multi-user applications: (1) painting on the facade and (2) solving a 15-puzzle. We gathered informal feedback during the ARS Electronica Festival in Linz, Austria and found that our interaction technique is (1) considered easy-to-learn, but (2) may leave users unaware of the actions of others.

/pdf/multi-user-interaction-on-media-facades-through-live-video-4rjy3sn2ng.pdf

Multi-user interaction on media facades through live video on mobile devices

Mobile phones have evolved significantly in recent years from single-purpose communication devices to multi-purpose computing devices. Despite this evolution, the interaction model for how incoming calls are handled has barely changed. Current-generation smartphones still use abrupt full-screen notifications to alert users to incoming calls, demanding a decision to either accept or decline the call. These full-screen notifications forcibly interrupt whatever activity the user was already engaged in. This might be undesirable when the user's primary task was more important than the incoming call. This paper explores the design space for how smartphones can alert users to incoming calls. We consider designs that allow users to postpone calls and also to multiplex by way of a smaller partial-screen notification. These design alternatives were evaluated in both a small-scale controlled lab study as well as a large-scale naturalistic in-the-wild study. Results show that a multiplex design solution works best because it allows people to continue working on their primary task while being made aware that there is a caller on the line. The contribution of this work is an enhanced interaction design for handling phone calls, and an understanding of how people use it for handling incoming calls.

Interrupted by a phone call: exploring designs for lowering the impact of call notifications for smartphone users

Media facades are one prominent example of how new technologies currently augment urban spaces. At the same time, they offer new, ubiquitous opportunities for novel applications. To achieve a usable and enjoyable outcome, however, designing interaction with media facades demands a structured design process. In this paper, we present our experiences designing iRiS, a system for remote interaction with media facades. We approached the development following a user-centered design approach and addressing the process at two points with additional means: (1) using a purpose-built prototyping toolkit testing and exploring both, content and hardware before deploying the system on the actual facade and (2) experimental use and adaptation of user experience (UX) evaluation methods to investigate the users actions and emotions more holistically in this context.

/pdf/designing-interaction-with-media-facades-a-case-study-rilqdky5os.pdf

Designing interaction with media façades: a case study

The electronic market has rapidly grown in the last few years. However, despite this success, consumers still enjoy visiting a "real store with real products". Therefore various common technologies have been installed in supermarkets to support the customer's shopping process and experience. In this paper, we introduce the IRL SmartCart - an instrumented shopping cart that acts as a user interface to support the shopping process. We show how RFID technology enables recognizing products that are put in the cart's basket. We are also able to determine the cart's position in an instrumented shopping environment. User input and visual output are possible by means of a touch screen, which is fitted in the IRL SmartCart's handle to support different tasks involved in the shopping process. We present and discuss different location- and context-based services that run on the cart interface system, e.g. a personalized shopping list sorted corresponding to the products in the user's vicinity or a navigation service to different products that the customer is searching for.

Sven Gehring

Papers

Back to the app: the costs of mobile application interruptions

Multi-user interaction on media facades through live video on mobile devices

Interrupted by a phone call: exploring designs for lowering the impact of call notifications for smartphone users

Designing interaction with media façades: a case study

IRL SmartCart - a user-adaptive context-aware interface for shopping assistance