An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

Improved techniques for managing enterprise applications on mobile devices are described herein. Each enterprise mobile application running on the mobile device has an associated policy through which it interacts with its environment. The policy selectively blocks or allows activities involving the enterprise application in accordance with rules established by the enterprise. Together, the enterprise applications running on the mobile device form a set of managed applications. Managed applications are typically allowed to exchange data with other managed applications, but are blocked from exchanging data with other applications, such as the user's own personal applications. Policies may be defined to manage data sharing, mobile resource management, application specific information, networking and data access solutions, device cloud and transfer, dual mode application software, enterprise app store access, and virtualized application and resources, among other things.

Policy-Based Application Management

The usefulness of mobile devices has increased greatly in recent years allowing users to perform more tasks in a mobile context. This increase in usefulness has come at the expense of the usability of these devices in some contexts. We conducted a small review of mobile usability models and found that usability is usually measured in terms of three attributes; effectiveness, efficiency and satisfaction. Other attributes, such as cognitive load, tend to be overlooked in the usability models that are most prominent despite their likely impact on the success or failure of an application. To remedy this we introduces the PACMAD (People At the Centre of Mobile Application Development) usability model which was designed to address the limitations of existing usability models when applied to mobile devices. PACMAD brings together significant attributes from different usability models in order to create a more comprehensive model. None of the attributes that it includes are new, but the existing prominent usability models ignore one or more of them. This could lead to an incomplete usability evaluation. We performed a literature search to compile a collection of studies that evaluate mobile applications and then evaluated the studies using our model.

/pdf/usability-of-mobile-applications-literature-review-and-2sgkvggy18.pdf

Usability of mobile applications: literature review and rationale for a new usability model

Ambient Intelligence (AmI) is a new paradigm in information technology aimed at empowering people's capabilities by means of digital environments that are sensitive, adaptive, and responsive to human needs, habits, gestures, and emotions. This futuristic vision of daily environment will enable innovative human-machine interactions characterized by pervasive, unobtrusive, and anticipatory communications. Such innovative interaction paradigms make AmI technology a suitable candidate for developing various real life solutions, including in the healthcare domain. This survey will discuss the emergence of AmI techniques in the healthcare domain, in order to provide the research community with the necessary background. We will examine the infrastructure and technology required for achieving the vision of AmI, such as smart environments and wearable medical devices. We will summarize the state-of-the-art artificial intelligence (AI) methodologies used for developing AmI system in the healthcare domain, including various learning techniques (for learning from user interaction), reasoning techniques (for reasoning about users' goals and intensions), and planning techniques (for planning activities and interactions). We will also discuss how AmI technology might support people affected by various physical or mental disabilities or chronic disease. Finally, we will point to some of the successful case studies in the area and we will look at the current and future challenges to draw upon the possible future research paths.

/pdf/a-survey-on-ambient-intelligence-in-healthcare-wkvwfodws9.pdf

A Survey on Ambient Intelligence in Healthcare

Stress can have long term adverse effects on individuals' physical and mental well-being. Changes in the speech production process is one of many physiological changes that happen during stress. Microphones, embedded in mobile phones and carried ubiquitously by people, provide the opportunity to continuously and non-invasively monitor stress in real-life situations. We propose StressSense for unobtrusively recognizing stress from human voice using smartphones. We investigate methods for adapting a one-size-fits-all stress model to individual speakers and scenarios. We demonstrate that the StressSense classifier can robustly identify stress across multiple individuals in diverse acoustic environments: using model adaptation StressSense achieves 81% and 76% accuracy for indoor and outdoor environments, respectively. We show that StressSense can be implemented on commodity Android phones and run in real-time. To the best of our knowledge, StressSense represents the first system to consider voice based stress detection and model adaptation in diverse real-life conversational situations using smartphones.

/pdf/stresssense-detecting-stress-in-unconstrained-acoustic-3py0pmhbhs.pdf

StressSense: detecting stress in unconstrained acoustic environments using smartphones

We present AffectAura, an emotional prosthetic that allows users to reflect on their emotional states over long periods of time. We designed a multimodal sensor set-up for continuous logging of audio, visual, physiological and contextual data, a classification scheme for predicting user affective state and an interface for user reflection. The system continuously predicts a user's valence, arousal and engage-ment, and correlates this with information on events, communications and data interactions. We evaluate the interface through a user study consisting of six users and over 240 hours of data, and demonstrate the utility of such a reflection tool. We show that users could reason forward and backward in time about their emotional experiences using the interface, and found this useful.

/pdf/affectaura-an-intelligent-system-for-emotional-memory-548ehz0ih5.pdf

AffectAura: an intelligent system for emotional memory

Automatically identifying the person you are talking with using continuous audio sensing has the potential to enable many pervasive computing applications from memory assistance to annotating life logging data. However, a number of challenges, including energy efficiency and training data acquisition, must be addressed before unobtrusive audio sensing is practical on mobile devices. We built SpeakerSense, a speaker identification prototype that uses a heterogeneous multi-processor hardware architecture that splits computation between a low power processor and the phone's application processor to enable continuous background sensing with minimal power requirements. Using SpeakerSense, we benchmarked several system parameters (sampling rate, GMM complexity, smoothing window size, and amount of training data needed) to identify thresholds that balance computation cost with performance. We also investigated channel compensation methods that make it feasible to acquire training data from phone calls and an automatic segmentation method for training speaker models based on one-to-one conversations.

/pdf/speakersense-energy-efficient-unobtrusive-speaker-3ds26lr67k.pdf

SpeakerSense: energy efficient unobtrusive speaker identification on mobile phones

Tag clouds have proliferated over the web over the last decade. They provide a visual summary of a collection of texts by visually depicting the tag frequency by font size. In use, tag clouds can evolve as the associated data source changes over time. Interesting discussions around tag clouds often include a series of tag clouds and consider how they evolve over time. However, since tag clouds do not explicitly represent trends or support comparisons, the cognitive demands placed on the person for perceiving trends in multiple tag clouds are high. In this paper, we introduce SparkClouds, which integrate sparklines into a tag cloud to convey trends between multiple tag clouds. We present results from a controlled study that compares SparkClouds with two traditional trend visualizations-multiple line graphs and stacked bar charts-as well as Parallel Tag Clouds. Results show that SparkClouds' ability to show trends compares favourably to the alternative visualizations.

/pdf/sparkclouds-visualizing-trends-in-tag-clouds-4x8b6i74ki.pdf

SparkClouds: Visualizing Trends in Tag Clouds

Mobile phones are becoming increasingly personalized in terms of the data they store and the types of services they provide. At the same time, field studies have reported that there are a variety of situations in which it is natural for people to share their phones with others. However, most mobile phones support a binary security model that offers all-or-nothing access to the phone. We interviewed 12 smartphone users to explore how security and data privacy concerns affected their willingness to share their mobile phones. The diversity of guest user categorizations and associated security constraints expressed by the participants suggests the need for a security model richer than today's binary model.

/pdf/can-i-borrow-your-phone-understanding-concerns-when-sharing-39zhewbjhr.pdf

Can i borrow your phone?: understanding concerns when sharing mobile phones

Research has demonstrated that information workers often manage several different computing devices in an effort to balance convenience, mobility, input efficiency, and content readability throughout their day. The high portability of the mobile phone has made it an increasingly valuable member of this ecosystem of devices. To understand how future technologies might better support productivity tasks as people transition between devices, we examined the mobile phone and PC usage patterns of sixteen information workers across several weeks. Our data logs, together with follow-up interview feedback from four of the participants, confirm that the phone is highly leveraged for digital information needs beyond calls and SMS, but suggest that these users do not currently traverse the device boundary within a given task.

/pdf/working-overtime-patterns-of-smartphone-and-pc-usage-in-the-2rldp1us7p.pdf

Amy Karlson

Papers

AffectAura: an intelligent system for emotional memory

SpeakerSense: energy efficient unobtrusive speaker identification on mobile phones

SparkClouds: Visualizing Trends in Tag Clouds

Can i borrow your phone?: understanding concerns when sharing mobile phones

Working Overtime: Patterns of Smartphone and PC Usage in the Day of an Information Worker