ZurichOpenRepositoryand
Archive
UniversityofZurich
UniversityLibrary
Strickhofstrasse39
CH-8057Zurich
www.zora.uzh.ch
Year:2014
FromToday’sAugmentedHousestoTomorrow’sSmartHomes:New
DirectionsforHomeAutomationResearch
Mennicken,Sarah;Vermeulen,Jo;Huang,ElaineMay
Abstract: Aconsiderableamountofresearchhasbeencarriedouttowardsmakinglong-standingsmart
homevisions technicallyfeasible.Thetechnologicallyaugmentedhomesmade possible bythiswork
arestartingtobecomereality,butthusfarlivinginandinteractingwithsuchhomeshasintroduced
signicantcomplexitywhileoeringlimitedbenet.Asthesetechnologiesareincreasinglyadopted,the
knowledgewegainfromtheirusesuggestsaneedtorevisittheopportunitiesandchallengestheypose.
Synthesizingabroadbodyofresearchonsmarthomeswithobservationsofindustryandexperiencesfrom
ourownempiricalwork,weprovideadiscussionofongoingandemergingchallenges,namelychallenges
formeaningfultechnologies,complexdomesticspaces,andhuman-homecollaboration.Withineachof
thesethreechallengeswediscussourvisionsforfuturesmarthomesandidentifypromisingdirectionsfor
theeld.
DOI:https://doi.org/10.1145/2632048.2636076
PostedattheZurichOpenRepositoryandArchive,UniversityofZurich
ZORAURL:https://doi.org/10.5167/uzh-98109
ConferenceorWorkshopItem
Originallypublishedat:
Mennicken,Sarah;Vermeulen, Jo;Huang, ElaineMay(2014).FromToday’sAugmented Housesto
Tomorrow’sSmartHomes: NewDirectionsforHomeAutomationResearch.In: UbiComp’14,Seattle,
WA,USA,13September2014-17September2014,ACM.
DOI:https://doi.org/10.1145/2632048.2636076
From Today's Augmented Houses to Tomorrow's Smart
Homes: New Directions for Home Automation Research
Sarah Mennicken
University of Zurich
Zurich, Switzerland
mennicken@ifi.uzh.ch
Jo Vermeulen
Hasselt University – tUL – iMinds
Diepenbeek, Belgium
jo.vermeulen@uhasselt.be
Elaine M. Huang
University of Zurich
Zurich, Switzerland
huang@ifi.uzh.ch
ABSTRACT
A considerable amount of research has been carried out
towards making long-standing smart home visions
technically feasible. The technologically augmented homes
made possible by this work are starting to become reality,
but thus far living in and interacting with such homes has
introduced significant complexity while offering limited
benefit. As these technologies are increasingly adopted, the
knowledge we gain from their use suggests a need to revisit
the opportunities and challenges they pose. Synthesizing a
broad body of research on smart homes with observations
of industry and experiences from our own empirical work,
we provide a discussion of ongoing and emerging
challenges, namely challenges for meaningful technologies,
complex domestic spaces, and human-home collaboration.
Within each of these three challenges we discuss our
visions for future smart homes and identify promising
directions for the field.
Author Keywords
Home automation; smart homes; domestic technologies.
ACM Classification Keywords
H.5.m. Information interfaces and presentation (e.g., HCI):
Miscellaneous.
INTRODUCTION
The vision of smart homes, homes that cleverly support
their inhabitants through technology, has been around for
several decades [2]. Previous work has discussed whether
or how visions of Ubiquitous Computing – technologies
seamlessly interwoven in daily life – have become a reality
[1, 42]. It has also been argued that smart homes, as an
important area of focus of this vision [27, 19], have gone
from being a vision to a reality [58]. Several previously
defined challenges, such as providing means to connect
different devices, have already been addressed to some
extent or could feasibly be addressed from a technical point
of view. But new technologies have also introduced new
challenges. For example, there is the increasing difficulty of
maintaining and securing home networks due to the
invisibility of connections introduced by wireless networks,
and an increasing complexity of installations due to a larger
quantity of devices.
Smart homes were an underexplored field of research a
decade ago, but as smart and technology-augmented homes
are now emerging “in the wild”, there is a new body of
knowledge from which we can draw insights and upon
which to build. There are varied efforts to gain
understanding, ranging from scientific approaches in
academia to research and development in industry. An
excellent synthesis of challenges for smart home research
was presented thirteen years ago by Edwards and Grinter
[19]. While these challenges are still relevant for the field,
in many cases they have evolved in terms of technical
feasibility and people’s expectations as a result of the
adoption of new technologies.
As people’s expectations of what technology can do for
them are changing, the vision of what a smart home entails
is continuously evolving as well. Nowadays, many people
call a home that can be remotely accessed to turn devices
on and off “smart”, even though there is in fact no actual
automation involved. Researchers in this field might only
call homes smart when they are responsive to their
inhabitants and adapt autonomously in sophisticated ways,
e.g., using intelligent machine learning algorithms to
predict user occupancy and control the heating system. In
industry, “smart” is often used simply as a marketing term
to describe programmable technologies in general or
devices that can perform some sort of action automatically.
In this work, we define a “smart home” as a home that
either increases the comfort of their inhabitants in things
they already do or enables functionalities that were not
possible before through the use of computing technologies.
In this paper, we provide a synthesis of current challenges
and promising directions for smart home research based on
an extensive literature review, an analysis of current smart
home solutions, and our own field studies of deployed
smart home technologies. First, we describe how we
surveyed existing research emphasizing effects on
inhabitants’ user experience. Then we describe and discuss
the challenges as well as our visions for three high-level
themes we identified for smart home research, namely for
creating meaningful technologies, addressing the
complexity of domestic spaces, and fostering human-home
collaboration. For each of these themes, we discuss the
aspects that make them challenging, describe our vision of
how future smart homes should address them and map out a
set of research directions to guide the design of future smart
home user experiences and technologies for the domestic
context.
BACKGROUND
The majority of research in the early years of ubiquitous
computing in general – and smart homes in particular – was
focused on addressing technical challenges in order to
realize the Ubiquitous Computing vision. Several of those
fundamental challenges have been addressed in the area of
smart home research, for example, providing basic sensing
infrastructure or means to actuate home appliances. Many
other challenges regarding underlying technologies have
been identified and described. One key example that
provided an overview of these challenges is the seminal
work by Edwards and Grinter [19] in which they offer
detailed insights on technical challenges, such as allowing
for the incremental addition of technologies, issues of
interoperability, reliability of domestic technologies, and
ambiguity in sensing. In addition, they also discuss socio-
technical repercussions of these challenges, such as low
adoption of such technologies due to inhabitants’ lack of
technical knowledge or the difficulty of predicting social
implications. Other work [20] has focused more strongly on
the sociological perspective of smart homes to identify
challenges of general computing technologies in the home
from inhabitants’ perspectives.
Living lab initiatives such as the Aware Home [27] or
MIT’s house_n [25] facilitated the study of smart home
technologies in more depth and in contexts that closely
resemble real world domestic spaces. Mozer’s approach of
installing various sensing and actuating technologies in his
own home to build Adaptive House [37] was another way
of attempting to study actual user experiences of living with
automation technologies. All these efforts focused further
on people’s direct interaction with the technologies and
allowed for the exploration of numerous prototypes for
novel ideas in the context of technologies in domestic
spaces.
Much work has looked more specifically into assisted living
as an area of application for smart home technologies.
While several of the challenges identified in the specific
context of this field, such as legal issues or ethical issues
[12], might not be as urgent for smart homes as for a more
general population, other challenges, such as reliability of
sensing systems or cost effectiveness [12] remain just as
important. Due to the breadth of this research, researchers
were able to draw a wide set of insights on how to approach
the design of smart home technologies.
More recently, researchers have been studying deployed
smart homes technologies in their actual context of use in
family homes [8, 35]. Work that looks at early adopters of
smart homes provides us with a better understanding of
what challenges and barriers result from transferring
research to practice. Our research aims to connect
synthesized research insights with challenges identified “in
the wild” to further facilitate the creation of useful solutions
in the context of home automation.
In our work, we explore new and ongoing challenges for
smart homes and home automation technologies with a
specific focus on the user experience and its implications
for technology, rather than the other way around. While we
believe it is impossible to separate smart home technologies
from the human experience of living with them, we
emphasize that the approach we take in this work is to draw
directions for the technology by starting from the inhabitant
experience. In the remainder of this paper, we describe our
approach and the resulting findings in detail.
METHOD
The insights we draw in this work result from a synthesis of
several research activities. We conducted a formal literature
review specifically to identify user experience-centered
challenges in the smart home research landscape. We also
drew insights from our previous research activities
investigating smart homes, including empirical field studies
and interviews with smart home inhabitants, interviews
with and observations of smart home industry professionals,
and surveys of current commercial smart home products.
Literature Review
To identify themes within related work we first assembled a
list of work known to the researchers and added any of the
50 top search results on the ACM Digital Library for “home
automation” and “smart home” not already included in the
known body of work resulting in a list of 131 papers,
posters, and reports. To address our intended focus on the
user experience, we then systematically filtered the
literature set to extract papers that explicitly address the
user experience to some extent. To achieve this, we
reviewed the abstracts of the work on that list and sorted
them into different categories of relevance:
(1) Focus on automation or building technologies in
domestic spaces including a discussion of end
user experiences (35 results)
(2) Focus on automation or building technologies in
domestic spaces, but no discussion of end user
experiences (12 results)
(3) Related to automation technologies in buildings,
but no discussion of end user experiences or
domestic spaces (84 results).
While technical contributions in the area of smart homes
have been crucial to the advances of this field, our focus
was to identify novel insights specifically for the user
experience in the smart home. In order to focus on the
insights and explanations of the researchers whose work we
reviewed rather than imposing on own, we excluded the
third category. In many cases this was work that solely
focuses on providing technical innovations, such as sensor
hardware, middleware, communication protocols, or
contributions to the field of electrical engineering.
Therefore, we reviewed the resulting set of 47 papers,
consisting of the first two categories, in greater depth. From
these papers, we extracted the parts specifically relevant to
user experience for further content analysis.
The relevant parts selected for further analysis focused
mostly on (a) understanding and intelligibility of smart
homes [6], (b) means for controlling smart home
technologies, or (c) potential social effects on users. We
also included sections that addressed other issues relevant
to user experience that we felt were relevant for the
analysis. Then we analyzed our data using the affinity
diagramming method [7], deriving themes that emerged
when iteratively clustering the excerpts. We started by
analyzing how different work addressed aspects (a), (b), or
(c). Then, to derive recurring themes or tensions to put them
into the broader, overarching context of user experience in a
smart home, we iterated our analysis of the insights across
these aspects.
Subsequent to our formal analysis of selected literature, we
were also made aware of additional relevant related work
through informal discussion with other members of the
research community. When applicable, we have synthesized
insights from these additional works into the analysis we
present in this paper.
Empirical Work
The findings presented in this paper are further based on the
authors’ cumulative research activities in the realm of smart
homes, including:
• A semi-structured interview study with 22
participants (10 inhabitants in 7 households living
in smart homes, 5 people in 3 households who
were in the process of planning or building a smart
home, and 7 smart home solution providers from
industry) as well as home tours to six of the smart
home inhabitants’ homes. The results of this work
have been published in [35].
• A mixed-methods study with five people without
technical backgrounds who live in smart homes.
The focus was to understand everyday
interactions, capturing positive and negative
aspects of living with automation technologies.
The methods used in this study have been
published in [36].
• Observations of two smart home interest group
meetings including presentations of new products.
In the first meeting one of the authors presented
and discussed research results from the previously
mentioned studies with the smart home inhabitants
who make up the interest group. In a second
meeting the same author attended presentations of
novel smart home products coming out of industry.
• Two visits to different smart home construction
sites guided by a smart home provider. Data was
collected using contextual inquiry and
participatory observation methods in order to
develop an understanding of practitioner’s
everyday problems and discuss the contrast to
approaches in research.
By drawing the data from various sources in research,
industry, and practice, we aim to provide a set of directions
for moving smart home research forward by taking a
comprehensive and multi-faceted perspective on the field.
CURRENT CHALLENGES AND PROMISING
DIRECTIONS
In the following section, we reflect on challenges presented
in related work as well as new challenges and themes we
have found through our own empirical work. To structure
the insights and findings that emerged from our data, we
discuss them along three high-level themes, namely
Meaningful Technologies, Complex Domestic Spaces, and
Human-Home Collaboration. While we discuss them
individually, these themes are all highly interconnected.
Within each of these themes we discuss the specific aspects
that emerged as most critical in our analysis, why they pose
challenges for research, our visions for smart homes that
overcome these challenges to provide better inhabitant
experiences, and actionable directions for research that we
believe have promise towards fulfilling these visions.
Meaningful Technologies
Technological innovation is often still driven by a strong
interest in providing a novel contribution and making
advances within a specific field of technical research. This
kind of systems-oriented research is indispensible for
advancing the field of smart homes as it allows researchers
who focus on applications to have access to more tools to
realize their concepts. However, systems or tools that have
been developed with a focus on pushing the boundaries of
certain technologies can also introduce the risk of shaping
the visions for future applications in a limiting or restrictive
way.
Interest in Social Values and High-level Goals
As argued by Taylor et al. [49], technology is less to be
understood as something intelligent, but more of a resource
for intelligence, in which intelligence emerges through our
interactions with technology. Similarly, Rogers argues for
more engaging technology that “enables people to do what
they want, need or never even considered before by acting
in and upon the environment.” [42] We therefore argue that
an important consideration for advancing smart homes lies
in supporting the goals and values of inhabitants.
In the review of related research we found that people are
strongly interested in their own activities and the effects of
their behavior in the home [33], often in order to assess
their efforts towards achieving a specific goal, e.g.,
reducing energy consumption [5] or “optimizing their own
resource use” [18]. In other cases, they wanted to learn
more about the home and the dynamics within it in order to
reflect on the way they live. Related work has identified
people’s interest in “feeling like good parents” [29] and
suggests that smart homes could “participate in the
construction of family identity” [16]. Other work poses the
question of “how technology physically embodied in the
home might support lifestyles such as green living, slow
living, or spirituality […]” [56] indicating an existing lack
of support for such values.
A major motivation for acquiring home automation is the
interest in achieving peace of mind [8, 35] or an interest in
feeling connected to one’s home [48]. These motivations
have resulted in security-oriented solutions for the home
[18] and suggest technologies that are “readily
introspectable” with regard to the user’s skills [48]. This
strong desire to achieve “peace of mind” in respect to one’s
home is not only evident in the fact that there was an early
emphasis within industry on developing solutions for a
building’s security, but also by inhabitants’ stated desire to
know that the things one cares about in the home are safe.
Recent industry efforts, such as Mother by sen.se
1
, or
WallyHome
2
– which allow people to use one or more
sensors to monitor their home environments – are intended
to address this desire and are indicative of the interest of the
market.
The Catch of Technological Advances
At the same time however, homes augmented with
technologies intended to provide “peace of mind”, for
example through remote connection via smart phones and
Internet-connected devices, can also introduce perceived
and actual threats to privacy and security.
The increased connectedness of our homes [21] can raise
questions about what data is being collected, whether it is
transferred outside of the domestic environment, and how it
is being accessed [51]. As reported by Chetty et al. [13],
users were often not aware that their homes were accessible
to others beyond their physical boundaries through wireless
or remote access. Data leaks from sensed data in the home
could potentially be very sensitive and might allow for
serious abuse, e.g., household rhythms that expose
appropriate times to rob a home [13] or means for access-
control [51] which could be hacked by others to turn off the
lights as a relatively harmless example
1
or potentially for
purposes with more malicious intent and worse
implications. These scenarios would actually lead to the
opposite of the intended goal.
These types of negative effects on complex environments
with multiple inhabitants are hard to predict, and research
efforts often focus on specific topics, rather than
considering the home environment as a whole. Such Smart
home research typically focuses on specific areas of
applications (such as cooking, or communication, support
for the elderly or disabled) or specific underlying
technologies such as occupancy sensing, activity
recognition or location tracking. As a result of this deep but
narrow focus, technologies are often studied in a rather
isolated manner focusing on their impact on the immediate
context of use. Even if they are deployed into actual
1
https://sen.se/store/mother/
2
https://www.wallyhome.com/
households, effects on the larger context of the household
and whether inhabitants’ larger goals and values are
supported have rarely been studied.
Our literature review also indicated that people often worry
about more philosophical issues, such as whether smart
homes might make them lazy [35]. Stringer et al. [46]
suggest that we need to design “technology [that] should
require human effort in ways that keep life as mentally and
physically challenging as possible as people age.” There is
a very delicate balance between enabling goals such as
“comfort” and “convenience” without crossing the
boundary to making inhabitants feel “lazy”. This balance is
incredibly difficult to meet, especially given the fact that
households are often inhabited by multiple people, each
with different values, needs, and roles.
Smart Homes Will Support Lifestyle Choices
We envision that an ideal smart home will support its
inhabitants in living the lifestyles they choose while still
being able to cope with “irrational” exceptions from them.
Instead of “rationalizing” the life of inhabitants, smart
homes should contribute to inhabitants’ lives by adding
meaning and supporting their unique values. Research on
smart home technologies or automation technologies in the
domestic context therefore needs to put a stronger focus on
whether it is in line with the intended users’ social values
and high-level goals.
Many questions remain unanswered that need to be
addressed in order for smart homes to support those
lifestyle choices: What kind of high-level goals do people
even have? How are these manifested in domestic spaces
and how are current technologies involved already? Are
there ways for researchers to learn how goals can be
mapped onto available technologies in order to create
solutions that address such a vision? If researchers are able
to target their efforts to address high-level goals, will it be
possible to know whether novel technologies to support
these goals will be successful when deployed “in the wild”?
Can we find ways to predict, model, and possibly even
deter potential negative side effects on domestic life?
Not only do researchers have to find ways to predict
technical conflicts resulting from different configurations of
systems, we also need to find ways to predict social
conflicts that may arise from attempting to support multiple
high-level goals. Conflicting values within a household also
need to be considered, for example if parents want to live in
a more energy-conscious fashion while the kids simply
want a maximum level of comfort. Although these types of
conflicts already exist in conventional households, smart
technology intended to support goals and values adds a new
level of socio-technical complexity that needs to be
addressed.
Learn In and From “The Wild”
One key approach towards the vision of a smart home that
can support inhabitants’ high-level goals entails putting a