Showing papers by "Payam Barnaghi published in 2020"

Semi-supervised Federated Learning for Activity Recognition

Abstract: Training deep learning models on in-home IoT sensory data is commonly used to recognise human activities. Recently, federated learning systems that use edge devices as clients to support local human activity recognition have emerged as a new paradigm to combine local (individual-level) and global (group-level) models. This approach provides better scalability and generalisability and also offers better privacy compared with the traditional centralised analysis and learning models. The assumption behind federated learning, however, relies on supervised learning on clients. This requires a large volume of labelled data, which is difficult to collect in uncontrolled IoT environments such as remote in-home monitoring. In this paper, we propose an activity recognition system that uses semi-supervised federated learning, wherein clients conduct unsupervised learning on autoencoders with unlabelled local data to learn general representations, and a cloud server conducts supervised learning on an activity classifier with labelled data. Our experimental results show that using a long short-term memory autoencoder and a Softmax classifier, the accuracy of our proposed system is higher than that of both centralised systems and semi-supervised federated learning using data augmentation. The accuracy is also comparable to that of supervised federated learning systems. Meanwhile, we demonstrate that our system can reduce the number of needed labels and the size of local models, and has faster local activity recognition speed than supervised federated learning does.

Privacy-preserving activity and health monitoring on databox

Abstract: Activity recognition using deep learning and sensor data can help monitor activities and health conditions of people who need assistance in their daily lives. Deep Neural Network (DNN) models to infer the activities require data collected by in-home sensory devices. These data are often sent to a centralised cloud to be used for training the model. Centralising the data introduces privacy risks. The collected data contain sensitive information about the subjects. The cloud-based approach increases the risk that the data be stored and reused for other purposes without the owner's control. We propose a system that uses edge devices to implement activity and health monitoring locally and applies federated learning to facilitate the training process. The devices use the Databox platform to manage sensor data collected in people's homes, conduct activity recognition locally, and collaboratively train a DNN model without transferring the collected data into the cloud. We illustrate the applicability of the processing time of activity recognition on edge devices. We use a hierarchical model in which a global model is generated in the cloud, without requiring the raw data, and local models are trained on edge devices. The activity inference accuracy of the global model converges to a sufficient level after a few rounds of communication between edge devices and the cloud.

A Digital Platform for Remote Healthcare Monitoring

Abstract: We describe a digital platform developed in collaboration with clinicians and user groups to provide remote healthcare monitoring and support in a dementia care application. The platform uses data from sensory devices that are deployed in participants’ homes and utilises a set machine learning and analytical algorithms to identify risks of adverse health conditions such as Urinary Tract Infections (UTIs) and hypertension in people with dementia. The platform includes a clinical interface that is used by a monitoring team to view alerts and notifications that are generated by the algorithms and to also browse the in-home activity and physiological data in a secure and privacy-aware system. The platform complies to the information governance requirements of the UK National Healthcare Service (NHS) and is registered as a class 1 medical device. The platform has been deployed and tested in over 150 homes.

Enabling Context-Aware Search using Extracted Insights from IoT Data Streams

Abstract: The rapid growth in collecting and sharing sensory observation form the urban environments provides opportunities to plan and manage the services in the cities better and allows citizens to also observe and understand the changes in their surrounding in a better way. The new urban sensory data also creates opportunities for further application and service development by creative industries and start-ups. However, as the size and diversity of this data increase, finding and accessing the right set of data in a timely manner is becoming more challenging. This paper describes a search engine designed for indexing, searching and accessing urban sensory data. We present the key feature and architecture of the system and demonstrate some of the functionalities that are provided by searching for raw sensory observations and also pattern search functions that are enabled by a pattern analysis algorithm, supported by monitoring of data streams for changes in quality of information and remediation.

Continual Learning Using Task Conditional Neural Networks.

Abstract: Conventional deep learning models have limited capacity in learning multiple tasks sequentially. The issue of forgetting the previously learned tasks in continual learning is known as catastrophic forgetting or interference. When the input data or the goal of learning change, a continual model will learn and adapt to the new status. However, the model will not remember or recognise any revisits to the previous states. This causes performance reduction and re-training curves in dealing with periodic or irregularly reoccurring changes in the data or goals. The changes in goals or data are referred to as new tasks in a continual learning model. Most of the continual learning methods have a task-known setup in which the task identities are known in advance to the learning model. We propose Multi-view Task Conditional Neural Networks (Mv-TCNN) that does not require to known the reoccurring tasks in advance. We evaluate our model on standard datasets using MNIST, CIFAR10, CIFAR100, and also a real-world dataset that we have collected in a remote healthcare monitoring study (i.e. TIHM dataset). The proposed model outperforms the state-of-the-art solutions in continual learning and adapting to new tasks that are not defined in advance.

Improving the Accuracy of the Video Popularity Prediction Models through User Grouping and Video Popularity Classification

Abstract: This article proposes a novel approach for enhancing the video popularity prediction models. Using the proposed approach, we enhance three popularity prediction techniques that outperform the accuracy of the prior state-of-the-art solutions. The major components of the proposed approach are two novel mechanisms for ”user grouping” and ”content classification.” The user grouping method is an unsupervised clustering approach that divides the users into an adequate number of user groups with similar interests. The content classification approach identifies the classes of videos with similar popularity growth trends. To predict the popularity of the newly-released videos, our proposed popularity prediction model trains its parameters in each user group and its associated video popularity classes. Evaluations are performed through a 5-fold cross validation and on a dataset containing one month video request records of 26,706 users of BBC iPlayer. Using the proposed grouping technique, user groups of similar interest and up to two video popularity classes for each user group were detected. Our analysis shows that the accuracy of the proposed solution outperforms the state-of-the-art, including Szabo-Huberman (SH), Multivariate Linear (ML), and Multivariate linear Radial Basis Functions (MRBF) models by an average of 45%, 33%, and 24%, respectively. Finally, we discuss how various systems in the network and service management domain such as cache deployment, advertising, and video broadcasting technologies benefit from our findings to illustrate the implications.

Big Data Intelligent Networking

Abstract: The thirteen articles in this special section focus on big data analytics for intelligent networking. The Internet of Things (IoT) is likely to have a significant impact on human lives as new services and applications are developed through integration of the physical and digital worlds. IoT is an umbrella term referring to a large number of sensing and actuation devices connected to the Internet. The vast amounts of data will be generated from those devices and form big data to provide smarter living and/or improve production efficiency. The huge amount of data opens new challenges in the era of new data-driven solutions, which also have significant influence on communication networks. Current networks are often designed based on static end-to-end design principles hindering the efficient and intelligent provisioning of big data. This special issue features recent and emerging advances in the areas of big data analytics in networking applications and networking for big data

Continual learning using multi-view task conditional neural networks

Abstract: Conventional deep learning models have limited capacity in learning multiple tasks sequentially. The issue of forgetting the previously learned tasks in continual learning is known as catastrophic forgetting or interference. When the input data or the goal of learning change, a continual model will learn and adapt to the new status. However, the model will not remember or recognise any revisits to the previous states. This causes performance reduction and re-training curves in dealing with periodic or irregularly reoccurring changes in the data or goals. The changes in goals or data are referred to as new tasks in a continual learning model. Most of the continual learning methods have a task-known setup in which the task identities are known in advance to the learning model. We propose Multi-view Task Conditional Neural Networks (Mv-TCNN) that does not require to known the reoccurring tasks in advance. We evaluate our model on standard datasets using MNIST, CIFAR10, CIFAR100, and also a real-world dataset that we have collected in a remote healthcare monitoring study (i.e. TIHM dataset). The proposed model outperforms the state-of-the-art solutions in continual learning and adapting to new tasks that are not defined in advance.

Verifying the Causes of Adversarial Examples

Abstract: The robustness of neural networks is challenged by adversarial examples that contain almost imperceptible perturbations to inputs, which mislead a classifier to incorrect outputs in high confidence. Limited by the extreme difficulty in examining a high-dimensional image space thoroughly, research on explaining and justifying the causes of adversarial examples falls behind studies on attacks and defenses. In this paper, we present a collection of potential causes of adversarial examples and verify (or partially verify) them through carefully-designed controlled experiments. The major causes of adversarial examples include model linearity, one-sum constraint, and geometry of the categories. To control the effect of those causes, multiple techniques are applied such as $L_2$ normalization, replacement of loss functions, construction of reference datasets, and novel models using multi-layer perceptron probabilistic neural networks (MLP-PNN) and density estimation (DE). Our experiment results show that geometric factors tend to be more direct causes and statistical factors magnify the phenomenon, especially for assigning high prediction confidence. We believe this paper will inspire more studies to rigorously investigate the root causes of adversarial examples, which in turn provide useful guidance on designing more robust models.

Emotive Response to a Hybrid-Face Robot and Translation to Consumer Social Robots

Abstract: We introduce the conceptual formulation, design, fabrication, control and commercial translation with IoT connection of a hybrid-face social robot and validation of human emotional response to its affective interactions. The hybrid-face robot integrates a 3D printed faceplate and a digital display to simplify conveyance of complex facial movements while providing the impression of three-dimensional depth for natural interaction. We map the space of potential emotions of the robot to specific facial feature parameters and characterise the recognisability of the humanoid hybrid-face robot's archetypal facial expressions. We introduce pupil dilation as an additional degree of freedom for conveyance of emotive states. Human interaction experiments demonstrate the ability to effectively convey emotion from the hybrid-robot face to human observers by mapping their neurophysiological electroencephalography (EEG) response to perceived emotional information and through interviews. Results show main hybrid-face robotic expressions can be discriminated with recognition rates above 80% and invoke human emotive response similar to that of actual human faces as measured by the face-specific N170 event-related potentials in EEG. The hybrid-face robot concept has been modified, implemented, and released in the commercial IoT robotic platform Miko (My Companion), an affective robot with facial and conversational features currently in use for human-robot interaction in children by Emotix Inc. We demonstrate that human EEG responses to Miko emotions are comparative to neurophysiological responses for actual human facial recognition. Finally, interviews show above 90% expression recognition rates in our commercial robot. We conclude that simplified hybrid-face abstraction conveys emotions effectively and enhances human-robot interaction.

Semi-supervised Learning for Analysing the Risk of Urinary Tract Infections in People with Dementia

Abstract: Machine learning techniques combined with in-home monitoring technologies provide a unique opportunity to automate diagnosis and early detection of adverse health conditions in long-term conditions such as dementia. However, accessing sufficient labelled training samples and integrating high-quality, routinely collected data from heterogeneous in-home monitoring technologies are main obstacles hindered utilising these technologies in real-world medicine. This work presents a semi-supervised model that can continuously learn from routinely collected in-home observation and measurement data. We show how our model can process highly imbalanced and dynamic data to make robust predictions in analysing the risk of Urinary Tract Infections (UTIs) in dementia. UTIs are common in older adults and constitute one of the main causes of avoidable hospital admissions in people with dementia (PwD). Health-related conditions, such as UTI, have a lower prevalence in individuals, which classifies them as sporadic cases (i.e. rare or scattered, yet important events). This limits the access to sufficient training data, without which the supervised learning models risk becoming overfitted or biased. We introduce a probabilistic semi-supervised learning framework to address these issues. The proposed method produces a risk analysis score for UTIs using routinely collected data by in-home sensing technologies.

Machine learning for risk analysis of Urinary Tract Infection in people with dementia.

Abstract: The Urinary Tract Infections (UTIs) are one of the top reasons for unplanned hospital admissions in people with dementia, and if detected early, they can be timely treated. However, the standard UTI diagnosis tests, e.g. urine tests, will be only taken if the patients are clinically suspected of having UTIs. This causes a delay in diagnosis and treatment of the conditions and in some cases like people with dementia, the symptoms can be difficult to observe. Delay in detection and treatment of dementia is one of the key reasons for unplanned hospital admissions in people with dementia. To address these issues, we have developed a technology-assisted monitoring system, which is a Class 1 medical device. The system uses off-the-shelf and low-cost in-home sensory devices to monitor environmental and physiological data of people with dementia within their own homes. We have designed a machine learning model to use the data and provide risk analysis for UTIs. We use a semi-supervised learning model which leverage the environmental data, i.e. the data collected from the motion sensors, smart plugs and network-connected body temperature monitoring devices in the home, to detect patterns that can show the risk of UTIs. Since the data is noisy and partially labelled, we combine the neural networks and probabilistic neural networks to train an auto-encoder, which is to extract the general representation of the data. We will demonstrate our smart home management by videos/online, and show how our model can pick up the UTI related patterns.

Deep Representation for Connected Health: Semi-supervised Learning for Analysing the Risk of Urinary Tract Infections in People with Dementia

Abstract: Machine learning techniques combined with in-home monitoring technologies provide a unique opportunity to automate diagnosis and early detection of adverse health conditions in long-term conditions such as dementia. However, accessing sufficient labelled training samples and integrating high-quality, routinely collected data from heterogeneous in-home monitoring technologies are main obstacles hindered utilising these technologies in real-world medicine. This work presents a semi-supervised model that can continuously learn from routinely collected in-home observation and measurement data. We show how our model can process highly imbalanced and dynamic data to make robust predictions in analysing the risk of Urinary Tract Infections (UTIs) in dementia. UTIs are common in older adults and constitute one of the main causes of avoidable hospital admissions in people with dementia (PwD). Health-related conditions, such as UTI, have a lower prevalence in individuals, which classifies them as sporadic cases (i.e. rare or scattered, yet important events). This limits the access to sufficient training data, without which the supervised learning models risk becoming overfitted or biased. We introduce a probabilistic semi-supervised learning framework to address these issues. The proposed method produces a risk analysis score for UTIs using routinely collected data by in-home sensing technologies.