▶ Addresses the accessibility, usability, and, ultimately, acceptability of Information Society Technologies by anyone, anywhere, at anytime, and through any media and device. ▶ Focuses on theoretical, methodological, and empirical research, of both technological and non-technological nature. ▶ Features papers that report on theories, methods, tools, empirical results, reviews, case studies, and best-practice examples.

Universal access in the information society

In this work we address the task of the contextual classification of an airborne LiDAR point cloud. For that purpose, we integrate a Random Forest classifier into a Conditional Random Field (CRF) framework. It is a flexible approach for obtaining a reliable classification result even in complex urban scenes. In this way, we benefit from the consideration of context on the one hand and from the opportunity to use a large amount of features on the other hand. Considering the interactions in our experiments increases the overall accuracy by 2%, though a larger improvement becomes apparent in the completeness and correctness of some of the seven classes discerned in our experiments. We compare the Random Forest approach to linear models for the computation of unary and pairwise potentials of the CRF, and investigate the relevance of different features for the LiDAR points as well as for the interaction of neighbouring points. In a second step, building objects are detected based on the classified point cloud. For that purpose, the CRF probabilities for the classes are plugged into a Markov Random Field as unary potentials, in which the pairwise potentials are based on a Potts model. The 2D binary building object masks are extracted and evaluated by the benchmark ISPRS Test Project on Urban Classification and 3D Building Reconstruction. The evaluation shows that the main buildings (larger than 50 m 2 ) can be detected very reliably with a correctness larger than 96% and a completeness of 100%.

Contextual classification of lidar data and building object detection in urban areas

Activity detection and classification are very important for autonomous monitoring of humans for applications, including assistive living, rehabilitation, and surveillance. Wearable sensors have found wide-spread use in recent years due to their ever-decreasing cost, ease of deployment and use, and ability to provide continuous monitoring as opposed to sensors installed at fixed locations. Since many smart phones are now equipped with a variety of sensors, such as accelerometer, gyroscope, and camera, it has become more feasible to develop activity monitoring algorithms employing one or more of these sensors with increased accessibility. We provide a complete and comprehensive survey on activity classification with wearable sensors, covering a variety of sensing modalities, including accelerometer, gyroscope, pressure sensors, and camera- and depth-based systems. We discuss differences in activity types tackled by this breadth of sensing modalities. For example, accelerometer, gyroscope, and magnetometer systems have a history of addressing whole body motion or global type activities, whereas camera systems provide the context necessary to classify local interactions, or interactions of individuals with objects. We also found that these single sensing modalities laid the foundation for hybrid works that tackle a mix of global and local interaction-type activities. In addition to the type of sensors and type of activities classified, we provide details on each wearable system that include on-body sensor location, employed learning approach, and extent of experimental setup. We further discuss where the processing is performed, i.e., local versus remote processing, for different systems. This is one of the first surveys to provide such breadth of coverage across different wearable sensor systems for activity classification.

A Survey on Activity Detection and Classification Using Wearable Sensors

Data fusion and multiple classifier systems for human activity detection and health monitoring: Review and open research directions

Increased life expectancy coupled with declining birth rates is leading to an aging population structure. Aging-caused changes, such as physical or cognitive decline, could affect people's quality of life, result in injuries, mental health or the lack of physical activity. Sensor-based human activity recognition (HAR) is one of the most promising assistive technologies to support older people's daily life, which has enabled enormous potential in human-centred applications. Recent surveys in HAR either only focus on the deep learning approaches or one specific sensor modality. This survey aims to provide a more comprehensive introduction for newcomers and researchers to HAR. We first introduce the state-of-art sensor modalities in HAR. We look more into the techniques involved in each step of wearable sensor modality centred HAR in terms of sensors, activities, data pre-processing, feature learning and classification, including both conventional approaches and deep learning methods. In the feature learning section, we focus on both hand-crafted features and automatically learned features using deep networks. We also present the ambient-sensor-based HAR, including camera-based systems, and the systems which combine the wearable and ambient sensors. Finally, we identify the corresponding challenges in HAR to pose research problems for further improvement in HAR.

A survey on wearable sensor modality centred human activity recognition in health care

We propose a novel pervasive system to recognise human daily activities from a wearable device. The system is designed in a form of reading glasses, named ‘Smart Glasses’, integrating a 3-axis accelerometer and a first-person view camera. Our aim is to classify user's activities of daily living (ADLs) based on both vision and head motion data. This ego-activity recognition system not only allows caretakers to track on a specific person (such as patient or elderly people), but also has the potential to remind/warn people with cognitive impairments of hazardous situations. We present the following contributions in this paper: a feature extraction method from accelerometer and video; a classification algorithm integrating both locomotive (body motions) and stationary activities (without or with small motions); a novel multi-scale dynamic graphical model structure for structured classification over time. We collect, train and validate our system on a large dataset containing 20 hours of ADLs data, including 12 daily activities under different environmental settings. Our method improves the classification performance (F-Score) of conventional approaches from 43.32%(video features) and 66.02%(acceleration features) by an average of 20–40% to 84.45%, with an overall accuracy of 90.04% in realistic ADLs.

Multi-scale Conditional Random Fields for first-person activity recognition

Multi-scale Conditional Random Fields for first-person activity recognition on elders and disabled patients

This paper proposes a novel activity recognition approach from video data obtained with a wearable camera. The objective is to recognise the user's activities from a tiny front-facing camera embedded in his/her glasses. Our system allows carers to remotely access the current status of a specified person, which can be broadly applied to those living with disabilities including the elderly who require cognitive assistance or guidance for daily activities. We collected, trained and tested our system on videos collected from different environmental settings. Sequences of four basic activities (drinking, walking, going upstairs and downstairs) are tested and evaluated in challenging real-world scenarios. An optical flow procedure is used as our primary feature extraction method, from which we downsize, reformat and classify sequence of activities using k-Nearest Neighbour algorithm (k-NN), LogitBoost (on Decision Stumps) and Support Vector Machine (SVM). We suggest the optimal settings of these classifiers through cross-validations and achieve an accuracy of 54.2% to 71.9%. Further smoothing using Hidden Markov Model (HMM) improves the result to 68.5%-82.1%.

Activity recognition from a wearable camera

This paper presents a study to determine users' attitudes toward smart devices. We conducted a web survey to elicit users' ratings for devices and combinations of tasks and devices; the results of this survey led to the development of a Recommender System (RS) for smart devices for particular tasks. We investigated user- and item-based Collaborative Filters, and compared their performance with that of global and demographic RS baselines. We then developed a technique based on Principal Components Analysis to select a subset of the original survey questions that supports the prediction of users' ratings for device-task combinations. Our results show that the accuracy of an RS that asks only a small subset of the survey questions is similar to that of an RS that predicts users' answers to one survey question on the basis of their answers to all the other questions.

Eliciting Users' Attitudes toward Smart Devices

In this paper, we propose a dense motion segmentation method for human daily activity recognition from a wearable device — "Smart Glasses". The glasses are embedded with a camera, which allows the system to automatically recognise the wearer's activities from a first-person perspective. This application can be broadly applied to patients, elderly, safety workers, e-health monitoring, or anyone requiring cognitive assistance or guidance on their activities of daily living (ADLs). We validate our system in challenging real-world scenarios, and compare two feature extraction approaches: averaged optical flow and a combined dense motion segmentation approach. We classify them using LogitBoost (on Decision Stumps) and Support Vector Machine (SVM). We also suggest the optimal settings of the classifiers through cross-validation over our ADLs database. The results show that the optical flow with average pooling has a good performance when classifying general locomotive activities. The results also indicate the benefits that dense motion segmentation features can have on reliably classify activities involving a moving object, such as hands. We achieve an overall accuracy of up to 69.76% on 12 ADLs using local classifiers, and with a Hidden Markov Model (HMM) process this accuracy improves to up to 89.59%.

Kai Zhan

Papers

Multi-scale Conditional Random Fields for first-person activity recognition

Multi-scale Conditional Random Fields for first-person activity recognition on elders and disabled patients

Activity recognition from a wearable camera

Eliciting Users' Attitudes toward Smart Devices

Dense motion segmentation for first-person activity recognition