This paper presents Soli, a new, robust, high-resolution, low-power, miniature gesture sensing technology for human-computer interaction based on millimeter-wave radar. We describe a new approach to developing a radar-based sensor optimized for human-computer interaction, building the sensor architecture from the ground up with the inclusion of radar design principles, high temporal resolution gesture tracking, a hardware abstraction layer (HAL), a solid-state radar chip and system architecture, interaction models and gesture vocabularies, and gesture recognition. We demonstrate that Soli can be used for robust gesture recognition and can track gestures with sub-millimeter accuracy, running at over 10,000 frames per second on embedded hardware.

/pdf/soli-ubiquitous-gesture-sensing-with-millimeter-wave-radar-mcplexjo41.pdf

Soli: ubiquitous gesture sensing with millimeter wave radar

The recent introduction of novel acquisition devices like the Leap Motion and the Kinect allows to obtain a very informative description of the hand pose that can be exploited for accurate gesture recognition. This paper proposes a novel hand gesture recognition scheme explicitly targeted to Leap Motion data. An ad-hoc feature set based on the positions and orientation of the fingertips is computed and fed into a multi-class SVM classifier in order to recognize the performed gestures. A set of features is also extracted from the depth computed from the Kinect and combined with the Leap Motion ones in order to improve the recognition performance. Experimental results present a comparison between the accuracy that can be obtained from the two devices on a subset of the American Manual Alphabet and show how, by combining the two features sets, it is possible to achieve a very high accuracy in real-time.

/pdf/hand-gesture-recognition-with-leap-motion-and-kinect-devices-lm3lxm555z.pdf

Hand gesture recognition with leap motion and kinect devices

We present the results of an evaluation of the performance of the Leap Motion Controller with the aid of a professional, high-precision, fast motion tracking system A set of static and dynamic measurements was performed with different numbers of tracking objects and configurations For the static measurements, a plastic arm model simulating a human arm was used A set of 37 reference locations was selected to cover the controller's sensory space For the dynamic measurements, a special V-shaped tool, consisting of two tracking objects maintaining a constant distance between them, was created to simulate two human fingers In the static scenario, the standard deviation was less than 05 mm The linear correlation revealed a significant increase in the standard deviation when moving away from the controller The results of the dynamic scenario revealed the inconsistent performance of the controller, with a significant drop in accuracy for samples taken more than 250 mm above the controller's surface The Leap Motion Controller undoubtedly represents a revolutionary input device for gesture-based human-computer interaction; however, due to its rather limited sensory space and inconsistent sampling frequency, in its current configuration it cannot currently be used as a professional tracking system

An Analysis of the Precision and Reliability of the Leap Motion Sensor and Its Suitability for Static and Dynamic Tracking

Three-dimensional hand gesture recognition has attracted increasing research interests in computer vision, pattern recognition, and human-computer interaction. The emerging depth sensors greatly inspired various hand gesture recognition approaches and applications, which were severely limited in the 2D domain with conventional cameras. This paper presents a survey of some recent works on hand gesture recognition using 3D depth sensors. We first review the commercial depth sensors and public data sets that are widely used in this field. Then, we review the state-of-the-art research for 3D hand gesture recognition in four aspects: 1) 3D hand modeling; 2) static hand gesture recognition; 3) hand trajectory gesture recognition; and 4) continuous hand gesture recognition. While the emphasis is on 3D hand gesture recognition approaches, the related applications and typical systems are also briefly summarized for practitioners.

Survey on 3D Hand Gesture Recognition

Dynamic hand gesture recognition is a crucial but challenging task in the pattern recognition and computer vision communities. In this paper, we propose a novel feature vector which is suitable for representing dynamic hand gestures, and presents a satisfactory solution to recognizing dynamic hand gestures with a Leap Motion controller (LMC) only. These have not been reported in other papers. The feature vector with depth information is computed and fed into the Hidden Conditional Neural Field (HCNF) classifier to recognize dynamic hand gestures. The systematic framework of the proposed method includes two main steps: feature extraction and classification with the HCNF classifier. The proposed method is evaluated on two dynamic hand gesture datasets with frames acquired with a LMC. The recognition accuracy is 89.5% for the LeapMotion-Gesture3D dataset and 95.0% for the Handicraft-Gesture dataset. Experimental results show that the proposed method is suitable for certain dynamic hand gesture recognition tasks.

Dynamic Hand Gesture Recognition With Leap Motion Controller

German nuclear power phase out in 2022 leads to significant reconstruction of the energy transmission system. Thus, efficient identification of practical transmission routes with minimum impact on ecological and economical interests is of growing importance. Due to the sensitivity of Germany’s public to grid expansion (especially in case of overhead lines), the participation and planning process needs to provide a high degree of openness and accountability. Therefore, a new methodological approach for the computer-assisted finding of optimal power-line routes considering planning, ecological and economic decision criteria is presented. The approach is implemented in a tool-chain for the determination of transmission line routes (and sets of transmission line route alternatives) based on multi-criteria optimisation. Additionally, a decision support system, based on common Geographic Information Systems (GIS), consisting of interactive visualisation and exploration of the solution space is proposed.

Multi-Objective Optimisation Based Planning of Power-Line Grid Expansions

Health monitoring of civil and industrial structures has been gaining importance since the collapse of the bridge in Genoa (Italy). It is vital for the creation and maintenance of reliable infrastructure. Traditional manual inspections for this task are crucial but time consuming. We present a novel approach for combining Unmanned Aerial Vehicles (UAVs) and artificial intelligence to tackle the above-mentioned challenges. Modern architectures in Convolutional Neural Networks (CNNs) were adapted to the special characteristics of data streams gathered from UAV visual sensors. The approach allows for automated detection and localization of various damages to steel structures, coatings, and fasteners, e.g., cracks or corrosion, under uncertain and real-life environments. The proposed model is based on a multi-stage cascaded classifier to account for the variety of detail level from the optical sensor captured during an UAV flight. This allows for reconciliation of the characteristics of gathered image data and crucial aspects from a steel engineer’s point of view. To improve performance of the system and minimize manual data annotation, we use transfer learning based on the well-known COCO dataset combined with field inspection images. This approach provides a solid data basis for object localization and classification with state-of-the-art CNN architectures.

https://www.mdpi.com/2504-3900/42/1/69/pdf

CNN-Based Deep Architecture for Health Monitoring of Civil and Industrial Structures Using UAVs

Critical infrastructure (CI) networks are essential for the survival and functionality of society and the economy. Disruptions to CI services and the cascading effects of these disruptions are not currently included in flood risk management (FRM). The work presented in this study integrates CI into every step of FRM, including flood risk analysis, risk mitigation and risk communication. A CI network modelling technique enables the flood consequences for CI to be quantified as part of the flood risk analysis. The CI consequences derived from this analysis include spatial overviews and the temporal succession of CI disruptions. The number of affected CI end-users and the duration of the disruption are arranged in a risk matrix and in a decision-making matrix. Thus, the total flood risk is extended with CI consequences. By integrating CI and CI network characteristics into the flood risk assessment and the mitigation steps, a wider range of measures for action can be considered. Additionally, the continuous participation of CI operators is introduced as beneficial for every step of the FRM. A case study in Accra, Ghana proves the benefits of CI integration for all FRM steps. During participatory CI stakeholder engagements for this study six CI sectors were identified for the assembly of the CI network. The backbone of the analysis is a multisectoral, layered CI network model with 433 point elements, 1216 connector elements and 486 polygon elements.

/pdf/integrating-critical-infrastructure-networks-into-flood-risk-3ia71v36.pdf

Integrating Critical Infrastructure Networks into Flood Risk Management

Die vorliegende Arbeit zeigt eine neuartige Herangehensweise zur Analyse von Keilschrifttafeln sowie zur Rekonstruktion von dreidimensionalen Schrifttragern uber eine raumbezogene relationale Datenbank auf. Dabei ist bereits die integrierte Datenhaltung von dreidimensionalen syntaktischen und semantischen Merkmalen fur Keilschrifttafeln innovativ, absolutes Neuland wird mit der Datenbank-gestutzten Ermittlung von 3D-Joins auf Basis von geometrischen Informationen sowie zeit- und ortsspezifischen palaographischen Kriterien betreten. In Kombination mit dieser zur Verfugung stehenden Veredelung bestehender Daten und einer nachhaltigen Erweiterung der Datenbasis, werden neue Moglichkeiten zur Gewinnung philologischer Erkenntnisse zur Verfugung stehen und vorhandene Ansatze effektiviert.

Modellierung einer raumbezogenen Datenbank zur Repräsentation und Analyse syntaktischer und semantischer Merkmale von Keilschrifttafeln

In recent years, the issue of high groundwater levels has caught attention. Unfavorable consequences of high groundwater levels are especially damage to buildings, infrastructure, and the environment. Processes that lead to high groundwater levels are hydrological (heavy or extended rainfall and flood events), or anthropogenic (reduced groundwater extractions, interaction with sewer networks, hydraulic engineering measures, structural interventions in the water balance, and mining activities). Several different map products have been prepared for the information of inhabitants and for planning purposes, and also methods for damage and risk analysis related to high groundwater levels have been developed. Groundwater management measures and structural measures are available to reduce the risk related to high groundwater levels. An operational management system could be combined from existing components, but operational forecasting systems for high groundwater levels are—different to flood forecasting systems—not yet common practice. A better understanding of the processes and the development of integrated approaches for modeling, design, planning, forecasting, and warning, as well as improvement of interdisciplinary collaboration between different organizations, are recommendations for the future.

Daniel Bachmann

Papers

Multi-Objective Optimisation Based Planning of Power-Line Grid Expansions

CNN-Based Deep Architecture for Health Monitoring of Civil and Industrial Structures Using UAVs

Integrating Critical Infrastructure Networks into Flood Risk Management

Modellierung einer raumbezogenen Datenbank zur Repräsentation und Analyse syntaktischer und semantischer Merkmale von Keilschrifttafeln

High groundwater levels: Processes, consequences, and management