Frieder Ganz

Bio: Frieder Ganz is an academic researcher from Adobe Systems. The author has contributed to research in topics: Artificial neural network & MNIST database. The author has an hindex of 9, co-authored 22 publications receiving 458 citations. Previous affiliations of Frieder Ganz include German Research Centre for Artificial Intelligence & University of Surrey.

A Practical Evaluation of Information Processing and Abstraction Techniques for the Internet of Things

Abstract: The term Internet of Things (IoT) refers to the interaction and communication between billions of devices that produce and exchange data related to real-world objects (i.e. things). Extracting higher level information from the raw sensory data captured by the devices and representing this data as machine-interpretable or human-understandable information has several interesting applications. Deriving raw data into higher level information representations demands mechanisms to find, extract, and characterize meaningful abstractions from the raw data. This meaningful abstractions then have to be presented in a human and/or machine-understandable representation. However, the heterogeneity of the data originated from different sensor devices and application scenarios such as e-health, environmental monitoring, and smart home applications, and the dynamic nature of sensor data make it difficult to apply only one particular information processing technique to the underlying data. A considerable amount of methods from machine-learning, the semantic web, as well as pattern and data mining have been used to abstract from sensor observations to information representations. This paper provides a survey of the requirements and solutions and describes challenges in the area of information abstraction and presents an efficient workflow to extract meaningful information from raw sensor data based on the current state-of-the-art in this area. This paper also identifies research directions at the edge of information abstraction for sensor data. To ease the understanding of the abstraction workflow process, we introduce a software toolkit that implements the introduced techniques and motivates to apply them on various data sets.

A Knowledge-Based Approach for Real-Time IoT Data Stream Annotation and Processing

Abstract: Internet of Things is a generic term that refers to interconnection of real-world services which are provided by smart objects and sensors that enable interaction with the physical world. Cities are also evolving into large interconnected ecosystems in an effort to improve sustainability and operational efficiency of the city services and infrastructure. However, it is often difficult to perform real-time analysis of large amount of heterogeneous data and sensory information that are provided by various sources. This paper describes a framework for real-time semantic annotation of streaming IoT data to support dynamic integration into the Web using the Advanced Message Queuing Protocol (AMPQ). This will enable delivery of large volume of data that can influence the performance of the smart city systems that use IoT data. We present an information model to represent summarisation and reliability of stream data. The framework is evaluated with the data size and average exchanged message time using summarised and raw sensor data. Based on a statistical analysis, a detailed comparison between various sensor points is made to investigate the memory and computational cost for the stream annotation framework.

Anycost GANs for Interactive Image Synthesis and Editing

Abstract: Generative adversarial networks (GANs) have enabled photorealistic image synthesis and editing. However, due to the high computational cost of large-scale generators (e.g., StyleGAN2), it usually takes seconds to see the results of a single edit on edge devices, prohibiting interactive user experience. In this paper, inspired by quick preview features in modern rendering software, we propose Anycost GAN for interactive natural image editing. We train the Anycost GAN to support elastic resolutions and channels for faster image generation at versatile speeds. Running subsets of the full generator produce outputs that are perceptually similar to the full generator, making them a good proxy for quick preview. By using sampling-based multi-resolution training, adaptive-channel training, and a generator-conditioned discriminator, the anycost generator can be evaluated at various configurations while achieving better image quality compared to separately trained models. Furthermore, we develop new encoder training and latent code optimization techniques to encourage consistency between the different sub-generators during image projection. Anycost GAN can be executed at various cost budgets (up to 10× computation reduction) and adapt to a wide range of hardware and la tency requirements. When deployed on desktop CPUs and edge devices, our model can provide perceptually similar previews at 6-12× speedup, enabling interactive image editing. The ${\color{RubineRed}{code}}$ and ${\color{RubineRed}{demo}}$ are publicly available.

Information Abstraction for Heterogeneous Real World Internet Data

Abstract: Everyday around 2.5 quintillion bytes of data are created. There is also a growing trend toward integrating real world data into the Internet, which is provided by sensory devices, smart phones, GPS, and many other sources that capture and communicate real world data. The term Internet of Things (IoT) refers to billions of devices that produce and exchange data related to real world objects (i.e., Things). This paper focuses on how to optimize the data exchange between the sensory devices and applications in IoT and Cyber-Physical systems. In particular, a method to construct higher-level abstractions of data at local gateways is proposed. This will reduce the traffic load imposed on the communication networks that provide the real world data. The proposed method is based on an information processing algorithm where gateways analyze the data collected from the sensors and create higher level abstractions. We enhance the symbolic aggregate approximation (SAX) algorithm that is used as a building block of the abstraction creation framework, into an optimized version for sensor data, called sensor SAX. We extend the parsimonious covering theory that is usually used for medical purposes with a probabilistic parsimonious criterion in the temporal domain to infer abstractions based on time-dependent sensor data. The proposed method is analyzed and evaluated over a real world data set and the results are discussed in terms of the data size reduction, accuracy, and latency needed to create the abstractions.

Automated Semantic Knowledge Acquisition From Sensor Data

Abstract: The gathering of real-world data is facilitated by many pervasive data sources such as sensor devices and smartphones. The abundance of the sensory data raises the need to make the data easily available and understandable for the potential users and applications. Using semantic enhancements is one approach to structure and organize the data and to make it processable and interoperable by machines. In particular, ontologies are used to represent information and their relations in machine interpretable forms. In this context, a significant amount of work has been done to create real-world data description ontologies and data description models; however, little effort has been done in creating and constructing meaningful topical ontologies from a vast amount of sensory data by automated processes. Topical ontologies represent the knowledge from a certain domain providing a basic understanding of the concepts that serve as building blocks for further processing. There is a lack of solution that construct the structure and relations of ontologies based on real-world data. To address this challenge, we introduce a knowledge acquisition method that processes real-world data to automatically create and evolve topical ontologies based on rules that are automatically extracted from external sources. We use an extended $k$ - means clustering method and apply a statistic model to extract and link relevant concepts from the raw sensor data and represent them in the form of a topical ontology. We use a rule-based system to label the concepts and make them understandable for the human user or semantic analysis and reasoning tools and software. The evaluation of our work shows that the construction of a topological ontology from raw sensor data is achievable with only small construction errors.

Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications

Abstract: This paper provides an overview of the Internet of Things (IoT) with emphasis on enabling technologies, protocols, and application issues. The IoT is enabled by the latest developments in RFID, smart sensors, communication technologies, and Internet protocols. The basic premise is to have smart sensors collaborate directly without human involvement to deliver a new class of applications. The current revolution in Internet, mobile, and machine-to-machine (M2M) technologies can be seen as the first phase of the IoT. In the coming years, the IoT is expected to bridge diverse technologies to enable new applications by connecting physical objects together in support of intelligent decision making. This paper starts by providing a horizontal overview of the IoT. Then, we give an overview of some technical details that pertain to the IoT enabling technologies, protocols, and applications. Compared to other survey papers in the field, our objective is to provide a more thorough summary of the most relevant protocols and application issues to enable researchers and application developers to get up to speed quickly on how the different protocols fit together to deliver desired functionalities without having to go through RFCs and the standards specifications. We also provide an overview of some of the key IoT challenges presented in the recent literature and provide a summary of related research work. Moreover, we explore the relation between the IoT and other emerging technologies including big data analytics and cloud and fog computing. We also present the need for better horizontal integration among IoT services. Finally, we present detailed service use-cases to illustrate how the different protocols presented in the paper fit together to deliver desired IoT services.

Learning without Forgetting

Abstract: When building a unified vision system or gradually adding new capabilities to a system, the usual assumption is that training data for all tasks is always available. However, as the number of tasks grows, storing and retraining on such data becomes infeasible. A new problem arises where we add new capabilities to a Convolutional Neural Network (CNN), but the training data for its existing capabilities are unavailable. We propose our Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities. Our method performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques and performs similarly to multitask learning that uses original task data we assume unavailable. A more surprising observation is that Learning without Forgetting may be able to replace fine-tuning with similar old and new task datasets for improved new task performance.

A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges

Abstract: Uncertainty quantification (UQ) plays a pivotal role in reduction of uncertainties during both optimization and decision making processes. It can be applied to solve a variety of real-world applications in science and engineering. Bayesian approximation and ensemble learning techniques are two most widely-used UQ methods in the literature. In this regard, researchers have proposed different UQ methods and examined their performance in a variety of applications such as computer vision (e.g., self-driving cars and object detection), image processing (e.g., image restoration), medical image analysis (e.g., medical image classification and segmentation), natural language processing (e.g., text classification, social media texts and recidivism risk-scoring), bioinformatics, etc. This study reviews recent advances in UQ methods used in deep learning. Moreover, we also investigate the application of these methods in reinforcement learning (RL). Then, we outline a few important applications of UQ methods. Finally, we briefly highlight the fundamental research challenges faced by UQ methods and discuss the future research directions in this field.

Semantics for the Internet of Things: Early Progress and Back to the Future

Abstract: The Internet of Things IoT has recently received considerable interest from both academia and industry that are working on technologies to develop the future Internet. It is a joint and complex discipline that requires synergetic efforts from several communities such as telecommunication industry, device manufacturers, semantic Web, and informatics and engineering. Much of the IoT initiative is supported by the capabilities of manufacturing low-cost and energy-efficient hardware for devices with communication capacities, the maturity of wireless sensor network technologies, and the interests in integrating the physical and cyber worlds. However, the heterogeneity of the "Things" makes interoperability among them a challenging problem, which prevents generic solutions from being adopted on a global scale. Furthermore, the volume, velocity and volatility of the IoT data impose significant challenges to existing information systems. Semantic technologies based on machine-interpretable representation formalism have shown promise for describing objects, sharing and integrating information, and inferring new knowledge together with other intelligent processing techniques. However, the dynamic and resource-constrained nature of the IoT requires special design considerations to be taken into account to effectively apply the semantic technologies on the real world data. In this article the authors review some of the recent developments on applying the semantic technologies to IoT.