Showing papers by "Andreas Spanias published in 2017"

A brief survey of machine learning methods and their sensor and IoT applications

Abstract: This paper provides a brief survey of the basic concepts and algorithms used for Machine Learning and its applications. We begin with a broader definition of machine learning and then introduce various learning modalities including supervised and unsupervised methods and deep learning paradigms. In the rest of the paper, we discuss applications of machine learning algorithms in various fields including pattern recognition, sensor networks, anomaly detection, Internet of Things (IoT) and health monitoring. In the final sections, we present some of the software tools and an extensive bibliography.

Solar energy management as an Internet of Things (IoT) application

Abstract: Photovoltaic (PV) array analytics and control have become necessary for remote solar farms and for intelligent fault detection and power optimization. The management of a PV array requires auxiliary electronics that are attached to each solar panel. A collaborative industry-university-government project was established to create a smart monitoring device (SMD) and establish associated algorithms and software for fault detection and solar array management. First generation smart monitoring devices (SMDs) were built in Japan. At the same time, Arizona State University initiated research in algorithms and software to monitor and control individual solar panels. Second generation SMDs were developed later and included sensors for monitoring voltage, current, temperature, and irradiance at each individual panel. The latest SMDs include a radio and relays which allow modifying solar array connection topologies. With each panel equipped with such a sophisticated SMD, solar panels in a PV array behave essentially as nodes in an Internet of Things (IoT) type of topology. This solar energy IoT system is currently programmable and can: a) provide mobile analytics, b) enable solar farm control, c) detect and remedy faults, d) optimize power under different shading conditions, and e) reduce inverter transients. A series of federal and industry grants sponsored research on statistical signal analysis, communications, and optimization of this system. A Cyber-Physical project, whose aim is to improve solar array efficiency and robustness using new machine learning and imaging methods, was launched recently.

A cyber-physical system approach for photovoltaic array monitoring and control

Abstract: In this paper, we describe a Cyber-Physical system approach to Photovoltaic (PV) array control. A machine learning and computer vision framework is proposed for improving the reliability of utility scale PV arrays by leveraging video analysis of local skyline imagery, customized machine learning methods for fault detection, and monitoring devices that sense data and actuate at each individual panel. Our approach promises to improve efficiency in renewable energy systems using cyber-enabled sensory analysis and fusion.

Distributed Node Counting in Wireless Sensor Networks in the Presence of Communication Noise

Abstract: Distributed node counting in wireless sensor networks can be important in various applications, such as network maintenance and information aggregation. In this paper, a distributed consensus algorithm for estimating the number of nodes in a wireless sensor network in the presence of communication noise is introduced. In networks with a fusion center, counting the number of nodes can easily be done by letting each node to transmit a fixed constant value to the fusion center. In a network without a fusion center, where nodes do not know the graph structure, estimating the number of nodes is not straightforward. The proposed algorithm is based on distributed average consensus and norm estimation. Different sources of error are explicitly discussed; the Fisher information and the distribution of the final estimate are derived. Several design parameters and how they affect the performance of the algorithm are studied, which provide guidelines toward making the estimation error smaller. Simulation results corroborating the theory are also provided.

Location Based Distributed Spectral Clustering for Wireless Sensor Networks

Abstract: A distributed spectral clustering algorithm to group sensors based on their location in a wireless sensor network (WSN) is proposed. For machine learning and data mining applications in WSN's, gathering data at a fusion center is vulnerable to attacks and creates data congestion. To avoid this, we propose a robust distributed clustering method without a fusion center. The algorithm combines distributed eigenvector computation and distributed K-means clustering. A distributed power iteration method is used to compute the eigenvector of the graph Laplacian. At steady state, all nodes converge to a value in the eigenvector of the algebraic connectivity of the graph Laplacian. Clustering is carried out on the eigenvector using a distributed K-means algorithm. Location information of the sensor is only used to establish the network topology and this information is not exchanged in the network. This algorithm works for any connected graph structure. Simulation results supporting the theory are also provided.

Attend and Diagnose: Clinical Time Series Analysis using Attention Models

Abstract: With widespread adoption of electronic health records, there is an increased emphasis for predictive models that can effectively deal with clinical time-series data. Powered by Recurrent Neural Network (RNN) architectures with Long Short-Term Memory (LSTM) units, deep neural networks have achieved state-of-the-art results in several clinical prediction tasks. Despite the success of RNNs, its sequential nature prohibits parallelized computing, thus making it inefficient particularly when processing long sequences. Recently, architectures which are based solely on attention mechanisms have shown remarkable success in transduction tasks in NLP, while being computationally superior. In this paper, for the first time, we utilize attention models for clinical time-series modeling, thereby dispensing recurrence entirely. We develop the \textit{SAnD} (Simply Attend and Diagnose) architecture, which employs a masked, self-attention mechanism, and uses positional encoding and dense interpolation strategies for incorporating temporal order. Furthermore, we develop a multi-task variant of \textit{SAnD} to jointly infer models with multiple diagnosis tasks. Using the recent MIMIC-III benchmark datasets, we demonstrate that the proposed approach achieves state-of-the-art performance in all tasks, outperforming LSTM models and classical baselines with hand-engineered features.

A Cyber-Physical Photovoltaic Array Monitoring and Control System

Abstract: A cyber physical system approach for a utility-scale photovoltaic PV array monitoring and control is presented in this article. This system consists of sensors that capture voltage, current, temper...

Development of signal processing online labs using HTML5 and mobile platforms

Abstract: Several web-based signal processing simulation packages for education have been developed in a Java environment. Although this environment has provided convenience and accessibility using standard browser technology, it has recently become vulnerable to cyber-attacks and is no longer compatible with secure browsers. In this paper, we describe our efforts to transform our award-winning J-DSP online laboratory by rebuilding it on an HTML5 framework. Along with a new simulation environment, we have redesigned the interface to enable several new functionalities and an entirely new educational experience. These new features include functions that enable real-time interfaces with sensor boards and mobile phones. The Web 4.0 HTML5 technology departs from older Java interfaces and provides an interactive graphical user interface (GUI) enabling seamless connectivity and both software and hardware experiences for students in DSP classes.

A deep learning approach to multiple kernel fusion

Abstract: Kernel fusion is a popular and effective approach for combining multiple features that characterize different aspects of data. Traditional approaches for Multiple Kernel Learning (MKL) attempt to learn the parameters for combining the kernels through sophisticated optimization procedures. In this paper, we propose an alternative approach that creates dense embeddings for data using the kernel similarities and adopts a deep neural network architecture for fusing the embeddings. In order to improve the effectiveness of this network, we introduce the kernel dropout regularization strategy coupled with the use of an expanded set of composition kernels. Experiment results on a real-world activity recognition dataset show that the proposed architecture is effective in fusing kernels and achieves state-of-the-art performance.

Signal processing and machine learning concepts using the reflections echolocation app

Abstract: This paper describes the use of a space usage determination algorithm for teaching signal processing and machine learning concepts to undergraduate electrical engineering and computer science students. An Android device transmits a high-frequency signal in an unknown space. The device determines the reflective properties of this unknown space by analyzing the received signal. Based on the features extracted from this signal, the app measures distances and determines how the space can be utilized for various application such as libraries, conference rooms, or laboratories. The application and related algorithms use concepts such cross-correlation, feature extraction, learning/training algorithms, and discrimination/decision making. These concepts are typically covered in undergraduate classes such as Digital Signal Processing, Control Systems, and Probability and Statistics; and graduate-level classes such as Pattern Recognition and Detection and Estimation Theory. The app is used to create compelling demonstrations and immersive exercises to teach basic concepts related to signal processing and machine learning. Undergraduate student hands-on workshops and outreach activities are planned to evaluate the effectiveness of this approach. Assessment results will be presented at the conference.

Reflections: An eModule for echolocation education

Abstract: An Android-based eModule app has been designed and developed for science, technology, engineering, and mathematics (STEM) education. The eModule consists of: (1) an Android demonstration of echolocation; (2) a set of notes describing the functionality of the app, the basics of echolocation, and its application to advanced signal processing systems such as RADAR, LIDAR, and SONAR; (3) quizzes to test the concepts introduced by the demonstration and the notes; and (4) companion videos. The eModule is, therefore, a holistic teaching and learning app that can be used across various grade levels including K-12, undergraduate signals and systems, and graduate DSP education.

A data-driven basis for direct estimation of functionals of distributions.

Abstract: A number of fundamental quantities in statistical signal processing and information theory can be expressed as integral functions of two probability density functions. Such quantities are called density functionals as they map density functions onto the real line. For example, information divergence functions measure the dissimilarity between two probability density functions and are particularly useful in a number of applications. Typically, estimating these quantities requires complete knowledge of the underlying distribution followed by multi-dimensional integration. Existing methods make parametric assumptions about the data distribution or use non-parametric density estimation followed by high-dimensional integration. In this paper, we propose a new alternative. We introduce the concept of "data-driven" basis functions - functions of distributions whose value we can estimate given only samples from the underlying distributions without requiring distribution fitting or direct integration. We derive a new data-driven complete basis that is similar to the deterministic Bernstein polynomial basis and develop two methods for performing basis expansions of functionals of two distributions. We also show that the new basis set allows us to approximate functions of distributions as closely as desired. Finally, we evaluate the methodology by developing data driven estimators for the Kullback-Leibler divergences and the Hellinger distance and by constructing tight data-driven bounds on the Bayes Error Rate.

Optimizing Kernel Machines using Deep Learning

Abstract: Building highly non-linear and non-parametric models is central to several state-of-the-art machine learning systems. Kernel methods form an important class of techniques that induce a reproducing kernel Hilbert space (RKHS) for inferring non-linear models through the construction of similarity functions from data. These methods are particularly preferred in cases where the training data sizes are limited and when prior knowledge of the data similarities is available. Despite their usefulness, they are limited by the computational complexity and their inability to support end-to-end learning with a task-specific objective. On the other hand, deep neural networks have become the de facto solution for end-to-end inference in several learning paradigms. In this article, we explore the idea of using deep architectures to perform kernel machine optimization, for both computational efficiency and end-to-end inferencing. To this end, we develop the DKMO (Deep Kernel Machine Optimization) framework, that creates an ensemble of dense embeddings using Nystrom kernel approximations and utilizes deep learning to generate task-specific representations through the fusion of the embeddings. Intuitively, the filters of the network are trained to fuse information from an ensemble of linear subspaces in the RKHS. Furthermore, we introduce the kernel dropout regularization to enable improved training convergence. Finally, we extend this framework to the multiple kernel case, by coupling a global fusion layer with pre-trained deep kernel machines for each of the constituent kernels. Using case studies with limited training data, and lack of explicit feature sources, we demonstrate the effectiveness of our framework over conventional model inferencing techniques.

Distributed center and coverage region estimation in wireless sensor networks using diffusion adaptation

Abstract: A fully distributed algorithm for estimating the center and coverage region of a wireless sensor network (WSN) is proposed. The proposed algorithm is useful in many applications, such as finding the required power for a certain level of connectivity in WSNs and localizing a service center in a network. The network coverage region is defined to be the smallest sphere that covers all the sensor nodes. The center and radius of the smallest covering sphere are estimated. The center estimation is formulated as a convex optimization problem using soft-max approximation. Then, diffusion adaptation is used for distributed optimization to estimate the center. After all the sensors obtain the center estimates, max consensus is used to calculate the radius distributively. The performance analysis of the proposed algorithm is provided, as a function of a design parameter controls the trade-off between the center estimation error and the convergence speed of the algorithm. Simulation results are provided.

A signal recovery method for array processing

Abstract: In array processing, if a few elements are turned off or malfunctioned, signal information and array resolution will degrade. Considerable research has been conducted to develop algorithms that can estimate the missing array data. In this paper, a data recovery method based on matrix completion (MC) theory is proposed. This approach utilizes the redundant information among array elements to recover the missing data. We verify the validity of our method using simulation.