Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Recent advances in physical reservoir computing: A review

2011 international joint conference on neural networks (ijcnn)

In many real-world application, e.g., speech recognition or sleep stage classification, data are captured over the course of time, constituting a Time-Series. Time-Series often contain temporal dependencies that cause two otherwise identical points of time to belong to different classes or predict different behavior. This characteristic generally increases the difficulty of analysing them. Existing techniques often depended on hand-crafted features that were expensive to create and required expert knowledge of the field. With the advent of Deep Learning new models of unsupervised learning of features for Time-series analysis and forecast have been developed. Such new developments are the topic of this paper: a review of the main Deep Learning techniques is presented, and some applications on Time-Series analysis are summaried. The results make it clear that Deep Learning has a lot to contribute to the field.

Deep Learning for Time-Series Analysis

Time-series forecasting is an important research and application area. Much effort has been devoted over the past several decades to develop and improve the time-series forecasting models. This paper introduces a new time-series forecasting model based on the flexible neural tree (FNT). The FNT model is generated initially as a flexible multi-layer feed-forward neural network and evolved using an evolutionary procedure. Very often it is a difficult task to select the proper input variables or time-lags for constructing a time-series model. Our research demonstrates that the FNT model is capable of handing the task automatically. The performance and effectiveness of the proposed method are evaluated using time series prediction problems and compared with those of related methods.

Time-series forecasting using flexible neural tree model

This paper presents a new methodology for the hardware implementation of neural networks (NNs) based on probabilistic laws. The proposed encoding scheme circumvents the limitations of classical stochastic computing (based on unipolar or bipolar encoding) extending the representation range to any real number using the ratio of two bipolar-encoded pulsed signals. Furthermore, the novel approach presents practically a total noise-immunity capability due to its specific codification. We introduce different designs for building the fundamental blocks needed to implement NNs. The validity of the present approach is demonstrated through a regression and a pattern recognition task. The low cost of the methodology in terms of hardware, along with its capacity to implement complex mathematical functions (such as the hyperbolic tangent), allows its use for building highly reliable systems and parallel computing.

A New Stochastic Computing Methodology for Efficient Neural Network Implementation

Hardware implementation of artificial neural networks (ANNs) allows exploiting the inherent parallelism of these systems. Nevertheless, they require a large amount of resources in terms of area and power dissipation. Recently, Reservoir Computing (RC) has arisen as a strategic technique to design recurrent neural networks (RNNs) with simple learning capabilities. In this work, we show a new approach to implement RC systems with digital gates. The proposed method is based on the use of probabilistic computing concepts to reduce the hardware required to implement different arithmetic operations.Theresult is the development of a highly functional system with low hardware resources. The presented methodology is applied to chaotic time-series forecasting.

/pdf/fpga-based-stochastic-echo-state-networks-for-time-series-2g6uwsnour.pdf

FPGA-based stochastic echo state networks for time-series forecasting

Virtual screening (VS) has become a key computational tool in early drug design and screening performance is of high relevance due to the large volume of data that must be processed to identify molecules with the sought activity-related pattern. At the same time, the hardware implementations of spiking neural networks (SNNs) arise as an emerging computing technique that can be applied to parallelize processes that normally present a high cost in terms of computing time and power. Consequently, SNN represents an attractive alternative to perform time-consuming processing tasks, such as VS. In this brief, we present a smart stochastic spiking neural architecture that implements the ultrafast shape recognition (USR) algorithm achieving two order of magnitude of speed improvement with respect to USR software implementations. The neural system is implemented in hardware using field-programmable gate arrays allowing a highly parallelized USR implementation. The results show that, due to the high parallelization of the system, millions of compounds can be checked in reasonable times. From these results, we can state that the proposed architecture arises as a feasible methodology to efficiently enhance time-consuming data-mining processes such as 3-D molecular similarity search.

A Stochastic Spiking Neural Network for Virtual Screening

In this article, we analyze the impact of within-die thermal gradients on clock skew, considering temperature's effect on active devices and the interconnect system. This effect, along with the fact that the test-induced thermal map can differ from the normal-mode thermal map, motivates the need for a careful consideration of the impact of temperature gradients on delay during test. After our analysis, we propose a dual-VDD clocking strategy that reduces temperature-related clock skew effects during test. Clock network design is a critical task in developing high-performance circuits because circuit performance and functionality depend directly on this subsystem's performance. When distributing the clock signal over the chip, clock edges might reach various circuit registers at different times. The difference in clock arrival time between the first and last registers receiving the signal is called clock skew. With tens of millions of transistors integrated on the chip, distributing the clock signal with near-zero skew introduces important constraints in the clock distribution network's physical implementation and affects overall circuit power and area

Impact of Thermal Gradients on Clock Skew and Testing

Minimal hardware implementations able to cope with the processing of large amounts of data in reasonable times are highly desired in our information-driven society. In this work we review the application of stochastic computing to probabilistic-based pattern-recognition analysis of huge database sets. The proposed technique consists in the hardware implementation of a parallel architecture implementing a similarity search of data with respect to different pre-stored categories. We design pulse-based stochastic-logic blocks to obtain an efficient pattern recognition system. The proposed architecture speeds up the screening process of huge databases by a factor of 7 when compared to a conventional digital implementation using the same hardware area.

/pdf/ultra-fast-data-mining-hardware-architecture-based-on-3v2rs5rnf1.pdf

Josep L. Rosselló

Papers

A New Stochastic Computing Methodology for Efficient Neural Network Implementation

FPGA-based stochastic echo state networks for time-series forecasting

A Stochastic Spiking Neural Network for Virtual Screening

Impact of Thermal Gradients on Clock Skew and Testing

Ultra-Fast Data-Mining Hardware Architecture Based on Stochastic Computing