https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Learning good image priors is of utmost importance for the study of vision, computer vision and image processing applications. Learning priors and optimizing over whole images can lead to tremendous computational challenges. In contrast, when we work with small image patches, it is possible to learn priors and perform patch restoration very efficiently. This raises three questions - do priors that give high likelihood to the data also lead to good performance in restoration? Can we use such patch based priors to restore a full image? Can we learn better patch priors? In this work we answer these questions. We compare the likelihood of several patch models and show that priors that give high likelihood to data perform better in patch restoration. Motivated by this result, we propose a generic framework which allows for whole image restoration using any patch based prior for which a MAP (or approximate MAP) estimate can be calculated. We show how to derive an appropriate cost function, how to optimize it and how to use it to restore whole images. Finally, we present a generic, surprisingly simple Gaussian Mixture prior, learned from a set of natural images. When used with the proposed framework, this Gaussian Mixture Model outperforms all other generic prior methods for image denoising, deblurring and inpainting.

/pdf/from-learning-models-of-natural-image-patches-to-whole-image-25iz5mt23l.pdf

From learning models of natural image patches to whole image restoration

In recent years, deep learning algorithms have become increasingly more prominent for their unparalleled ability to automatically learn discriminant features from large amounts of data. However, within the field of electromyography-based gesture recognition, deep learning algorithms are seldom employed as they require an unreasonable amount of effort from a single person, to generate tens of thousands of examples. This paper’s hypothesis is that general, informative features can be learned from the large amounts of data generated by aggregating the signals of multiple users, thus reducing the recording burden while enhancing gesture recognition. Consequently, this paper proposes applying transfer learning on aggregated data from multiple users while leveraging the capacity of deep learning algorithms to learn discriminant features from large datasets. Two datasets comprised 19 and 17 able-bodied participants, respectively (the first one is employed for pre-training), were recorded for this work, using the Myo armband. A third Myo armband dataset was taken from the NinaPro database and is comprised ten able-bodied participants. Three different deep learning networks employing three different modalities as input (raw EMG, spectrograms, and continuous wavelet transform (CWT)) are tested on the second and third dataset. The proposed transfer learning scheme is shown to systematically and significantly enhance the performance for all three networks on the two datasets, achieving an offline accuracy of 98.31% for 7 gestures over 17 participants for the CWT-based ConvNet and 68.98% for 18 gestures over 10 participants for the raw EMG-based ConvNet. Finally, a use-case study employing eight able-bodied participants suggests that real-time feedback allows users to adapt their muscle activation strategy which reduces the degradation in accuracy normally experienced over time.

/pdf/deep-learning-for-electromyographic-hand-gesture-signal-35yh5gi4gl.pdf

Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning

Due to their numerous advantages, communications over multicarrier schemes constitute an appealing approach for broadband wireless systems. Especially, the strong penetration of orthogonal frequency division multiplexing (OFDM) into the communications standards has triggered heavy investigation on multicarrier systems, leading to re-consideration of different approaches as an alternative to OFDM. The goal of the present survey is not only to provide a unified review of waveform design options for multicarrier schemes, but also to pave the way for the evolution of the multicarrier schemes from the current state of the art to future technologies. In particular, a generalized framework on multicarrier schemes is presented, based on what to transmit, i.e., symbols, how to transmit, i.e., filters, and where/when to transmit, i.e., lattice. Capitalizing on this framework, different variations of orthogonal, bi-orthogonal, and non-orthogonal multicarrier schemes are discussed. In addition, filter designs for various multicarrier systems are reviewed considering four different design perspectives: energy concentration, rapid decay, spectrum nulling, and channel/hardware characteristics. Subsequently, evaluation tools which may be used to compare different filters in multicarrier schemes are studied. Finally, multicarrier schemes are evaluated from the perspective of practical implementation aspects, such as lattice adaptation, equalization, synchronization, multiple antennas, and hardware impairments.

A Survey on Multicarrier Communications: Prototype Filters, Lattice Structures, and Implementation Aspects

We propose a deep learning approach to free-hand sketch recognition that achieves state-of-the-art performance, significantly surpassing that of humans. Our superior performance is a result of modelling and exploiting the unique characteristics of free-hand sketches, i.e., consisting of an ordered set of strokes but lacking visual cues such as colour and texture, being highly iconic and abstract, and exhibiting extremely large appearance variations due to different levels of abstraction and deformation. Specifically, our deep neural network, termed Sketch-a-Net has the following novel components: (i) we propose a network architecture designed for sketch rather than natural photo statistics. (ii) Two novel data augmentation strategies are developed which exploit the unique sketch-domain properties to modify and synthesise sketch training data at multiple abstraction levels. Based on this idea we are able to both significantly increase the volume and diversity of sketches for training, and address the challenge of varying levels of sketching detail commonplace in free-hand sketches. (iii) We explore different network ensemble fusion strategies, including a re-purposed joint Bayesian scheme, to further improve recognition performance. We show that state-of-the-art deep networks specifically engineered for photos of natural objects fail to perform well on sketch recognition, regardless whether they are trained using photos or sketches. Furthermore, through visualising the learned filters, we offer useful insights in to where the superior performance of our network comes from.

/pdf/sketch-a-net-a-deep-neural-network-that-beats-humans-ltrcpidhj3.pdf

Sketch-a-Net: A Deep Neural Network that Beats Humans

Hitherto communication theory was based on two alternative methods of signal analysis. One is the description of the signal as a function of time; the other is Fourier analysis. Both are idealizations, as the first method operates with sharply defined instants of time, the second with infinite wave-trains of rigorously defined frequencies. But our everyday experiences?especially our auditory sensations?insist on a description in terms of both time and frequency. In the present paper this point of view is developed in quantitative language. Signals are represented in two dimensions, with time and frequency as co-ordinates. Such two-dimensional representations can be called ?information diagrams,? as areas in them are proportional to the number of independent data which they can convey. This is a consequence of the fact that the frequency of a signal which is not of infinite duration can be defined only with a certain inaccuracy, which is inversely proportional to the duration, and vice versa. This ?uncertainty relation? suggests a new method of description, intermediate between the two extremes of time analysis and spectral analysis. There are certain ?elementary signals? which occupy the smallest possible area in the information diagram. They are harmonic oscillations modulated by a ?probability pulse.? Each elementary signal can be considered as conveying exactly one datum, or one ?quantum of information.? Any signal can be expanded in terms of these by a process which includes time analysis and Fourier analysis as extreme cases. These new methods of analysis, which involve some of the mathematical apparatus of quantum theory, are illustrated by application to some problems of transmission theory, such as direct generation of single sidebands, signals transmitted in minimum time through limited frequency channels, frequency modulation and time-division multiplex telephony.

D. Gabor

Papers

Theory of communication. Part 1: The analysis of information