Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Recent work has demonstrated that deep neural networks are vulnerable to adversarial examples---inputs that are almost indistinguishable from natural data and yet classified incorrectly by the network. In fact, some of the latest findings suggest that the existence of adversarial attacks may be an inherent weakness of deep learning models. To address this problem, we study the adversarial robustness of neural networks through the lens of robust optimization. This approach provides us with a broad and unifying view on much of the prior work on this topic. Its principled nature also enables us to identify methods for both training and attacking neural networks that are reliable and, in a certain sense, universal. In particular, they specify a concrete security guarantee that would protect against any adversary. These methods let us train networks with significantly improved resistance to a wide range of adversarial attacks. They also suggest the notion of security against a first-order adversary as a natural and broad security guarantee. We believe that robustness against such well-defined classes of adversaries is an important stepping stone towards fully resistant deep learning models. Code and pre-trained models are available at this https URL and this https URL.

Towards Deep Learning Models Resistant to Adversarial Attacks

From the Publisher: 
The accessible presentation of this book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods. A CD-ROM with every copy of the text contains source code for programming practice, color images, and illustrative movies. Comprehensive and up-to-date, this book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.

Computer vision : a modern approach = 计算机视觉 : 一种现代的方法

Information Theory and Reliable Communication

The broadcast nature of the wireless medium makes the communication over this medium vulnerable to eavesdropping. This paper considers the problem of secret communication between two nodes, over a fading wireless medium, in the presence of a passive eavesdropper. The assumption used is that the transmitter and its helpers (amplifying relays) have more antennas than the eavesdropper. The transmitter ensures secrecy of communication by utilizing some of the available power to produce 'artificial noise', such that only the eavesdropper's channel is degraded. Two scenarios are considered, one where the transmitter has multiple transmit antennas, and the other where amplifying relays simulate the effect of multiple antennas. The channel state information (CSI) is assumed to be publicly known, and hence, the secrecy of communication is independent of the secrecy of CSI.

Guaranteeing Secrecy using Artificial Noise

This paper considers the transmission of confidential data over wireless channels. Based on an information-theoretic formulation of the problem, in which two legitimates partners communicate over a quasi-static fading channel and an eavesdropper observes their transmissions through a second independent quasi-static fading channel, the important role of fading is characterized in terms of average secure communication rates and outage probability. Based on the insights from this analysis, a practical secure communication protocol is developed, which uses a four-step procedure to ensure wireless information-theoretic security: (i) common randomness via opportunistic transmission, (ii) message reconciliation, (iii) common key generation via privacy amplification, and (iv) message protection with a secret key. A reconciliation procedure based on multilevel coding and optimized low-density parity-check (LDPC) codes is introduced, which allows to achieve communication rates close to the fundamental security limits in several relevant instances. Finally, a set of metrics for assessing average secure key generation rates is established, and it is shown that the protocol is effective in secure key renewal-even in the presence of imperfect channel state information.

/pdf/wireless-information-theoretic-security-2nbzpb2s0k.pdf

Wireless Information-Theoretic Security

We consider the transmission of confidential data over wireless channels with multiple communicating parties. Based on an information-theoretic problem formulation in which two legitimate partners communicate over a quasi-static fading channel and an eavesdropper observes their transmissions through another independent quasi-static fading channel, we define the secrecy capacity in terms of outage probability and provide a complete characterization of the maximum transmission rate at which the eavesdropper is unable to decode any information. In sharp contrast with known results for Gaussian wiretap channels (without feedback), our contribution shows that in the presence of fading information-theoretic security is achievable even when the eavesdropper has a better average signal-to-noise ratio (SNR) than the legitimate receiver ? fading thus turns out to be a friend and not a foe.

Secrecy Capacity of Wireless Channels

The generalization error of deep neural networks via their classification margin is studied in this paper. Our approach is based on the Jacobian matrix of a deep neural network and can be applied to networks with arbitrary nonlinearities and pooling layers, and to networks with different architectures such as feed forward networks and residual networks. Our analysis leads to the conclusion that a bounded spectral norm of the network's Jacobian matrix in the neighbourhood of the training samples is crucial for a deep neural network of arbitrary depth and width to generalize well. This is a significant improvement over the current bounds in the literature, which imply that the generalization error grows with either the width or the depth of the network. Moreover, it shows that the recently proposed batch normalization and weight normalization reparametrizations enjoy good generalization properties, and leads to a novel network regularizer based on the network's Jacobian matrix. The analysis is supported with experimental results on the MNIST, CIFAR-10, LaRED, and ImageNet datasets.

Robust Large Margin Deep Neural Networks

In this paper, we investigate the linear precoding and power allocation policies that maximize the mutual information for general multiple-input-multiple-output (MIMO) Gaussian channels with arbitrary input distributions, by capitalizing on the relationship between mutual information and minimum mean-square error (MMSE). The optimal linear precoder satisfies a fixed-point equation as a function of the channel and the input constellation. For non-Gaussian inputs, a nondiagonal precoding matrix in general increases the information transmission rate, even for parallel noninteracting channels. Whenever precoding is precluded, the optimal power allocation policy also satisfies a fixed-point equation; we put forth a generalization of the mercury/waterfilling algorithm, previously proposed for parallel noninterfering channels, in which the mercury level accounts not only for the non-Gaussian input distributions, but also for the interference among inputs.

MIMO Gaussian Channels With Arbitrary Inputs: Optimal Precoding and Power Allocation

This paper investigates the transmission of Frequency Division Multiplexed (FDM) signals, where carrier orthogonality is intentionally violated in order to increase bandwidth efficiency. In analogy to conventional OFDM, signal generation relies on an Inverse Fractional Fourier Transform (IFRFT) that can be implemented with O(N log2 N) algorithmic complexity. Optimal Maximum Likelihood (ML) detection is overly complex due to the presence of substantial Intercarrier Interference (ICI). Consequently, we investigate an alternative detection mechanism based on the Generalized Sphere Decoding (GSD) algorithm. We examine the bandwidth efficiency and the error performance in Additive White Gaussian Noise (AWGN), for various FDM signal parameters. In particular, we show that it is possible to detect optimally and efficiently FDM signals, with 25% bandwidth gain with respect to analogous OFDM signals. This indicates that the transmission of spectrally efficient non orthogonal FDM signals is tangible.

Miguel R. D. Rodrigues

Papers

Wireless Information-Theoretic Security

Secrecy Capacity of Wireless Channels

Robust Large Margin Deep Neural Networks

MIMO Gaussian Channels With Arbitrary Inputs: Optimal Precoding and Power Allocation

Spectrally Efficient FDM Signals: Bandwidth Gain at the Expense of Receiver Complexity