Université de Montréal

About: Université de Montréal is a education organization based out in Montreal, Quebec, Canada. It is known for research contribution in the topics: Population & Context (language use). The organization has 45641 authors who have published 100476 publications receiving 4004007 citations. The organization is also known as: University of Montreal & UdeM.

A closer look at memorization in deep networks

Abstract: We examine the role of memorization in deep learning, drawing connections to capacity, generalization, and adversarial robustness. While deep networks are capable of memorizing noise data, our results suggest that they tend to prioritize learning simple patterns first. In our experiments, we expose qualitative differences in gradient-based optimization of deep neural networks (DNNs) on noise vs. real data. We also demonstrate that for appropriately tuned explicit regularization (e.g., dropout) we can degrade DNN training performance on noise datasets without compromising generalization on real data. Our analysis suggests that the notions of effective capacity which are dataset independent are unlikely to explain the generalization performance of deep networks when trained with gradient based methods because training data itself plays an important role in determining the degree of memorization.

Finite spectrum assignment problem for systems with delays

Abstract: In this paper linear systems with delays in state and/or control variables are considered. The objective is to design a feedback law which yields a finite spectrum of the closed-loop system, located at an arbitrarily preassigned set of n points in the complex plane. It is shown that in case of systems with delays in control only the problem is solvable if and only if some function space controllability criterion is met. The solution is then easily obtainable by standard spectrum assignment methods, while the resulting feedback law involves integrals over the past control. In case of delays in state variables it is shown that a technique based on the finite Laplace transform, related to a recent work on function space controllability, leads to a constructive design procedure. The resulting feedback consists of proportional and (finite interval) integral terms over present and past values of state variables. Some indications on how to combine these results in case of systems including both state and control delays are given. Sensitivity of the design to parameter variations is briefly analyzed.

Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering

Abstract: Several unsupervised learning algorithms based on an eigendecomposition provide either an embedding or a clustering only for given training points, with no straightforward extension for out-of-sample examples short of recomputing eigenvectors. This paper provides a unified framework for extending Local Linear Embedding (LLE), Isomap, Laplacian Eigenmaps, Multi-Dimensional Scaling (for dimensionality reduction) as well as for Spectral Clustering. This framework is based on seeing these algorithms as learning eigenfunctions of a data-dependent kernel. Numerical experiments show that the generalizations performed have a level of error comparable to the variability of the embedding algorithms due to the choice of training data.

Table of nonprotein respiratory quotient: an update.

Abstract: The purpose of this paper is to point out some limits and inconsistencies in the table of nonprotein respiratory quotient that is universally used. This table, developed by Lusk in 1924, was derived from biochemical and physical data that are now outdated. A new table of nonprotein respiratory quotient, consistent with modern chemical and physical data, is proposed. The revised table is based on (a) the average composition of human triacylglycerol stores, (b) energy potential of fatty acids and glucose, and (c) the volumes occupied by one mole of oxygen or carbon dioxide (which are not ideal gases) under STPD conditions.