Yoshua Bengio

TL;DR: In this article, the variance of the cross-validation estimate of the generalization error was investigated, and two new estimators of this variance were proposed, which were shown to perform well relative to the statistics considered in Dietterich (1998).

TL;DR: Experiments reported here evaluate the use of clipping gradients, spanning longer time ranges with leaky integration, advanced momentum techniques, using more powerful output probability models, and encouraging sparser gradients to help symmetry breaking and credit assignment.

TL;DR: This paper proposes to examine some of the challenges of scaling deep learning algorithms to much larger models and datasets, reducing optimization difficulties due to ill-conditioning or local minima, designing more efficient and powerful inference and sampling procedures, and learning to disentangle the factors of variation underlying the observed data.

TL;DR: This paper presents the techniques used for the University of Montréal's team submissions to the 2013 Emotion Recognition in the Wild Challenge, a challenge to classify the emotions expressed by the primary human subject in short video clips extracted from feature length movies.

TL;DR: Manifold Mixup as discussed by the authors leverages semantic interpolations as additional training signal, obtaining neural networks with smoother decision boundaries at multiple levels of representation, as a result, neural networks trained with Manifold mixup learn class-representations with fewer directions of variance.