Yoshua Bengio

TL;DR: In this article, the authors show that unexpected event signals predict subsequent changes in responses to expected and unexpected stimuli in individual neurons and distal apical dendrites that are tracked over a period of days.

TL;DR: This paper showed that batch normalization can lead to faster convergence of the training criterion but doesn't seem to improve the generalization performance on both language modelling and speech recognition tasks. But they also showed that applying batch normalisation to the hidden-to-hidden transitions of RNNs doesn't help the training procedure.

TL;DR: Wang et al. as mentioned in this paper proposed a framework by using the recurrent neural network (RNN) as both a discriminative model for recognizing Chinese characters and a generator model for drawing (generating) Chinese characters.

TL;DR: This work proposes a novel parameter initialization strategy that avoids explosion/vanishment of information across layers for weight normalized networks with and without residual connections and shows that the proposed initialization outperforms existing initialization methods in terms of generalization performance, robustness to hyper-parameter values and variance between seeds.

TL;DR: Sub-trajectory balance as mentioned in this paper is a GFlowNet training objective that can learn from partial action subsequences of varying lengths, which accelerates sampler convergence in previously studied and new environments and enables training GFlowNets in environments with longer action sequences and sparser reward landscapes than what was possible before.