Rotational Unit of Memory: A Novel Representation Unit for RNNs with Scalable Applications
TLDR
This work derives a phase-coded representation of the memory state, Rotational Unit of Memory (RUM), that unifies the concepts of unitary learning and associative memory and shows experimentally that RNNs based on RUMs can solve basic sequential tasks such as memory copying and memory recall much better than LSTMs/GRUs.Abstract:
Stacking long short-term memory (LSTM) cells or gated recurrent units (GRUs) as part of a recurrent neural network (RNN) has become a standard approach to solving a number of tasks ranging from lan...read more
Citations
More filters
Journal ArticleDOI
Review: Deep Learning in Electron Microscopy
TL;DR: In this paper, a review of deep learning in electron microscopy is presented, with a focus on hardware and software needed to get started with deep learning and interface with electron microscopes.
Posted Content
A Divide-and-Conquer Approach to the Summarization of Long Documents
TL;DR: This work exploits the discourse structure of the document and uses sentence similarity to split the problem into an ensemble of smaller summarization problems, which can decompose the problem of long document summarization into smaller and simpler problems, reducing computational complexity and creating more training examples.
Journal ArticleDOI
A Divide-and-Conquer Approach to the Summarization of Long Documents
TL;DR: The authors proposed a divide-and-conquer method for the neural summarization of long documents, which exploits the discourse structure of the document and uses sentence similarity to split the problem into an ensemble of smaller summarization problems.
Journal ArticleDOI
Deep reinforcement and transfer learning for abstractive text summarization: A review
TL;DR: Automatic Text Summarization (ATS) is an important area in NLP as mentioned in this paper with the goal of shortening a long text into a more compact version by conveying the most important points in a readable form.
Journal ArticleDOI
Deep reinforcement and transfer learning for abstractive text summarization: A review
TL;DR: Automatic Text Summarization (ATS) is an important area in Natural Language Processing (NLP) with the goal of shortening a long text into a more compact version by conveying the most important points in a readable form as mentioned in this paper.
References
More filters
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Journal ArticleDOI
Long short-term memory
TL;DR: A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
Journal Article
Dropout: a simple way to prevent neural networks from overfitting
TL;DR: It is shown that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
Proceedings Article
Neural Machine Translation by Jointly Learning to Align and Translate
TL;DR: It is conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and it is proposed to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly.
Proceedings ArticleDOI
Learning Phrase Representations using RNN Encoder--Decoder for Statistical Machine Translation
Kyunghyun Cho,Bart van Merriënboer,Caglar Gulcehre,Dzmitry Bahdanau,Fethi Bougares,Holger Schwenk,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio +8 more
TL;DR: In this paper, the encoder and decoder of the RNN Encoder-Decoder model are jointly trained to maximize the conditional probability of a target sequence given a source sequence.