Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks
Citations
3,114 citations
Cites methods from "Convolutional, Long Short-Term Memo..."
...accuracy rates higher than previous classification techniques. In short, DL is transforming computational processing of complex data in many domains such as vision [24], [37], speech recognition [15], [32], [33], language processing [13], financial fraud detection [23], and recently malware detection [14]. This increasing use of deep learning is creating incentives for adversaries to manipulate DNNs to ...
[...]
2,279 citations
Cites background or methods from "Convolutional, Long Short-Term Memo..."
...The state-of-the-art model on this dataset is a CLDNN-HMM system that was described in [22]....
[...]
...Data augmentation was performed using a room simulator, adding different types of noise and reverberations; the noise sources were obtained from YouTube and environmental recordings of daily events [22]....
[...]
...Additionally, the LAS model does not use convolutional filters which have been reported to yield 5-7% WER relative improvement [22]....
[...]
...This increased the amount of audio data by 20 times with a SNR between 5dB and 30dB [22]....
[...]
1,973 citations
Cites background or methods from "Convolutional, Long Short-Term Memo..."
...Additionally, we dynamically augment the dataset by adding unique noise every epoch with an SNR between 0dB and 30dB, just as in (Hannun et al., 2014a; Sainath et al., 2015)....
[...]
..., 2014) and work well with convolutional layers for the feature extraction (Sainath et al., 2015)....
[...]
1,896 citations
Cites background from "Convolutional, Long Short-Term Memo..."
...Nevertheless, such an approach is not strictly part of the architecture, and in the time series domain, we can see some examples of CNNs where not every convolutional layer is followed by a pooling operation [16]....
[...]
...The combination of CNNs and LSTMs in a unified framework has already offered state-of-the-art results in the speech recognition domain, where modelling temporal information is required [16]....
[...]
...Further improvements in terms of time series classification have been obtained in the speech recognition domain, by combining convolutional and recurrent layers in a unified deep framework, which contains either standard recurrent units [29] or LSTM cells [16]....
[...]
1,789 citations
References
72,897 citations
16,717 citations
9,500 citations
"Convolutional, Long Short-Term Memo..." refers methods in this paper
...The weights for all CNN and DNN layers are initialized using the Glorot-Bengio strategy described in [18]....
[...]
9,091 citations
3,549 citations
Additional excerpts
...One may argue that if the LSTMs are better initialized, such that better temporal modeling can be performed, are CNNs really necessary? Our initial experiments with LSTMs use Gaussian random weight initialization, which produces eigenvalues of the initial recurrent network which are close to zero, thus increasing the chances for vanishing gradients [19]....
[...]