Z
Zihang Dai
Researcher at Google
Publications - 57
Citations - 14783
Zihang Dai is an academic researcher from Google. The author has contributed to research in topics: Language model & Computer science. The author has an hindex of 28, co-authored 52 publications receiving 9340 citations. Previous affiliations of Zihang Dai include Baidu & Carnegie Mellon University.
Papers
More filters
Proceedings Article
CoAtNet: Marrying Convolution and Attention for All Data Sizes
TL;DR: CoAtNets as mentioned in this paper is a family of hybrid models built from two key insights: depthwise convolution and self-attention can be unified via simple relative attention and vertically stacking convolution layers and attention layers in a principled way.
Posted Content
Re-examination of the Role of Latent Variables in Sequence Modeling
TL;DR: Over a diverse set of sequential data, including human speech, MIDI music, handwriting trajectory and frame-permuted speech, the results show that stochastic recurrent models fail to exhibit any practical advantage despite the claimed theoretical superiority.
Journal ArticleDOI
Fast and Simple Mixture of Softmaxes with BPE and Hybrid-LightRNN for Language Generation
TL;DR: This paper proposed improved word coding schemes, which could effectively reduce the vocabulary size and hence relieve the memory and computation burden of the mixture of softmaxes (MoS) model, and achieved the state-of-the-art result on WMT 2014 English to German task.
Posted Content
Fast and Simple Mixture of Softmaxes with BPE and Hybrid-LightRNN for Language Generation
TL;DR: Both BPE and the proposed Hybrid-LightRNN lead to improved encoding mechanisms that can halve the time and memory consumption of MoS without performance losses, and are shown to be effective at addressing the expressiveness limitation of Softmax-based models.