Y
Yichun Yin
Researcher at Huawei
Publications - 38
Citations - 1790
Yichun Yin is an academic researcher from Huawei. The author has contributed to research in topics: Language model & Computer science. The author has an hindex of 9, co-authored 32 publications receiving 888 citations. Previous affiliations of Yichun Yin include Microsoft & Huazhong University of Science and Technology.
Papers
More filters
Proceedings ArticleDOI
TinyBERT: Distilling BERT for Natural Language Understanding
TL;DR: TinyBERT as discussed by the authors proposes a two-stage learning framework for TinyBERT, which performs transformer distillation at both the pre-training and task-specific learning stages to capture the general-domain as well as the task specific knowledge in BERT.
Posted Content
TinyBERT: Distilling BERT for Natural Language Understanding
TL;DR: A novel Transformer distillation method that is specially designed for knowledge distillation (KD) of the Transformer-based models is proposed and, by leveraging this new KD method, the plenty of knowledge encoded in a large “teacher” BERT can be effectively transferred to a small “student” TinyBERT.
Proceedings Article
Unsupervised word and dependency path embeddings for aspect term extraction
TL;DR: A novel approach to aspect term extraction based on unsupervised learning of distributed representations of words and dependency paths, where the multi-hop dependency paths are treated as a sequence of grammatical relations and modeled by a recurrent neural network.
Proceedings ArticleDOI
TernaryBERT: Distillation-aware Ultra-low Bit BERT
TL;DR: This work proposes TernaryBERT, which ternarizes the weights in a fine-tuned BERT model, and leverages the knowledge distillation technique in the training process to reduce the accuracy degradation caused by the lower capacity of low bits.
Proceedings ArticleDOI
Document-Level Multi-Aspect Sentiment Classification as Machine Comprehension
TL;DR: This paper model the document-level multi-aspect sentiment classification task as a machine comprehension problem where pseudo question-answer pairs are constructed by a small number of aspect-related keywords and aspect ratings.