Hiroyuki Shindo

TL;DR: New pretrained contextualized representations of words and entities based on the bidirectional transformer, and an entity-aware self-attention mechanism that considers the types of tokens (words or entities) when computing attention scores are proposed.

TL;DR: In this paper, the authors proposed a novel embedding method for NED, which jointly maps words and entities into the same continuous vector space by using skip-gram model and anchor context model, and achieved state-of-the-art accuracy of 93.1% on the standard CoNLL dataset.

TL;DR: A novel embedding method specifically designed for NED that jointly maps words and entities into the same continuous vector space and extends the skip-gram model by using two models.

TL;DR: This paper restores interpretability to adversarial training methods by restricting the directions of perturbations toward the existing words in the input embedding space and can straightforwardly reconstruct each input with perturbATIONS to an actual text by considering the perturbation to be the replacement of words in a sentence while maintaining or even improving the task performance.

TL;DR: This article proposed new pretrained contextualized representations of words and entities based on the bidirectional transformer, which treats words and entity in a given text as independent tokens, and outputs contextualised representations of them.