T
Tae Hwan Jung
Publications - 2
Tae Hwan Jung is an academic researcher. The author has contributed to research in topics: Key (cryptography) & Product (mathematics). The author has co-authored 2 publications.
Papers
More filters
Posted Content
Large Product Key Memory for Pretrained Language Models
Gyuwan Kim,Tae Hwan Jung +1 more
TL;DR: This article proposed a new memory usage metric, and careful observation using this metric reveals that most memory slots remain outdated during the training of PKM-augmented models, and propose simple but effective solutions: (1) initialization from the model weights pretrained without memory and (2) augmenting PKM by addition rather than replacing a feed-forward network.
Proceedings ArticleDOI
Large Product Key Memory for Pretrained Language Models
Gyuwan Kim,Tae Hwan Jung +1 more
TL;DR: A new memory usage metric is defined, and careful observation reveals that most memory slots remain outdated during the training of PKM-augmented models, enhancing memory utilization and downstream performance.