Kevin Robinson

TL;DR: A 540-billion parameter, densely activated, Transformer language model, which is called PaLM achieves breakthrough performance, outperforming the state-of-the-art on a suite of multi-step reasoning tasks, and outperforming average human performance on the recently released BIG-bench benchmark.

TL;DR: The PaLM 2 model as mentioned in this paper is a Transformer-based model trained using a mixture of objectives, which has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM.

TL;DR: The authors show that a temporal shift between evaluation data and pretraining data leads to performance degradation, which is not overcome by finetuning, and explore the effect of quality and toxicity filters, showing a tradeoff between performance on standard benchmarks and risk of toxic generations.