Huaixiu Zheng

TL;DR: The authors presented LaMDA: Language Models for Dialog Applications, a family of Transformer-based neural language models specialized for dialog, which have up to 137B parameters and are pre-trained on 1.56T words of public dialog data and web text and demonstrate that fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements towards the two key challenges of safety and factual grounding.

TL;DR: UL2 achieves SOTA performance on 50 well-established supervised NLP tasks ranging from language generation, language understanding, text classification, question answering, commonsense reasoning, long text reasoning, structured knowledge grounding and information retrieval.

TL;DR: Through extensive empirical experiments, it is demonstrated that HyperPrompt can achieve superior performances over strong T5 multi-task learning baselines and parameter-efficient adapter variants including Prompt-Tuning and HyperFormer++ on Natural Language Understanding benchmarks of GLUE and SuperGLUE across many model sizes.

TL;DR: By scaling the model up to 20B parameters, this paper achieves SOTA performance on 50 well-established supervised NLP tasks ranging from language generation, language understanding, text classification, question answering, commonsense reasoning, long text reasoning, structured knowledge grounding and information retrieval.

TL;DR: U-PaLM outperforms PaLM on many few-shot setups, i.e., English NLP tasks, reasoning tasks with chain-of-thought, multilingual tasks, MMLU and challenging BIG-Bench tasks, and is able to substantially improve the scaling properties of large language models on downstream metrics.