A
Abhishek Rao
Publications - 2
Citations - 1805
Abhishek Rao is an academic researcher. The author has contributed to research in topics: Computer science. The author has an hindex of 2, co-authored 2 publications receiving 1805 citations.
Papers
More filters
Journal Article
PaLM: Scaling Language Modeling with Pathways
Aakanksha Chowdhery,Sharan Narang,Jacob Devlin,Maarten Bosma,Gaurav Mishra,Adam Roberts,Paul Barham,Hyung Won Chung,Charles Sutton,Sebastian Gehrmann,Parker Schuh,Kensen Shi,Sasha Tsvyashchenko,Joshua Maynez,Abhishek Rao,Parker Barnes,Yi Tay,Noam Shazeer,Velu Prabhakaran,Emily Reif,Nan Du,B. C. Hutchinson,Reiner Pope,James Bradbury,Jacob Austin,Michael Isard,Guy Gur-Ari,Peng Yin,Toju Duke,Anselm Levskaya,Sanjay Ghemawat,Sunipa Dev,Henryk Michalewski,Xavier Garcia,Vedant Misra,Kevin Robinson,L Fedus,Denny Zhou,Daphne Ippolito,David Luan,Hyeontaek Lim,Barret Zoph,Alexander Spiridonov,Ryan Sepassi,David Dohan,Shivani Agrawal,Mark Omernick,Andrew M. Dai,Thanumalayan Sankaranarayana Pillai,Marie Pellat,Aitor Lewkowycz,Erica Oliveira Moreira,Rewon Child,Oleksandr Polozov,Katherine Lee,Zong Tuan Zhou,Xuezhi Wang,Brennan Saeta,Mark Díaz,Orhan Firat,M. Catasta,Jason Loh Seong Wei,Kathleen S. Meier-Hellstern,Douglas Eck,Jeffrey Dean,Slav Petrov,Noah Fiedel +66 more
TL;DR: A 540-billion parameter, densely activated, Transformer language model, which is called PaLM achieves breakthrough performance, outperforming the state-of-the-art on a suite of multi-step reasoning tasks, and outperforming average human performance on the recently released BIG-bench benchmark.
Journal Article
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava,Abhinav Rastogi,Abhishek Rao,Abu Awal Md Shoeb,Abubakar Abid,Adam Fisch,Adam R. Brown,Adam Santoro,Aditya Gupta,Adrià Garriga-Alonso,Agnieszka Kluska,Aitor Lewkowycz,Akshat Agarwal,Alethea. Power,Alex Ray,Alex Warstadt,Alexander W. Kocurek,Ali Safaya,Ali Tazarv,Alice Xiang,Alicia Parrish,Allen Nie,Aman Hussain,Amanda Askell,Amanda Dsouza,Ameet Annasaheb Rahane,Anantharaman S. Iyer,Anders Andreassen,Andrea Santilli,Andreas Stuhlmuller,Andrew M. Dai,Andrew D. La,Andrew K. Lampinen,Andy Zou,Angela Jiang,Angelica Chen,Anh Vuong,Animesh Gupta,Anna Gottardi,Antonio Norelli,Anushree Venkatesh,Arash Gholamidavoodi,Arfa Tabassum,Arul Menezes,Arun Kirubarajan,Asher Mullokandov,Ashish Sabharwal,Austin Herrick,Avia Efrat,Aykut Erdem,Ayla Karakacs,Bridget R. Roberts,Bao Sheng Loe,Barret Zoph,Bartlomiej Bojanowski,Batuhan Ozyurt,Behnam Hedayatnia,Behnam Neyshabur,Benjamin Inden,Benno Stein,Berk Ekmekci,Bill Yuchen Lin,Blake Howald,Cameron Diao,Cameron Dour,Catherine Stinson,Cedrick Argueta,C'esar Ferri Ram'irez,Chandan Singh,Charles Rathkopf,Chenlin Meng,Chitta Baral,Chiyu Wu,Chris Callison-Burch,Chris Waites,Christian Voigt,Christopher D. Manning,C. W. Potts,Cindy Tatiana Ramirez,Clara E. Rivera,Clemencia Siro,Colin Raffel,Courtney Ashcraft,Cristina Gârbacea,Damien Sileo,Daniel H Garrette,Dan Hendrycks,D. Kilman,Dan Roth,Daniel Freeman,Daniel Khashabi,Daniel Levy,Daniel Gonz'alez,Danny Hernandez,Danqi Chen,Daphne Ippolito,Dar Gilboa,David Dohan,D. Drakard,David A. Jurgens,Debajyoti Datta,Deep Ganguli,Denis Emelin,Denis Kleyko,Deniz Yuret,Derek Chen,Derek Tam,Dieuwke Hupkes,Diganta Misra,Dilyar Buzan,Dimitri Coelho Mollo,Diyi Yang,Dongho Lee,E P Shutova,Ekin D. Cubuk,Elad Segal,Eleanor Hagerman,Elizabeth A. Barnes,Elizabeth P. Donoway,Ellie Pavlick,Emanuele Rodolà,Emma F C Lam,Eric Chu,Eric Tang,Erkut Erdem,Ernie Chang,Ethan A. Chi,Ethan Dyer,Ethan Jerzak,Ethan Kim,Eunice Engefu Manyasi,Evgenii Zheltonozhskii,Fan Xia,F. Siar,Fernando Mart'inez-Plumed,Francesca Happ'e,Francois Chollet,Frieda Rong,Gaurav Mishra,Genta Indra Winata,Gerard de Melo,G. Kruszewski,Giambattista Parascandolo,Giorgio Mariani,Gloria Wang,Gonzalo Jaimovitch-L'opez,Gregor Betz,Guy Gur-Ari,Hana Galijasevic,Han Sol Kim,Hannah Rashkin,Hanna Hajishirzi,Harsh Mehta,H. Bogar,Henry Shevlin,Hinrich Schuetze,Hiromu Yakura,Hongming Zhang,Hubert Wong,Ian A. S. Ng,Isaac Noble,Julien Jumelet,Jack Geissinger,John Kernion,Jacob Hilton,Jae-Hoon Lee,Jaime F. Fisac,J. Brooker Simon,James Koppel,James Zheng,James Zou,Jan Koco'n,Jana Thompson,Jared Kaplan,Jarema Radom,Jascha Sohl-Dickstein,Jason Phang,Jason Loh Seong Wei,Jason Yosinski,Jekaterina Novikova,Jelle Bosscher,Jennifer Violet Marsh,Jeremy Kim,Jeroen Taal,Jesse Engel,Jesujoba O. Alabi,Jiacheng Xu,Jiaming Song,Jillian Tang,Jane W Waweru,John Burden,John A. Miller,John U. Balis,Jonathan Berant,Jorg Frohberg,Jos Rozen,José Hernández-Orallo,Joseph Boudeman,Josephine R. Jones,Joshua B. Tenenbaum,Joshua Rule,Joyce Hui Ping Chua,Kamil Kanclerz,Karen Livescu,Karl Krauth,Karthik Gopalakrishnan,Katerina Ignatyeva,Katja Markert,Kaustubh Dhole,K Gimpel,Kevin O Omondi,K. Mathewson,Kristen Chiafullo,Ksenia Shkaruta,Kumar Shridhar,Kyle McDonell,Kyle Richardson,Laria Reynolds,Leo Gao,Li Zhang,Liam Dugan,Lianhui Qin,Lidia Contreras-Ochando,Louis-Philippe Morency,Luca Moschella,Luca Lam,Lucy Noble,Ludwig Schmidt,Luheng He,Luis Oliveros Col'on,Luke Metz,Lutfi Kerem cSenel,Maarten Bosma,Maarten Sap,Maartje ter Hoeve,Madotto Andrea,Maheen Saleem Farooqi,Manaal Faruqui,Mantas Mazeika,Marco Baturan,Marco Marelli,Marco Maru,M Quintana,Marie Tolkiehn,Mario Giulianelli,M. Lewis,M. Potthast,Matthew Leavitt,Matthias Hagen,M. Schubert,Medina Baitemirova,M Arnaud,Melvin Andrew McElrath,Michael A Yee,Michael Cohen,Mi-jin Gu,Michael I. Ivanitskiy,Michael Starritt,Michael Strube,Michal Swkedrowski,Michele Bevilacqua,Michihiro Yasunaga,Mihir Kale,Michael D. Cain,Mimee Xu,Mirac M. Suzgun,Monica Tiwari,Mohit Bansal,Moin Aminnaseri,Mor Geva,Mozhdeh Gheini,T. MukundVarma,Nanyun Peng,Nathan Chi,Nayeon Lee,Neta Gur-Ari Krakover,Nicholas Cameron,Nicholas S. Roberts,Nicholas Doiron,Nikita Nangia,Niklas Deckers,Niklas Muennighoff,Nitish Shirish Keskar,Niveditha Iyer,Noah Constant,Noah Fiedel,Nuan Wen,Oliver Zhang,Omar Agha,Omar Elbaghdadi,Omer Levy,Owain Evans,P. A. M. Casares,P. S. Doshi,Pascale Fung,Paul Pu Liang,Paul Adrian Vicol,Pegah Alipoormolabashi,Peiyuan Liao,Percy Liang,Peter W. Chang,Peter Eckersley,Phu Mon Htut,Pi-Bei Hwang,Piotr Miłkowski,P. T. Patil,Pouya Pezeshkpour,Priti Oli,Qiaozhu Mei,Qing Lyu,Qinlang Chen,Rabin Banjade,Rachel E. Rudolph,Raefer Gabriel,Rahel Habacker,Ramon Delgado,Raphael Milliere,Rhythm Garg,Richard Barnes,Rif A. Saurous,Riku Arakawa,Robbe Raymaekers,Robert Frank,Rohan Sikand,Roman Novak,Roman Sitelew,Ronan LeBras,Rosanne Liu,Rowan Jacobs,Rui Zhang,Ruslan Salakhutdinov,Ryan Chi,Ryan Lee,Ryan Stovall,Ryan Teehan,Rylan Yang,Sahib Singh,Saif M. Mohammad,Sajant Anand,Sam Dillavou,Sam Shleifer,Sam Wiseman,Samuel Gruetter,Sam W. Bowman,Samuel S. Schoenholz,Sanghyun Han,Sanjeev Kwatra,Sarah Adler Rous,Sarik Ghazarian,Sayan Ghosh,Sean Casey,Sebastian Bischoff,Sebastian Gehrmann,Sebastian Schuster,Sepideh Sadeghi,Shadi Sameh Hamdan,Sharon Zhou,S. K. Srivastava,Sherry Shi,Shikhar Kumar Singh,Shima Asaadi,Shixiang Gu,Shubh Pachchigar,Shubham Toshniwal,Shyam Upadhyay,Shyamolima Debnath,Siamak Shakeri,Simon Thormeyer,Simone Melzi,Siva Koti Reddy,Sneha Priscilla Makini,Soo-Hwan Lee,Spencer Bradley Torene,Sriharsha Hatwar,Stanislas Dehaene,Stefan Divic,Stefano Ermon,Stella Biderman,Stephanie C. Lin,S. Prasad,Steven Piantadosi,Stuart M. Shieber,Summer Misherghi,Svetlana Kiritchenko,Swaroop Mishra,Tal Linzen,T. Schuster,Tao Li,Tariq Ali,Tatsuo Hashimoto,Teng Wu,Theo Desbordes,Theodore Rothschild,Thomas Ngoc Phan,Tianle Wang,Tiberius Tabulu Nkinyili,Timo Schick,T. N. Kornev,Timothy Telleen-Lawton,T. Tunduny,Tobias Gerstenberg,T.P. Chang,Trishala Neeraj,Tushar Khot,T. Shultz,Vedant Misra,Vera Demberg,Victoria Nyamai,Vikas Raunak,Vinay Ramasesh,Vinay Uday Prabhu,Vishakh Padmakumar,V. Srikumar,William Fedus,William Saunders,William Zhang,W. Vossen,Xiangyuan Ren,Xiaoyu Tong,Xinyi Wu,Xudong Shen,Yadollah Yaghoobzadeh,Yair Lakretz,Yang Song,Yasaman Bahri,Ye Ji Choi,Yichi Yang,Yiding Hao,Yifu Chen,Yonatan Belinkov,Yu Hou,Yuntao Bai,Zachary Seid,Zhao Xin-ran,Zhuoye Zhao,Zi Fu Wang,Zijie J. Wang,Ziyi Wu,Sahib Singh,Uri Shaham +439 more
TL;DR: Evaluation of OpenAI's GPT models, Google-internal dense transformer architectures, and Switch-style sparse transformers on BIG-bench, across model sizes spanning millions to hundreds of billions of parameters finds that model performance and calibration both improve with scale, but are poor in absolute terms.