Z
Zhewei Yao
Researcher at University of California, Berkeley
Publications - 73
Citations - 2774
Zhewei Yao is an academic researcher from University of California, Berkeley. The author has contributed to research in topics: Quantization (signal processing) & Computer science. The author has an hindex of 21, co-authored 61 publications receiving 1384 citations. Previous affiliations of Zhewei Yao include Shanghai Jiao Tong University.
Papers
More filters
Journal ArticleDOI
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
Sheng Shen,Zhen Dong,Jiayu Ye,Linjian Ma,Zhewei Yao,Amir Gholami,Michael W. Mahoney,Kurt Keutzer +7 more
TL;DR: This work performs an extensive analysis of fine-tuned BERT models using second order Hessian information, and uses the results to propose a novel method for quantizing BERT Models to ultra low precision, which is based on a new group-wise quantization scheme and Hessian-based mix-precision method.
Proceedings ArticleDOI
HAWQ: Hessian AWare Quantization of Neural Networks With Mixed-Precision
TL;DR: Hessian AWare Quantization (HAWQ), a novel second-order quantization method that allows for the automatic selection of the relative quantization precision of each layer, based on the layer's Hessian spectrum, is introduced.
Proceedings ArticleDOI
ZeroQ: A Novel Zero Shot Quantization Framework
TL;DR: THE AUTHORS' enables mixed-precision quantization without any access to the training or validation data, and it can finish the entire quantization process in less than 30s, which is very low computational overhead.
Posted Content
PyHessian: Neural Networks Through the Lens of the Hessian
TL;DR: PYHESSIAN, a new scalable framework that enables fast computation of Hessian (i.e., second-order derivative) information for deep neural networks, shows new finer-scale insights, demonstrating that while conventional wisdom is sometimes validated, in other cases it is simply incorrect.
Posted Content
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
TL;DR: AdaHessian is introduced, a second order stochastic optimization algorithm which dynamically incorporates the curvature of the loss function via ADAptive estimates of the Hessian, and it exhibits robustness towards its hyperparameters.