Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

The second edition of this acclaimed graduate text provides a unified treatment of two methods used in contemporary econometric research, cross section and data panel methods. By focusing on assumptions that can be given behavioral content, the book maintains an appropriate level of rigor while emphasizing intuitive thinking. The analysis covers both linear and nonlinear models, including models with dynamics and/or individual heterogeneity. In addition to general estimation frameworks (particular methods of moments and maximum likelihood), specific linear and nonlinear methods are covered in detail, including probit and logit models and their multivariate, Tobit models, models for count data, censored and missing data schemes, causal (or treatment) effects, and duration analysis. Econometric Analysis of Cross Section and Panel Data was the first graduate econometrics text to focus on microeconomic data structures, allowing assumptions to be separated into population and sampling assumptions. This second edition has been substantially updated and revised. Improvements include a broader class of models for missing data problems; more detailed treatment of cluster problems, an important topic for empirical researchers; expanded discussion of "generalized instrumental variables" (GIV) estimation; new coverage (based on the author's own recent research) of inverse probability weighting; a more complete framework for estimating treatment effects with panel data, and a firmly established link between econometric approaches to nonlinear panel data and the "generalized estimating equation" literature popular in statistics and other fields. New attention is given to explaining when particular econometric methods can be applied; the goal is not only to tell readers what does work, but why certain "obvious" procedures do not. The numerous included exercises, both theoretical and computer-based, allow the reader to extend methods covered in the text and discover new insights.

/pdf/econometric-analysis-of-cross-section-and-panel-data-3jzr8emmtf.pdf

Econometric Analysis of Cross Section and Panel Data

This paper presents specification tests that are applicable after estimating a dynamic model from panel data by the generalized method of moments (GMM), and studies the practical performance of these procedures using both generated and real data. Our GMM estimator optimally exploits all the linear moment restrictions that follow from the assumption of no serial correlation in the errors, in an equation which contains individual effects, lagged dependent variables and no strictly exogenous variables. We propose a test of serial correlation based on the GMM residuals and compare this with Sargan tests of over-identifying restrictions and Hausman specification tests.

http://people.stern.nyu.edu/wgreene/Econometrics/Arellano-Bond.pdf

Some Tests of Specification for Panel Data: Monte Carlo Evidence and an Application to Employment Equations.

This paper presents a parameter covariance matrix estimator which is consistent even when the disturbances of a linear regression model are heteroskedastic. This estimator does not depend on a formal model of the structure of the heteroskedasticity. By comparing the elements of the new estimator to those of the usual covariance estimator, one obtains a direct test for heteroskedasticity, since in the absence of heteroskedasticity, the two estimators will be approximately equal, but will generally diverge otherwise. The test has an appealing least squares interpretation.

A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity

Traditional econometric models assume a constant one-period forecast variance. To generalize this implausible assumption, a new class of stochastic processes called autoregressive conditional heteroscedastic (ARCH) processes are introduced in this paper. These are mean zero, serially uncorrelated processes with nonconstant variances conditional on the past, but constant unconditional variances. For such processes, the recent past gives information about the one-period forecast variance. A regression model is then introduced with disturbances following an ARCH process. Maximum likelihood estimators are described and a simple scoring iteration formulated. Ordinary least squares maintains its optimality properties in this set-up, but maximum likelihood is more efficient. The relative efficiency is calculated and can be infinite. To test whether the disturbances follow an ARCH process, the Lagrange multiplier procedure is employed. The test is based simply on the autocorrelation of the squared OLS residuals. This model is used to estimate the means and variances of inflation in the U.K. The ARCH effect is found to be significant and the estimated variances increase substantially during the chaotic seventies.

Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation

Causal discourse in a game of incomplete information

Background: Several major statin trials have shown increased incident diabetes and/or modest average elevations in HbA1C (e.g. PROVE-IT, JUPITER) with statins, though average effects on glucose are...

Abstract 055: Statins Raise Glucose Preferentially among Men Who are Older and at Greater Metabolic Risk

This paper proposes methods for estimation and inference in multivariate, multi-quantile models. The theory can simultaneously accommodate models with multiple random variables, multiple confidence levels, and multiple lags of the associated quantiles. The proposed framework can be conveniently thought of as a vector autoregressive (VAR) extension to quantile models. We estimate a simple version of the model using market equity returns data to analyse spillovers in the values at risk (VaR) between a market index and financial institutions. We construct impulse-response functions for the quantiles of a sample of 230 financial institutions around the world and study how financial institution-specific and system-wide shocks are absorbed by the system. We show how the long-run risk of the largest and most leveraged financial institutions is very sensitive to market wide shocks in situations of financial distress, suggesting that our methodology can prove a valuable addition to the traditional toolkit of policy makers and supervisors.

VAR for VaR: Measuring Tail Dependence Using Multivariate Regression Quantiles

http://people.bu.edu/hkaido/pdf/TwoStage.pdf

A two-stage procedure for partially identified models

Sharpe style regression has become a widespread analytic tool in the financial community. The style regression allows one to investigate such interesting issues as style composition, style sensitivity, and style change over time. All previous methods to obtain the distribution and confidence intervals of the style coefficients are statistically valid only in the special case in which none of the true style weights are zero or one. In practice it is quite plausible to have zero or one for the values of some style weights. In this paper we apply new results of Andrews (1997a, 1999) and develop a comparable Bayesian method to obtain statistically valid distributions and confidence intervals regardless of the true values of style weights.

/pdf/asymptotic-and-bayesian-confidence-intervals-for-sharpe-bz394un2i4.pdf

Halbert White

Papers

Causal discourse in a game of incomplete information

Abstract 055: Statins Raise Glucose Preferentially among Men Who are Older and at Greater Metabolic Risk

VAR for VaR: Measuring Tail Dependence Using Multivariate Regression Quantiles

A two-stage procedure for partially identified models

Asymptotic and Bayesian Confidence Intervals for Sharpe Style Weights