Showing papers by "Victor Chernozhukov published in 2015"

Uniform post-selection inference for least absolute deviation regression and other Z-estimation problems

Abstract: We develop uniformly valid confidence regions for regression coefficients in a high-dimensional sparse median regression model with homoscedastic errors. Our methods are based on a moment equation that is immunized against non-regular estimation of the nuisance part of the median regression function by using Neyman's orthogonalization. We establish that the resulting instrumental median regression estimator of a target regression coefficient is asymptotically normally distributed uniformly with respect to the underlying sparse model and is semi-parametrically efficient. We also generalize our method to a general non-smooth Z-estimation framework with the number of target parameters $p_1$ being possibly much larger than the sample size $n$. We extend Huber's results on asymptotic normality to this setting, demonstrating uniform asymptotic normality of the proposed estimators over $p_1$-dimensional rectangles, constructing simultaneous confidence bands on all of the $p_1$ target parameters, and establishing asymptotic validity of the bands uniformly over underlying approximately sparse models. Keywords: Instrument; Post-selection inference; Sparsity; Neyman's Orthogonal Score test; Uniformly valid inference; Z-estimation.

Monge-Kantorovich Depth, Quantiles, Ranks and Signs

Abstract: We propose new concepts of statistical depth, multivariate quantiles,ranks and signs, based on canonical transportation maps between a distributionof interest on IRd and a reference distribution on the d-dimensionalunit ball. The new depth concept, called Monge-Kantorovich depth, specializesto halfspace depth in the case of elliptical distributions, but, for more generaldistributions, differs from the latter in the ability for its contours to account fornon convex features of the distribution of interest. We propose empirical counterpartsto the population versions of those Monge-Kantorovich depth contours,quantiles, ranks and signs, and show their consistency by establishing a uniformconvergence property for transport maps, which is of independent interest.

Valid Post-Selection and Post-Regularization Inference: An Elementary, General Approach

Abstract: We present an expository, general analysis of valid post-selection or post-regularization inference about a low-dimensional target parameter in the presence of a very high-dimensional nuisance parameter that is estimated using selection or regularization methods. Our analysis provides a set of high-level conditions under which inference for the low-dimensional parameter based on testing or point estimation methods will be regular despite selection or regularization biases occurring in the estimation of the high-dimensional nuisance parameter. A key element is the use of so-called immunized or orthogonal estimating equations that are locally insensitive to small mistakes in the estimation of the high-dimensional nuisance parameter. As an illustration, we analyze affine-quadratic models and specialize these results to a linear instrumental variables model with many regressors and many instruments. We conclude with a review of other developments in post-selection inference and note that many can be viewed as...

Post-Selection and Post-Regularization Inference in Linear Models with Many Controls and Instruments

Abstract: In this note, we offer an approach to estimating structural parameters in the presence of many instruments and controls based on methods for estimating sparse high-dimensional models. We use these high-dimensional methods to select which instruments and which control variables to use. The approach we take extends Belloni et al. (2012), which covers selection of instruments for IV models with a small number of controls, and extends Belloni, Chernozhukov, and Hansen (2014), which covers selection of controls in models where the variable of inter est is exogenous conditional on observables, to accommodate both a large number of controls and a large number of instruments. We illustrate the approach with a simulation and an empirical example. Technical supporting material is available in the online Appendix.

Post-Selection and Post-Regularization Inference in Linear Models with Many Controls and Instruments

Abstract: In this note, we offer an approach to estimating causal/structural parameters in the presence of many instruments and controls based on methods for estimating sparse high-dimensional models. We use these high-dimensional methods to select both which instruments and which control variables to use. The approach we take extends BCCH2012, which covers selection of instruments for IV models with a small number of controls, and extends BCH2014, which covers selection of controls in models where the variable of interest is exogenous conditional on observables, to accommodate both a large number of controls and a large number of instruments. We illustrate the approach with a simulation and an empirical example. Technical supporting material is available in a supplementary online appendix.

Valid Post-Selection and Post-Regularization Inference: An Elementary, General Approach

Abstract: Here we present an expository, general analysis of valid post-selection or post-regularization inference about a low-dimensional target parameter, $\alpha$, in the presence of a very high-dimensional nuisance parameter, $\eta$, which is estimated using modern selection or regularization methods. Our analysis relies on high-level, easy-to-interpret conditions that allow one to clearly see the structures needed for achieving valid post-regularization inference. Simple, readily verifiable sufficient conditions are provided for a class of affine-quadratic models. We focus our discussion on estimation and inference procedures based on using the empirical analog of theoretical equations $$M(\alpha, \eta)=0$$ which identify $\alpha$. Within this structure, we show that setting up such equations in a manner such that the orthogonality/immunization condition $$\partial_\eta M(\alpha, \eta) = 0$$ at the true parameter values is satisfied, coupled with plausible conditions on the smoothness of $M$ and the quality of the estimator $\hat \eta$, guarantees that inference on for the main parameter $\alpha$ based on testing or point estimation methods discussed below will be regular despite selection or regularization biases occurring in estimation of $\eta$. In particular, the estimator of $\alpha$ will often be uniformly consistent at the root-$n$ rate and uniformly asymptotically normal even though estimators $\hat \eta$ will generally not be asymptotically linear and regular. The uniformity holds over large classes of models that do not impose highly implausible "beta-min" conditions. We also show that inference can be carried out by inverting tests formed from Neyman's $C(\alpha)$ (orthogonal score) statistics.

Empirical and multiplier bootstraps for suprema of empirical processes of increasing complexity, and related Gaussian couplings

Abstract: We derive strong approximations to the supremum of the non-centered empirical process indexed by a possibly unbounded VC-type class of functions by the suprema of the Gaussian and bootstrap processes. The bounds of these approximations are non-asymptotic, which allows us to work with classes of functions whose complexity increases with the sample size. The construction of couplings is not of the Hungarian type and is instead based on the Slepian-Stein methods and Gaussian comparison inequalities. The increasing complexity of classes of functions and non-centrality of the processes make the results useful for applications in modern nonparametric statistics (Gine and Nickl, 2015), in particular allowing us to study the power properties of nonparametric tests using Gaussian and bootstrap approximations.

Constrained conditional moment restriction models

Abstract: This paper examines a general class of inferential problems in semiparametric and nonparametric models defined by conditional moment restrictions. We construct tests for the hypothesis that at least one element of the identified set satisfies a conjectured (Banach space) "equality" and/or (a Banach lattice) "inequality" constraint. Our procedure is applicable to identified and partially identified models, and is shown to control the level, and under some conditions the size, asymptotically uniformly in an appropriate class of distributions. The critical values are obtained by building a strong approximation to the statistic and then bootstrapping a (conservatively) relaxed form of the statistic. Sufficient conditions are provided, including strong approximations using Koltchinskii's coupling. Leading important special cases encompassed by the framework we study include: (i) Tests of shape restrictions for infinite dimensional parameters; (ii) Confidence regions for functionals that impose shape restrictions on the underlying parameter; (iii) Inference for functionals in semiparametric and nonparametric models defined by conditional moment (in)equalities; and (iv) Uniform inference in possibly nonlinear and severely ill-posed problems.

Identification and Efficient Semiparametric Estimation of a Dynamic Discrete Game

Abstract: In this paper, we study the identification and estimation of a dynamic discrete game allowing for discrete or continuous state variables. We first provide a general nonparametric identification result under the imposition of an exclusion restriction on agent payoffs. Next we analyze large sample statistical properties of nonparametric and semiparametric estimators for the econometric dynamic game model. We also show how to achieve semiparametric efficiency of dynamic discrete choice models using a sieve based conditional moment framework. Numerical simulations are used to demonstrate the finite sample properties of the dynamic game estimators. An empirical application to the dynamic demand of the potato chip market shows that this technique can provide a useful tool to distinguish long term demand from short term demand by heterogeneous consumers.Institutional subscribers to the NBER working paper series, and residents of developing countries may download this paper without additional charge at www.nber.org.

A lava attack on the recovery of sums of dense and sparse signals

Abstract: Common high-dimensional methods for prediction rely on having either a sparse signal model, a model in which most parameters are zero and there are a small number of nonzero parameters that are large in magnitude, or a dense signal model, a model with no large parameters and very many small nonzero parameters. We consider a generalization of these two basic models, termed here a “sparse $+$ dense” model, in which the signal is given by the sum of a sparse signal and a dense signal. Such a structure poses problems for traditional sparse estimators, such as the lasso, and for traditional dense estimation methods, such as ridge estimation. We propose a new penalization-based method, called lava, which is computationally efficient. With suitable choices of penalty parameters, the proposed method strictly dominates both lasso and ridge. We derive analytic expressions for the finite-sample risk function of the lava estimator in the Gaussian sequence model. We also provide a deviation bound for the prediction risk in the Gaussian regression model with fixed design. In both cases, we provide Stein’s unbiased estimator for lava’s prediction risk. A simulation example compares the performance of lava to lasso, ridge and elastic net in a regression example using data-dependent penalty parameters and illustrates lava’s improved performance relative to these benchmarks.

The Sorted Effects Method: Discovering Heterogeneous Effects Beyond Their Averages

Abstract: The partial (ceteris paribus) effects of interest in nonlinear and interactive linear models are heterogeneous as they can vary dramatically with the underlying observed or unobserved covariates. Despite the apparent importance of heterogeneity, a common practice in modern empirical work is to largely ignore it by reporting average partial effects (or, at best, average effects for some groups). While average effects provide very convenient scalar summaries of typical effects, by definition they fail to reflect the entire variety of the heterogeneous effects. In order to discover these effects much more fully, we propose to estimate and report sorted effects—a collection of estimated partial effects sorted in increasing order and indexed by percentiles. By construction, the sorted effect curves completely represent and help visualize the range of the heterogeneous effects in one plot. They are as convenient and easy to report in practice as the conventional average partial effects. They also serve as a basis for classification analysis, where we divide the observational units into most or least affected groups and summarize their characteristics. We provide a quantification of uncertainty (standard errors and confidence bands) for the estimated sorted effects and related classification analysis, and provide confidence sets for the most and least affected groups. The derived statistical results rely on establishing key, new mathematical results on Hadamard differentiability of a multivariate sorting operator and a related classification operator, which are of independent interest. We apply the sorted effects method and classification analysis to demonstrate several striking patterns in the gender wage gap. We find that this gap is particularly strong for married women, ranging from −60% to 0% between the 2% and 98% percentiles, as a function of observed and unobserved characteristics; while the gap for never married women ranges from −40% to +20%. The most adversely affected women tend to be married, do not have college degrees, work in sales, and have high levels of potential experience.

The Sorted Effects Method: Discovering Heterogeneous Effects Beyond Their Averages

Abstract: The partial (ceteris paribus) effects of interest in nonlinear and interactive linear models are heterogeneous as they can vary dramatically with the underlying observed or unobserved covariates. Despite the apparent importance of heterogeneity, a common practice in modern empirical work is to largely ignore it by reporting average partial effects (or, at best, average effects for some groups). While average effects provide very convenient scalar summaries of typical effects, by definition they fail to reflect the entire variety of the heterogeneous effects. In order to discover these effects much more fully, we propose to estimate and report sorted effects -- a collection of estimated partial effects sorted in increasing order and indexed by percentiles. By construction the sorted effect curves completely represent and help visualize the range of the heterogeneous effects in one plot. They are as convenient and easy to report in practice as the conventional average partial effects. They also serve as a basis for classification analysis, where we divide the observational units into most or least affected groups and summarize their characteristics. We provide a quantification of uncertainty (standard errors and confidence bands) for the estimated sorted effects and related classification analysis, and provide confidence sets for the most and least affected groups. The derived statistical results rely on establishing key, new mathematical results on Hadamard differentiability of a multivariate sorting operator and a related classification operator, which are of independent interest. We apply the sorted effects method and classification analysis to demonstrate several striking patterns in the gender wage gap.

A lava attack on the recovery of sums of dense and sparse signals

Abstract: Common high-dimensional methods for prediction rely on having either a sparse signal model, a model in which most parameters are zero and there are a small number of non-zero parameters that are large in magnitude, or a dense signal model, a model with no large parameters and very many small non-zero parameters. We consider a generalization of these two basic models, termed here a "sparse+dense" model, in which the signal is given by the sum of a sparse signal and a dense signal. Such a structure poses problems for traditional sparse estimators, such as the lasso, and for traditional dense estimation methods, such as ridge estimation. We propose a new penalization-based method, called lava, which is computationally efficient. With suitable choices of penalty parameters, the proposed method strictly dominates both lasso and ridge. We derive analytic expressions for the finite-sample risk function of the lava estimator in the Gaussian sequence model. We also provide an deviation bound for the prediction risk in the Gaussian regression model with fixed design. In both cases, we provide Stein's unbiased estimator for lava's prediction risk. A simulation example compares the performance of lava to lasso, ridge, and elastic net in a regression example using feasible, data-dependent penalty parameters and illustrates lava's improved performance relative to these benchmarks.

Uniformly Valid Post-Regularization Confidence Regions for Many Functional Parameters in Z-Estimation Framework

Abstract: In this paper we develop procedures to construct simultaneous confidence bands for $\tilde p$ potentially infinite-dimensional parameters after model selection for general moment condition models where $\tilde p$ is potentially much larger than the sample size of available data, $n$. This allows us to cover settings with functional response data where each of the $\tilde p$ parameters is a function. The procedure is based on the construction of score functions that satisfy certain orthogonality condition. The proposed simultaneous confidence bands rely on uniform central limit theorems for high-dimensional vectors (and not on Donsker arguments as we allow for $\tilde p \gg n$). To construct the bands, we employ a multiplier bootstrap procedure which is computationally efficient as it only involves resampling the estimated score functions (and does not require resolving the high-dimensional optimization problems). We formally apply the general theory to inference on regression coefficient process in the distribution regression model with a logistic link, where two implementations are analyzed in detail. Simulations and an application to real data are provided to help illustrate the applicability of the results.

Inference on sets in finance

Abstract: We consider the problem of inference on a class of sets describing a collection of admissible models as solutions to a single smooth inequality. Classical and recent examples include the Hansen–Jagannathan sets of admissible stochastic discount factors, Markowitz–Fama mean–variance sets for asset portfolio returns, and the set of structural elasticities in Chetty's (2012) analysis of demand with optimization frictions. The econometric structure of the problem allows us to construct convenient and powerful confidence regions based on the weighted likelihood ratio and weighted Wald statistics. Our statistics differ from existing statistics in that they enforce either exact or first‐order equivariance to transformations of parameters, making them especially appealing in the target applications. We show that the resulting inference procedures are more powerful than the structured projection methods. Last, our framework is also useful for analyzing intersection bounds, namely sets defined as solutions to multiple smooth inequalities, since multiple inequalities can be conservatively approximated by a single smooth inequality. We present two empirical examples showing how the new econometric methods are able to generate sharp economic conclusions. Hansen–Jagannathan bound Markowitz–Fama bounds Chetty bounds mean–variance sets optimization frictions inference confidence set C10 C50

The sorted effects method: discovering heterogeneous effects beyond their averages

Abstract: The partial (ceteris paribus) e?ects of interest in nonlinear and interactive linear models are heterogeneous as they can vary dramatically with the underlying observed or unobserved covariates. Despite the apparent importance of heterogeneity, a common practice in modern empirical work is to largely ignore it by reporting average partial e?ects (or, at best, average e?ects for some groups, see e.g. Angrist and Pischke (2008)). While average e?ects provide very convenient scalar summaries of typical e?ects, by de?nition they fail to re?ect the entire variety of the heterogenous e?ects. In order to discover these e?ects much more fully, we propose to estimate and report sorted e?ects – a collection of estimated partial e?ects sorted in increasing order and indexed by percentiles. By construction the sorted e?ect curves completely represent and help visualize all of the heterogeneous e?ects in one plot. They are as convenient and easy to report in practice as the conventional average partial e?ects. We also provide a quanti?cation of uncertainty (standard errors and con?dence bands) for the estimated sorted e?ects. We apply the sorted e?ects method to demonstrate several striking patterns of gender-based discrimination in wages, and of race-based discrimination in mortgage lending. Using di?erential geometry and functional delta methods, we establish that the estimated sorted e?ects are consistent for the true sorted e?ects, and derive asymptotic normality and bootstrap approximation results, enabling construction of pointwise con?dence bands (point-wise with respect to percentile indices). We also derive functional central limit theorems and bootstrap approximation results, enabling construction of simultaneous con?dence bands (simultaneous with respect to percentile indices). The derived statistical results in turn rely on establishing Hadamard di?erentiability of the multivariate sorting operator, a result of independent mathematical interest.

Inference: An Elementary, General Approach

Abstract: We present an expository, general analysis of valid post-selection or post-regularization inference about a low-dimensional target parameter in the presence of a very high-dimensional nuisance parameter that is estimated using selection or regularization methods. Our analysis provides a set of high-level conditions under which inference for the low-dimensional parameter based on testing or point estimation methodswillberegulardespiteselectionorregularizationbiasesoccurringin the estimation of the high-dimensional nuisance parameter. A key element is the use of so-called immunized or orthogonal estimating equations that are locally insensitive to small mistakes in the estimation of the high-dimensional nuisance parameter. As an illustration, we analyze affine-quadratic models and specialize these results to a linear instrumentalvariablesmodelwithmanyregressorsandmanyinstruments. We conclude with a review of other developments in post-selection inference and note that many can be viewed as special cases of the general encompassing framework of orthogonal estimating equations provided in this article.

Constrained conditional moment restriction models

Abstract: This paper examines a general class of inferential problems in semiparametric and nonparametric models defined by conditional moment restrictions. We construct tests for the hypothesis that at least one element of the identified set satisfies a conjectured (Banach space) “equality” and/or (a Banach lattice) “inequality” constraint. Our procedure is applicable to identified and partially identified models, and is shown to control the level, and under some conditions the size, asymptotically uniformly in an appropriate class of distributions. The critical values are obtained by building a strong approximation to the statistic and then bootstrapping a (conservatively) relaxed form of the statistic. Sufficient conditions are provided, including strong approximations using Koltchinskii's coupling. Leading important special cases encompassed by the framework we study include: (i) Tests of shape restrictions for infinite dimensional parameters; (ii) Confidence regions for functionals that impose shape restrictions on the underlying parameter; (iii) Inference for functionals in semiparametric and nonparametric models defined by conditional moment (in)equalities; and (iv) Uniform inference in possibly nonlinear and severely ill-posed problems.

Coupling inequalities for suprema of non-centered empirical and bootstrap processes

Abstract: We derive strong approximations to the supremum of the non-centered empirical process indexed by a possibly unbounded VC-type class of functions by the suprema of the Gaussian and bootstrap processes. The bounds of these approximations are non-asymptotic, which allows us to work with classes of functions whose complexity increases with the sample size. The construction of couplings is not of the Hungarian type and is instead based on the Slepian-Stein methods and Gaussian comparison inequalities. The increasing complexity of classes of functions and non-centrality of the processes make the results useful for applications in modern nonparametric statistics (Gine and Nickl, 2015), in particular allowing us to study the power properties of nonparametric tests using Gaussian and bootstrap approximations.

Post-Selection and Post-Regularization Inference in Linear Models with Many Controls and Instruments

Abstract: In this note, we offer an approach to estimating structural parameters in the presence of many instruments and controls based on methods for estimating sparse high-dimensional models. We use these high-dimensional methods to select both which instruments and which control variables to use. The approach we take extends Belloni et al. (2012), which covers selection of instruments for IV models with a small number of controls, and extends Belloni, Chernozhukov and Hansen (2014), which covers selection of controls in models where the variable of interest is exogenous conditional on observables, to accommodate both a large number of controls and a large number of instruments. We illustrate the approach with a simulation and an empirical example. Technical supporting material is available in a supplementary appendix.

Monge-Kantorovich Depth, Quantiles, Ranks, and Signs

Abstract: We propose new concepts of statistical depth, multivariate quantiles, ranks and signs, based on canonical transportation maps between a distribution of interest on Rd and a reference distribution on the d-dimensional unit ball. The new depth concept, called Monge-Kantorovich depth, specializes to halfspace depth in the case of elliptical distributions, but, for more general distributions, differs from the latter in the ability for its contours to account for non convex features of the distribution of interest. We propose empirical counterparts to the population versions of those Monge-Kantorovich depth contours, quantiles, ranks and signs, and show their consistency by establishing a uniform convergence property for empirical transport maps, which is of independent interest.

Identification and Efficient Semiparametric Estimation of a Dynamic Discrete Game

Abstract: In this paper, we study the identification and estimation of a dynamic discrete game allowing for discrete or continuous state variables. We first provide a general nonparametric identification result under the imposition of an exclusion restriction on agent payoffs. Next we analyze large sample statistical properties of nonparametric and semiparametric estimators for the econometric dynamic game model. We also show how to achieve semiparametric efficiency of dynamic discrete choice models using a sieve based conditional moment framework. Numerical simulations are used to demonstrate the finite sample properties of the dynamic game estimators. An empirical application to the dynamic demand of the potato chip market shows that this technique can provide a useful tool to distinguish long term demand from short term demand by heterogeneous consumers.

Post-selection and post-regularization inference in linear models with many controls and instruments

Abstract: In this note, we offer an approach to estimating structural parameters in the presence of many instruments and controls based on methods for estimating sparse high-dimensional models. We use these high-dimensional methods to select both which instruments and which control variables to use. The approach we take extends Belloni et al. (2012), which covers selection of instruments for IV models with a small number of controls, and extends Belloni, Chernozhukov and Hansen (2014), which covers selection of controls in models where the variable of interest is exogenous conditional on observables, to accommodate both a large number of controls and a large number of instruments. We illustrate the approach with a simulation and an empirical example. Technical supporting material is available in a supplementary appendix here.

A Lava Attack on the Recovery of Sums of Dense and Sparse Signals

Abstract: Common high-dimensional methods for prediction rely on having either a sparse signal model, a model in which most parameters are zero and there are a small number of non-zero parameters that are large in magnitude, or a dense signal model, a model with no large parameters and very many small non-zero parameters. We consider a generalization of these two basic models, termed here a 'sparse dense' model, in which the signal is given by the sum of a sparse signal and a dense signal. Such a structure poses problems for traditional sparse estimators, such as the lasso, and for traditional dense estimation methods, such as ridge estimation. We propose a new penalization-based method, called lava, which is computationally efficient. With suitable choices of penalty parameters, the proposed method strictly dominates both lasso and ridge. We derive analytic expressions for the finite-sample risk function of the lava estimator in the Gaussian sequence model. We also provide an deviation bound for the prediction risk in the Gaussian regression model with fixed design. In both cases, we provide Stein's unbiased estimator for lava's prediction risk. A simulation example compares the performance of lava to lasso, ridge, and elastic net in a regression example using feasible, data-dependent penalty parameters and illustrates lava's improved performance relative to these benchmarks.