Multiple regression analysis is one of the most widely used statistical procedures for both scholarly and applied marketing research. Yet, correlated predictor variables—and potential collinearity ...

/pdf/collinearity-power-and-interpretation-of-multiple-regression-ob8eta25a8.pdf

Collinearity, power, and interpretation of multiple regression analysis.

Bootstrap resampling methods have emerged as powerful tools for constructing inferential procedures in modern statistical data analysis. Although these methods depend on the availability of fast, inexpensive computing, they offer the potential for highly accurate methods of inference. Moreover, they can even eliminate the need to impose a convenient statistical model that does not have a strong scientific basis. In this article, we review some bootstrap methods, emphasizing applications through illustrations with some real data. Special attention is given to regression, problems with dependent data, and choosing tuning parameters for optimal performance.

Bootstrap technology and applications

Publisher Summary This chapter reviews Efron's method called the bootstrap, and briefly mentions its relation to the jackknife, with a particular emphasis on econometric applications. Bootstrap literature has made tremendous progress in solving an old statistical problem: making reliable confidence statements in complicated small sample, multi-step, dependent, and non-normal cases. Resampling methods provide radically new solutions to several modeling problems involving interdependence, simultaneity, nonlinearity, nonstationarity, instability, nonnormality, heteroscedasticity, small or missing data, Hawthorne effect, and more solutions. The bootstrap handles these problems nonparametrically and intuitively, avoiding complicated power functions, Cramer–Rao lower bounds, bias corrections for Wald or Lagrange multiplier tests, and such. Many early applications of the bootstrap in econometrics attempts to provide an alternative to asymptotic standard error estimates. The jackknife is also used to find improved estimates of the standard errors. The bootstrap offers a potentially valuable insight into the sampling distributions, beyond simpler and improved estimation of standard errors. When two or more statistical tests are used, their power is difficult to determine analytically. The bootstrap sampling distribution can eliminate the need for tedious computations of the power in some cases. The chapter also discusses the post hoc technique for cleverly manipulating the bootstrap replications, computational aspects of bootstrap methods, and simultaneous equation and dynamic econometric models which require a special setup different from the traditional bootstrap.

23 Bootstrap methods: Applications in econometrics

Influential data points can affect the results of a regression analysis; for example, the usual sum mary statistics and tests of significance may be misleading. The importance of regression diagnostics in detecting influential points is discussed, and five statistics are recommended for the applied researcher. The suggested diagnostics were used on a small dataset to detect an influen tial data point, and the effects were analyzed. Colinearity-based diagnostics also are discussed and illustrated on the same dataset. The non- robustness of the least squares estimates in the presence of influential points is emphasized. Diagnostics for multiple influential points, multi variate regression, multicolinearity, nonlinear regression, and other multivariate procedures also are discussed.

/pdf/a-review-of-regression-diagnostics-for-behavioral-research-136m88bn2z.pdf

A review of regression diagnostics for behavioral research

Neural networks are being used as tools for data analysis in a variety of applications. Neural network technique is cited in the literature as a 'Black Box' approach and criticized most for the lack of interpretability of the network weights obtained during the model building process. Some attempts have been made in the past in this direction to interpret the contributions of explanatory variables in prediction problem using the weights of neural network. In the present study, a new approach is proposed to interpret the relative importance of independent variables in neural networks and a comparison with the connection weight approach is presented. The performance of this approach is studied for various data characteristics and is found to be a better method in comparison to a well known method existing in the literature. An example from behavioral science is also considered to illustrate how the performance of the proposed approach translates to a real life situation.

Assessing the contribution of variables in feed forward neural network

Several existing methods for the choice of the ridge parameter are reviewed, and a bootstrap method is proposed. The bootstrap provides independent measures of prediction errors based on multiple predictions along with an estimate of the standard error of prediction. The bootstrap and selected competitors are compared through Monte Carlo simulations for various degrees of design matrix collinearity and varying levels of signal-to-noise ratio. The procedure is also illustrated by application to two published data sets. In one case, the bootstrap choice of the ridge parameter leads to a smaller mean squared error of prediction than the ridge trace method. In the second case, an optimal choice of no perturbation is confirmed. Benefits of the bootstrap choice include its less subjective nature, ease of implementation, and robustness.

Nancy Jo Delaney

Papers

Use of the Bootstrap and Cross-Validation in Ridge Regression