Statistical difficulties of detecting interactions and moderator effects
01 Sep 1993-Psychological Bulletin (American Psychological Association)-Vol. 114, Iss: 2, pp 376-390
TL;DR: It is demonstrated that the differential efficiency of experimental and field tests of interactions is also attributable to the differential residual variances of such interactions once the component main effects have been partialed out.
Abstract: Although interaction effects are frequently found in experimental studies, field researchers report considerable difficulty in finding theorized moderator effects. Previous discussions of this discrepancy have considered responsible factors including differences in measurement error and use of nonlinear scales. In this article we demonstrate that the differential efficiency of experimental and field tests of interactions is also attributable to the differential residual variances of such interactions once the component main effects have been partialed out. We derive an expression for this residual variance in terms of the joint distribution of the component variables and explore how properties of the distribution affect the efficiency of tests of moderator effects. We show that tests of interactions in field studies will often have less than 20% of the efficiency of optimal experimental tests, and we discuss implications for the design of field studies.
Citations
More filters
5,872 citations
Cites background from "Statistical difficulties of detecti..."
...Even under ideal circumstances, however, the statistical power is much lower for detecting interactions than for main effects (Breslow, 1980; McClelland and Judd, 1993)....
[...]
...Although not significant, tests of interaction, particularly in the presence of multicollinearity, tend to have low sensitivity (McClelland and Judd, 1993)....
[...]
TL;DR: A new latent variable modeling approach is provided that can give more accurate estimates of interaction effects by accounting for the measurement error that attenuates the estimated relationships.
Abstract: The ability to detect and accurately estimate the strength of interaction effects are critical issues that are fundamental to social science research in general and IS research in particular. Within the IS discipline, a significant percentage of research has been devoted to examining the conditions and contexts under which relationships may vary, often under the general umbrella of contingency theory (cf. McKeen et al. 1994, Weill and Olson 1989). In our survey of such studies, the majority failed to either detect or provide an estimate of the effect size. In cases where effect sizes are estimated, the numbers are generally small. These results have led some researchers to question both the usefulness of contingency theory and the need to detect interaction effects (e.g., Weill and Olson 1989). This paper addresses this issue by providing a new latent variable modeling approach that can give more accurate estimates of interaction effects by accounting for the measurement error that attenuates the estimated relationships. The capacity of this approach at recovering true effects in comparison to summated regression is demonstrated in a Monte Carlo study that creates a simulated data set in which the underlying true effects are known. Analysis of a second, empirical data set is included to demonstrate the technique's use within IS theory. In this second analysis, substantial direct and interaction effects of enjoyment on electronic-mail adoption are shown to exist.
5,639 citations
TL;DR: In this paper, the authors describe differences between moderator and mediator effects, and provide non-technical descriptions of how to examine each type of effect, including study design, analysis, and interpretation of results.
Abstract: The goals of this article are to (a) describe differences between moderator and mediator effects; (b) provide nontechnical descriptions of how to examine each type of effect, including study design, analysis, and interpretation of results; (c) demonstrate how to analyze each type of effect; and (d) provide suggestions for further reading. The authors focus on the use of multiple regression because it is an accessible data-analytic technique contained in major statistical packages. When appropriate, they also note limitations of using regression to detect moderator and mediator effects and describe alternative procedures, particularly structural equation modeling. Finally, to illustrate areas of confusion in counseling psychology research, they review research testing moderation and mediation that was published in the Journal of Counseling Psychology during 2001.
4,012 citations
Cites background from "Statistical difficulties of detecti..."
...Some (Aguinis, 1995; Jaccard & Wan, 1995; Judd et al., 1995; McClelland & Judd, 1993 ) also have suggested raising the alpha level above the traditional .05 level to maximize power, with various caveats....
[...]
...Low power is a particular problem in nonexperimental studies, which have much less power for detecting interaction effects than do experiments ( McClelland & Judd, 1993 )....
[...]
...Product terms must be entered into the regression equation after the predictor and moderator variables from which they were created (Aiken & West, 1991; Cohen et al., 2003; Dunlap & Kemery, 1987; Holmbeck, 1997; Jaccard et al., 1990; McClelland & Judd, 1993; West et al., 1996)....
[...]
...The second issue concerns restriction in range, which also reduces power (Aguinis, 1995; Aguinis & Stone-Romero, 1997; McClelland & Judd, 1993 )....
[...]
... McClelland and Judd (1993) provided specific recommendations regarding oversampling techniques that can be used to address this issue (see also Cohen et al., 2003, pp. 298 –299)....
[...]
TL;DR: In this article, a meta-analysis of meta-analyses is used to quantify the intention-behavior gap and a conceptual analysis of intention discrepancy is presented, and the scope of the intention construct is discussed in light of recent evidence concerning the role of habits and automaticity in human behavior.
Abstract: This chapter addresses two questions; how big is the “gap” between intentions and behavior, and what psychological variables might be able to “bridge” the intention–behavior gap? A meta-analysis of meta-analyses is used to quantify the gap and a conceptual analysis of intention–behavior discrepancies is presented. Research is described on the extent to which four groups of variables—behavior type, intention type, properties of intention, and cognitive and personality variables—moderate intention–behavior relations. Finally, the scope of the intention construct is discussed in the light of recent evidence concerning the role of habits and automaticity in human behavior.
2,996 citations
TL;DR: In this article, the authors who submit manuscripts to JIBS that appear to suffer from common method variance (CMV) are asked to perform validity checks and resubmit their manuscripts.
Abstract: JIBS receives many manuscripts that report findings from analyzing survey data based on same-respondent replies. This can be problematic since same-respondent studies can suffer from common method variance (CMV). Currently, authors who submit manuscripts to JIBS that appear to suffer from CMV are asked to perform validity checks and resubmit their manuscripts. This letter from the Editors is designed to outline the current state of best practice for handling CMV in international business research.
2,640 citations
Cites background from "Statistical difficulties of detecti..."
...See, for instance, McClelland and Judd (1993),...
[...]
...6See, for instance, McClelland and Judd (1993), who point out difficulties in detecting and interpreting interactions and moderator effects....
[...]
References
More filters
TL;DR: This article seeks to make theorists and researchers aware of the importance of not using the terms moderator and mediator interchangeably by carefully elaborating the many ways in which moderators and mediators differ, and delineates the conceptual and strategic implications of making use of such distinctions with regard to a wide range of phenomena.
Abstract: In this article, we attempt to distinguish between the properties of moderator and mediator variables at a number of levels. First, we seek to make theorists and researchers aware of the importance of not using the terms moderator and mediator interchangeably by carefully elaborating, both conceptually and strategically, the many ways in which moderators and mediators differ. We then go beyond this largely pedagogical function and delineate the conceptual and strategic implications of making use of such distinctions with regard to a wide range of phenomena, including control and stress, attitudes, and personality traits. We also provide a specific compendium of analytic procedures appropriate for making the most effective use of the moderator and mediator distinction, both separately and in terms of a broader causal system that includes both moderators and mediators.
80,095 citations
"Statistical difficulties of detecti..." refers background in this paper
...among others, formalized and verified Saunders's suggestion; Baron and Kenny (1986) distinguished between the testing of moderator effects and mediating effects; and Jaccard, Turrisi,...
[...]
...Birnbaum (1973) and Anderson and Shanteau (1977) demonstrated that even when the true model is entirely multiplicative (i....
[...]
Book•
01 Jan 1991TL;DR: In this article, the effects of predictor scaling on the coefficients of regression equations are investigated. But, they focus mainly on the effect of predictors scaling on coefficients of regressions.
Abstract: Introduction Interactions between Continuous Predictors in Multiple Regression The Effects of Predictor Scaling on Coefficients of Regression Equations Testing and Probing Three-Way Interactions Structuring Regression Equations to Reflect Higher Order Relationships Model and Effect Testing with Higher Order Terms Interactions between Categorical and Continuous Variables Reliability and Statistical Power Conclusion Some Contrasts Between ANOVA and MR in Practice
27,897 citations
[...]
TL;DR: This study takes involuntary job disruptions as illustrating life events and shows how they adversely affect enduring role strains, economic strains in particular, which erode positive concepts of self, such as self-esteem and mastery.
Abstract: This study uses longitudinal data to observe how life events, chronic life strains, self concepts, coping, and social supports come together to form a process of stress. It takes involuntary job disruptions as illustrating life events and shows how they adversely affect enduring role strains, economic strains in particular. These exacerbated strains, in turn, erode positive concepts of self, such as self-esteem and mastery. The diminished self-concepts then leave one especially vulnerable to experiencing symptoms of stress, of which depression is of special interest to this analysis. The interventions of coping and social supports are mainly indirect; that is, they do not act directly to buffer depression. Instead, they minimize the elevation of depression by dampening the antecedent process.
5,694 citations
TL;DR: In this article, the authors developed measures of multivariate skewness and kurtosis by extending certain studies on robustness of the t statistic, and the asymptotic distributions of the measures for samples from a multivariate normal population are derived and a test for multivariate normality is proposed.
Abstract: SUMMARY Measures of multivariate skewness and kurtosis are developed by extending certain studies on robustness of the t statistic. These measures are shown to possess desirable properties. The asymptotic distributions of the measures for samples from a multivariate normal population are derived and a test of multivariate normality is proposed. The effect of nonnormality on the size of the one-sample Hotelling's T2 test is studied empirically with the help of these measures, and it is found that Hotelling's T2 test is more sensitive to the measure of skewness than to the measure of kurtosis. measures have proved useful (i) in selecting a member of a family such as from the Karl Pearson family, (ii) in developing a test of normality, and (iii) in investigating the robustness of the standard normal theory procedures. The role of the tests of normality in modern statistics has recently been summarized by Shapiro & Wilk (1965). With these applications in mind for the multivariate situations, we propose measures of multivariate skewness and kurtosis. These measures of skewness and kurtosis are developed naturally by extending certain aspects of some robustness studies for the t statistic which involve I1 and 32. It should be noted that measures of multivariate dispersion have been available for quite some time (Wilks, 1932, 1960; Hotelling, 1951). We deal with the measure of skewness in ? 2 and with the measure of kurtosis in ? 3. In ? 4 we give two important applications of these measures, namely, a test of multivariate normality and a study of the effect of nonnormality on the size of the one-sample Hotelling's T2 test. Both of these problems have attracted attention recently. The first problem has been treated by Wagle (1968) and Day (1969) and the second by Arnold (1964), but our approach differs from theirs.
3,774 citations
Book•
01 Jan 1990
TL;DR: In this article, the authors present a survey of the existing literature on the analysis of moderated relationships involving continuous variables, focusing on analyzing interaction effects in the context of multiple regression and structural equation analyses.
Abstract: This monograph is concerned primarily with the statistical analysis of moderated relationships or as they are more commonly known interaction effects where all variables involved are continuous in nature. The focus is on analyzing interaction effects in the context of multiple regression and structural equation analyses. There currently exists a great deal of confusion about the analysis of moderated relationships involving continuous variables. The statistical and substantive literatures are replete with contradictory advice and admonitions about the best way to test models involving moderated relationships. Further the relevant statistical literature is scattered throughout a range of disciplines including sociology psychology political science economics biology and statistics. The major purpose of this monograph is to bring together this rather diverse literature and to explicate the central issues involved in conducting analyses of moderated relationships involving continuous variables. The principal finding is that interaction analysis is most straightforward when it is theoretically motivated; theory guides the specification of appropriate interaction models using multiple regression analysis. Traditional product terms with continuous variables assess interaction of a specific form namely bilinear interactions. The authors organize their analysis around 3 principal questions: 1) given the sample data can it be inferred that an interaction effect exists in the population; 2) if so what is the strength of the effect; and 3) if so what is the nature of the effect? When formulating research to test for interaction effects one should consider issues related to sample size (for purposes of power analysis) levels of measurement measurement error potential multicollinearity and other methodological/substantive issues discussed above. The monograph concludes with 10 empirical applications that have used multiple regression analysis for the analysis of moderated relationships.
3,193 citations
"Statistical difficulties of detecti..." refers background in this paper
...” Cohen (1978) and Arnold and Evans (1979), among others, formalized and verified Saunderss suggestion; Baron and Kenny (1986) distinguished between the testing of moderator effects and mediating effects; and Jaccard, Turrisi, and Wan (1990) and Aiken and West (1991) provided thorough, modern treatments of testing and interpreting moderator effects....
[...]