The authors used meta-analytic procedures to examine the relationship between specified training design and evaluation features and the effectiveness of training in organizations. Results of the meta-analysis revealed training effectiveness sample-weighted mean ds of 0.60 (k 15, N 936) for reaction criteria, 0.63 (k 234, N 15,014) for learning criteria, 0.62 ( k 122, N 15,627) for behavioral criteria, and 0.62 (k 26, N 1,748) for results criteria. These results suggest a medium to large effect size for organizational training. In addition, the training method used, the skill or task characteristic trained, and the choice of evaluation criteria were related to the effectiveness of training programs. Limitations of the study along with suggestions for future research are discussed. The continued need for individual and organizational development can be traced to numerous demands, including maintaining superiority in the marketplace, enhancing employee skills and knowledge, and increasing productivity. Training is one of the most pervasive methods for enhancing the productivity of individuals and communicating organizational goals to new personnel. In 2000, U.S. organizations with 100 or more employees budgeted to spend $54 billion on formal training (“Industry Report,” 2000). Given the importance and potential impact of training on organizations and the costs associated with the development and implementation of training, it is important that both researchers and practitioners have a better understanding of the relationship between design and evaluation features and the effectiveness of training and development efforts. Meta-analysis quantitatively aggregates the results of primary studies to arrive at an overall conclusion or summary across these studies. In addition, meta-analysis makes it possible to assess relationships not investigated in the original primary studies. These, among others (see Arthur, Bennett, & Huffcutt, 2001), are some of the advantages of meta-analysis over narrative reviews. Although there have been a multitude of meta-analyses in other domains of industrial/organizational psychology (e.g., cognitive ability, employment interviews, assessment centers, and employment-related personality testing) that now allow researchers to make broad summary statements about observable effects and relationships in these domains, summaries of the training effectiveness literature appear to be limited to the periodic narrative Annual Reviews. A notable exception is Burke and Day (1986), who, however, limited their meta-analysis to the effectiveness of only managerial training. Consequently, the goal of the present article is to address this gap in the training effectiveness literature by conducting a metaanalysis of the relationship between specified design and evaluation features and the effectiveness of training in organizations. We accomplish this goal by first identifying design and evaluation features related to the effectiveness of organizational training programs and interventions, focusing specifically on those features over which practitioners and researchers have a reasonable degree of control. We then discuss our use of meta-analytic procedures to quantify the effect of each feature and conclude with a discussion of the implications of our findings for both practitioners and researchers.

/pdf/effectiveness-of-training-in-organizations-a-meta-analysis-2twtct8wcr.pdf

Effectiveness of training in organizations: a meta-analysis of design and evaluation features.

This chapter advocates the good scientific practice of systematic research syntheses in Management and Organizational Science (MOS). A research synthesis is the systematic accumulation, analysis and reflective interpretation of the full body of relevant empirical evidence related to a question. It is the critical first step in effective use of scientific evidence. Synthesis is not a conventional literature review. Literature reviews are often position papers, cherry-picking studies to advocate a point of view. Instead, syntheses systematically identify where research findings are clear (and where they aren't), a key first step to establishing the conclusions science supports. Syntheses are also important for identifying contested findings and productive lines for future research. Uses of MOS evidence, that is, the motives for undertaking a research synthesis include scientific discovery and explanation, improved management practice guidelines, and formulating public policy. We identify six criteria for establishing the evidentiary value of a body of primary studies in MOS. We then pinpoint the stumbling blocks currently keeping the field from making effective use of its ever-expanding base of empirical studies. Finally, this chapter outlines a) an approach to research synthesis suitable to the domain of MOS and b) supporting practices to make synthesis a collective MOS project.

Evidence in Management and Organizational Science: Assembling the Field's Full Weight of Scientific Knowledge through Syntheses

In this paper, the state of research on the assessment of competencies in higher education is reviewed. Fundamental conceptual and methodological issues are clarified by showing that current controversies are built on misleading dichotomies. By systematically sketching conceptual controversies, competing competence definitions are unpacked (analytic/trait vs. holistic/real-world performance) and common- places are identified. Disagreements are also highlighted. Similarly, competing statistical approaches to assessing competencies, namely item- response theory (latent trait) versus generalizability theory (sampling error variance), are unpacked. The resulting framework moves beyond dichotomies and shows how the different approaches complement each other. Competence is viewed along a continuum from traits that underlie perception, interpretation, and decision-making skills, which in turn give rise to observed behavior in real-world situations. Statistical approaches are also viewed along a continuum from linear to nonlinear models that serve different purposes. Item response theory (IRT) models may be used for scaling item responses and modeling structural relations, and generalizability theory (GT) models pinpoint sources of measurement error variance, thereby enabling the design of reliable measurements. The proposed framework suggests multiple new research studies and may serve as a ''grand'' structural model.

/pdf/beyond-dichotomies-competence-viewed-as-a-continuum-3adhfgr9zv.pdf

Beyond dichotomies: Competence viewed as a continuum

This study builds on insights from both upper echelons and agency perspectives to examine the effects on corporate social responsibility (CSR) practices of CEO's narcissism. Drawing on prior theory about CEO narcissism, we argue that CSR can be a response to leaders' personal needs for attention and image reinforcement and hypothesize that CEO narcissism has positive effects on levels and profile of organizational CSR; additionally, CEO narcissism will reduce the effect of CSR on performance. We find support for our ideas with a sample of Fortune 500 CEOs, operationalizing CEO narcissism with a novel media-based measurement technique that uses third-party ratings of CEO characteristics with validated psychometric scales. Copyright © 2014 John Wiley & Sons, Ltd.

Corporate social responsibility or CEO narcissism? CSR motivations and organizational performance

This chapter advocates the good scientific practice of systematic research syntheses in Management and Organizational Science (MOS). A research synthesis is the systematic accumulation, analysis and reflective interpretation of the full body of relevant empirical evidence related to a question. It is the critical first step in effective use of scientific evidence. Synthesis is not a conventional literature review. Literature reviews are often position papers, cherry‐picking studies to advocate a point of view. Instead, syntheses systematically identify where research findings are clear (and where they aren’t), a key first step to establishing the conclusions science supports. Syntheses are also important for identifying contested findings and productive lines for future research. Uses of MOS evidence, that is, the motives for undertaking a research synthesis include scientific discovery and explanation, improved management practice guidelines, and formulating public policy. We identify six criter...

11 Evidence in Management and Organizational Science: Assembling the Field’s Full Weight of Scientific Knowledge Through Syntheses

We used meta-analytic procedures to investigate the criterion-related validity of assessment center dimension ratings. By focusing on dimension-level information, we were able to assess the extent to which specific constructs account for the criterion-related validity of assessment centers. From a total of 34 articles that reported dimension-level validities, we collapsed 168 assessment center dimension labels into an overriding set of 6 dimensions: (a) consideration/awareness of others, (b) communication, (c) drive, (d) influencing others, (e) organizing and planning, and (f) problem solving. Based on this set of 6 dimensions, we extracted 258 independent data points. Results showed a range of estimated true criterion-related validities from .25 to .39. A regression-based composite consisting of 4 out of the 6 dimensions accounted for the criterion-related validity of assessment center ratings and explained more variance in performance (20%) than Gaugler, Rosenthal, Thornton, and Bentson (1987) were able to explain using the overall assessment center rating (14%).

Pamela S. Edens

Papers

A meta‐analysis of the criterion‐related validity of assessment center dimensions