Showing papers in "Industrial and Organizational Psychology in 2008"

The Meaning of Employee Engagement

Abstract: The meaning of employee engagement is ambiguous among both academic researchers and among practitioners who use it in conversations with clients. We show that the term is used at different times to refer to psychological states, traits, and behaviors as well as their antecedents and outcomes. Drawing on diverse relevant literatures, we offer a series of propositions about (a) psychological state engagement; (b) behavioral engagement; and (c) trait engagement. In addition, we offer propositions regarding the effects of job attributes and leadership as main effects on state and behavioral engagement and as moderators of the relationships among the 3 facets of engagement. We conclude with thoughts about the measurement of the 3 facets of engagement and potential antecedents, especially measurement via employee surveys.

Stubborn reliance on intuition and subjectivity in employee selection.

Abstract: The focus of this article is on implicit beliefs that inhibit adoption of selection decision aids (e.g., paper-and-pencil tests, structured interviews, mechanical combination of predictors). Understanding these beliefs is just as important as understanding organizational constraints to the adoption of selection technologies and may be more useful for informing the design of successful interventions. One of these is the implicit belief that it is theoretically possible to achieve near-perfect precision in predicting performance on the job. That is, people have an inherent resistance to analytical approaches to selection because they fail to view selection as probabilistic and subject to error. Another is the implicit belief that prediction of human behavior is improved through experience. This myth of expertise results in an overreliance on intuition and a reluctance to undermine one’s own credibility by using a selection decision aid.

Employee Engagement From a Self-Determination Theory Perspective

Abstract: JOHN P. MEYERThe University of Western OntarioMARYLE`NE GAGNE´Concordia UniversityMaceyandSchneider(2008)drawonnumer-ous theories to explain what engagement isand how it is similar to and different fromrelated constructs in the organizationalbehavior literature. As a result, we now havea better understanding of some of the key‘‘components’’ofengagement.Whatappearsto be missing, however, is a strong unifyingtheory to guide research and practice. Webelieve that such a theory exists in the formof self-determination theory (SDT; Deci R Ryan & Deci, 2000) and its var-ious corollaries, self-concordance theory(SCT; Sheldon & Elliot, 1999), hierarchicaltheory (Vallerand, 1997), and passion theory(Vallerandetal.,2003).AlthoughMaceyandSchneider acknowledged the relevance ofSDT and SCT, we believe that much greateruseof thesetheoriescouldbe madetojustifyand extend their conceptual model.Self-Determination TheorySDT(Deci&Ryan,1985)proposestwoover-arching forms of motivation. Intrinsic moti-vation refers to doing an activity for its ownsake out of enjoyment and interest. Extrinsicmotivation refers to doing an activity forinstrumental reasons. Although extrinsicmotivation is arguably predominant in aworkcontext, ittoo can takedifferent forms.According to SDT, extrinsic motivation canreflectadesiretogainrewardsoravoidpun-ishment (external regulation), boost one’sego or avoid feelings of guilt (introjection),attainavaluedpersonalgoal(identification),or express one’s sense of self (integration).Identification and integration involve a highlevel of volition and, along with intrinsicmotivation, are considered forms of autono-mous regulation. External regulation andintrojectioninvolvemore external influenceand less authenticity and are consideredforms of controlledregulation. Autonomousregulation, which is also at the heart ofSheldon’s concept of self-concordance andVallerand’s characterization of harmoniouspassion, has been demonstrated to lead tohigher levels of performance, persistence,initiative,andcreativity(Ryan&Deci,2000).The concept of autonomous regulationoverlaps considerably with Macey andSchneider’s conceptualization of state en-gagement.Moreover,thebehavioraloutcomesfoundtobeassociatedwithautonomousreg-ulationcorrespondwith what they describedas behavioral engagement.Incontrastto

Gender Stereotypes Are Alive, Well, and Busy Producing Workplace Discrimination

Abstract: Acriticalexaminationofresearchontherelationshipbetweenstereotypingandworkplace discrimination must meet three requirementsThefirstrequirementisanunderstanding of the theory that guides this research The second requirement is anunbiased review of relevant research The third requirement is comprehension of the ways thatdifferenttypesofresearchareinformative about behavior in organizations Landy (2008) meets none of these requirements He misstates the consensual social scientific theory about the relation between stereotyping and discrimination, presents onlya selective portion of the relevant research, and misconstrues the basis for generalizing researchfindingstoorganizationsAsaresult, Landy misrepresents the evidence for stereotype-based workplace discrimination For brevity, we consider only sex discrimination Also, consistent with Landy’s emphasis, we address the consequences of stereotypes that describe women and men as opposed to stereotypes that prescribe normatively acceptable behavior for them and thus sanction behavior deviating from gender norms (see Eagly & Karau, 2002; Heilman, 2001)

Been There, Bottled That: Are State and Behavioral Work Engagement New and Useful Construct “Wines”?

Abstract: DANIEL A. NEWMANTexas A&M UniversityDAVID A. HARRISONPennsylvania State UniversityMacey and Schneider (2008) have providedan informative treatise on the psychologicalconstruct of employee engagement. Webelieve their conceptual work is a thought-provoking exemplar of how intuitive con-structs (such as engagement) can begin tobe legitimized for the academic audience,hopefully strengthening communicationbetween scientists and practitioners. Theyattempt to specify the meaning of the popu-lar concept vis-a`-vis more long-standingconstructsofjobsatisfaction,organizationalcommitment, job involvement, positiveaffect and affectivity, and proactive and cit-izenship behavior. The engagement label issummarily applied to describe psychologi-cal states, traits, and behaviors. Much effortis spent parsing among established con-structswithregardtotheirpartialconceptualoverlap with the newer label.We offer three reactions to Macey andSchneider’s paper. First, because employeeengagement comprises no new conceptualcontent,butratherablendofoldcontent,itismost appropriately specified as a higherorder latent construct. Second, the utility ofa state engagement construct depends uponevidence for its discriminant validity fromrelated higherorderconstructs,namelyover-alljobattitude.Third,thetermengagement—even as typically used by practitioners andlaypersons—canbeparsimoniouslyconcep-tualized as a second-order factor of widelystudied work behaviors, including focal jobperformance, withdrawal behavior, and citi-zenship behavior. That term and constructhave already been forwarded and expli-catedinpriorwork,includingameta-analyticsummary of attitude–behavior relationshipsinvolving more than 500 original studies(Harrison, Newman, & Roth, 2006). Thisprevious work supported a broad attitude–engagementmodelthat unifiedseveralindi-vidual-level constructs in both the predictorand the criterion spaces and offered analternative view on employee engagement,suggesting that (a) engagement should bedistinguished from job attitudes and (b)engagement can be both intuitivelyand par-simoniously modeled as a higher orderbehavioral construct.Is Engagement a New Construct?Welaudthegeneralgoalofcoalescingideasand evidence into abstractions that can

Explaining the Weak Relationship Between Job Performance and Ratings of Job Performance

Abstract: Ratings of job performance are widely viewed as poor measures of job performance. Three models of the performance–performance rating relationship offer very different explanations and solutions for this seemingly weak relationship. One-factor models suggest that measurement error is the main difference between performance and performance ratings and they offer a simple solution—that is, the correction for attenuation. Multifactor models suggest that the effects of job performance on performance ratings are often masked by a range of systematic nonperformance factors that also influence these ratings. These models suggest isolating and dampening the effects of these nonperformance factors. Mediated models suggest that intentional distortions are a key reason that ratings often fail to reflect ratee performance. These models suggest that raters must be given both the tools and the incentive to perform well as measurement instruments and that systematic efforts to remove the negative consequences of giving honest performance ratings are needed if we hope to use performance ratings as serious measures of job performance.

The Meaning and Bleeding of Employee Engagement: How Muddy Is the Water?

Abstract: If we are to accept engagement as a new and unique concept, it must be distinct from other more established constructs. However, according to Macey and Schneider (2008), engagement behavior is an ‘‘aggregate multidimensional construct.’’ In other words, it is a little bit of this, a little bit of that, some of this, and some of that. For example, they state that behavioral engagement is ‘‘simultaneously citizenship behavior (OCB), role expansion, proactive behavior, and demonstrating personal initiative, all strategically focused in service of organizational objectives’’ (p. 19). Thus, engagement is much more than ‘‘old wine in a new bottle.’’ It is made up of many different wines and spirits all mixed up into a perfect blend—a kind of cocktail construct. However, if the meaning of engagement ‘‘bleeds’’ into so many other more developed constructs, then engagement just becomes an umbrella term for whatever one wants it to be. Macey and Schneider appear to subscribe to the view that engagement is a repackaging of other constructs or the so-called jangle fallacy. However, they take this further than others by describing a trait, state, and behavioral engagement package. Unfortunately, by creating three packages of engagement (trait, state, and behavioral), one is left wondering which of the three should be measured and be the focus of future research. Although they argue that ‘‘identifying these different conceptualizations will help researchers and practitioners have a firmer idea about the locus of the issue when they work with it’’ (p. 6), I believe it will only perpetuate the confusion and inconsistency surrounding the meaning and measurement of engagement. Furthermore, it is questionable whether state engagement is an antecedent that precedes behavioral engagement. In fact, state engagement might occur during and/or after behavioral engagement. At any rate, given that state engagement is an indirect indicator of behavioral engagement, which is observable and ultimately what organizations are most concerned about, the focus and measurement of future research should be behavioral engagement. Macey and Schneider also define engagement in terms of ‘‘discretionary effort or a form of in-role or extra-role effort or behavior’’ (p. 6) that involves adaptive and innovative performance as well as ‘‘behaviors (that) are typically not prescribed and that they go beyond preserving the status quo, and instead focus on initiating or fostering change in the sense of doing something more and/or different’’ (p. 24) and ‘‘going beyond the usual or typical’’ (p. 19). This is problematic for a number of reasons. First, it is not consistent with how engagement has been defined in the academic Correspondence concerning this article should be addressed to Alan M. Saks. E-mail: saks@utsc.utoronto.ca Address: Centre for Industrial Relations and Human Resources, University of Toronto, 121 St. George Street, Toronto, Ontario, Canada M5S 2E8 Alan M. Saks, Centre for Industrial Relations and Human Resources, University of Toronto. Industrial and Organizational Psychology, 1 (2008), 40–43. Copyright a 2008 Society for Industrial and Organizational Psychology. 1754-9426/08

Personality Testing and Industrial–Organizational Psychology: Reflections, Progress, and Prospects

Abstract: As the title suggests, this article takes a broad perspective on personality as it is conceptualized and measured in organizational research, and in the spirit of this Society for Industrial and Organizational Psychology journal, we framed the article as a series of 7 questions. These 7 questions deal with (1) personality and multidimensional models of performance, (2) personality taxonomies and the five-factor model, (3) the effects of situations on personality–performance relationships, (4) the incremental validity of personality over cognitive ability, (5) the need to differentiate personality constructs from personality measures, (6) the concern with faking on personality tests, and (7) the use of personality tests in attempting to address adverse impact. We dovetail these questions with our perspectives and insights in the hope that this will stimulate further discussion with our readership.

Stereotypes, Bias, and Personnel Decisions: Strange and Stranger

Abstract: Research on stereotyping as related to workplace evaluations and decisions has been going on for more than 30 years. Recently, implicit association theory has emerged as a less conscious manifestation of stereotyping mechanisms. In this article, I review the relevance of research on both stereotyping and one of the more popular tests of implicit associations, the Implicit Association Test (IAT). Claims have been made that both stereotyping research and, more recently, IAT research provide theoretical and empirical support for the argument that protected demographic groups (e.g., ethnic minorities, women) are the victims of biased personnel decisions and evaluations. My review of the literature suggests that both stereotyping and IAT research study designs are sufficiently far removed from real work settings as to render them largely useless for drawing inferences about most, but not all, forms of employment discrimination.

Engaged in Engagement: We Are Delighted We Did It

Abstract: We emphasize that science and practice issues are equally salient when pursuing thinking and research on employee engagement. We agree with much of what the commentaries have to say, especially that organizational competitive advantage is the relevant focus of engagement research and practice and that engagement is not a new construct but one that required clarification vis-a-vis existing constructs. We also agree that state engagement can be highly variable, that disengagement needs study, that negative situations can induce engagement behaviors, that engagement surveys should yield actionable data, and that people can be hired who are more likely to be engaged. We disagree with the idea that all employee attitudes are essentially equal and that existing conceptualizations of performance make engagement behavior a nonuseful construct.

Why Assessment Centers Do Not Work the Way They Are Supposed To

Abstract: Assessment centers (ACs) are often designed with the intent of measuring a number of dimensions as they are assessed in various exercises, but after 25 years of research, it is now clear that AC ratings that are completed at the end of each exercise (commonly known as postexercise dimension ratings) substantially reflect the effects of the exercises in which they were completed and not the dimensions they were designed to reflect. This is the crux of the long-standing “construct validity problem” for AC ratings. I review the existing research on AC construct validity and conclude that (a) contrary to previous notions, AC candidate behavior is inherently cross-situationally (i.e., cross-exercise) specific, not cross-situationally consistent as was once thought, (b) assessors rather accurately assess candidate behavior, and (c) these facts should be recognized in the redesign of ACs toward task- or role-based ACs and away from traditional dimension-based ACs.

Defining Employee Engagement for Productive Research and Practice

Abstract: Both the applied and the academic literatures on employee engagement are unfortunately in a state of disarray. In two important areas of their otherwise admirable attempt to demarcate the construct space of employee engagement, Macey and Schneider (2008) may inadvertently have contributed to the muddle. The first is the manner in which Macey and Schneider conceptualize the psychological state of engagement, and the second is their use of the term engagement as a rubric that encapsulates not only cognitive–affective but also dispositional and behavioral constructs. Clarifying these points will allow for a better definition of employee engagement. This in turn will provide a firmer basis for future research and practice in the area.

Transforming Our Models of Learning and Development: Web‐Based Instruction as Enabler of Third‐Generation Instruction

Abstract: In this paper, I suggest that our methods for how to train are based historically on what we understand learning to be. I briefly trace the history of instructional design models, which embody both first-generation objectivist and second-generation cognitive constructivist views of learning. I then suggest we are on the cusp of a third-generation instructional model, one that places greater emphasis on the learner forming understanding largely through a process of social negotiation, either in training or on the job. I argue that Web-based instruction offers a technology ideally suited to the social constructivist approach at the core of this third-generation model. I discuss the importance of interaction in learning and suggest why Web-based networked learning may result in more, not less, interaction during training. Finally, I discuss other applications of third-generation learning such as peer mentoring and professional forums.

Conceptual Versus Empirical Distinctions Among Constructs: Implications for Discriminant Validity

Abstract: In recent years, the term ‘‘employee engagement’’ has taken on a life of its own, probably based, in part, on its resonance as a term describing a deeper level of involvement and enthusiasm from the employee than the terms ‘‘job satisfaction’’ or ‘‘organizational commitment’’ might imply. The newer emphasis on absorption, passion, and affect may, in fact, better reflect the reason work attitudes matter to organizations. Metaanalyses show that job attitudes substantially relate to individual-level performance (Judge, Thoresen, Bono, & Patton, 2001) and business unit–level outcomes such as profit, sales, customer ratings, accidents, and turnover (Harter, Schmidt, & Hayes, 2002; Harter, Schmidt, Killham, & Asplund, 2007). These practically important relationships are likely a reflection of differences in the level of cognitive and emotional involvement and enthusiasm employees have for their work and companies, leading to discretionary effort, which in turn leads to a host of positive business outcomes. A key question is whether the newer constructs of engagement have discriminant validity relative to the older constructs of job satisfaction and organizational commitment that have been studied for decades. Macey and Schneider’s paper is based on an implicit assumption that may not hold true: If they and other researchers can make a logical or conceptual distinction between constructs or measures, then this distinction will exist in the minds of employees or respondents to surveys. The area of jobrelated attitudes may be an excellent example of the kind of construct proliferation that can cause industrial–organizational (I–O) psychology and organizational behavior to be major violators of the mandate for parsimony in science. The ability of I–O psychologists to create a wide variety of conceptual distinctions—many quite subtle—leading to a proliferation of proposed constructs (the jangle fallacy cited by Macey and Schneider [2008]) seems quite expansive. Macey and Schneider’s paper is a well-intentioned attempt to deal with this problem but is based on an implicit assumption that substantial new evidence suggests is false. Advanced measurement methods can now serve as critical theoretical tools to determine whether constructs that can be argued to be conceptually distinct are in fact empirically distinct. If they are empirically distinct, then they will correlate considerably less than 1.00 after the appropriate Correspondence concerning this article should be addressed to James K. Harter. E-mail: jim_harter@ gallup.com Address: The Gallup Organization, 1001 Gallup Drive, Omaha, NE 68103 James K. Harter, Workplace Management, The Gallup Organization; Frank L. Schmidt, Tippie College of Business, University of Iowa. Industrial and Organizational Psychology, 1 (2008), 36–39. Copyright a 2008 Society for Industrial and Organizational Psychology. 1754-9426/08

International Perspectives on the Legal Environment for Selection

Abstract: Perspectives from 22 countries on aspects of the legal environment for selection are presented in this article. Issues addressed include (a) whether there are racial/ethnic/religious subgroups viewed as “disadvantaged,” (b) whether research documents mean differences between groups on individual difference measures relevant to job performance, (c) whether there are laws prohibiting discrimination against specific groups, (d) the evidence required to make and refute a claim of discrimination, (e) the consequences of violation of the laws, (f) whether particular selection methods are limited or banned, (g) whether preferential treatment of members of disadvantaged groups is permitted, and (h) whether the practice of industrial and organizational psychology has been affected by the legal environment.

The Failure of Social Desirability Measures to Capture Applicant Faking Behavior

Abstract: The use of social desirability (SD) measures to examine the issue of applicant faking has a long but rather unproductive history. Indeed, in the focal article, Hough and Oswald (2008) state that response bias (or faking), as measured by these scales, does not affect the validity of personality measures. The list of relevant studies supporting this assertion is far too long to detail here (many are cited in the focal article), but most have resulted in conclusions indicating that when measures of SD are used as a proxy variable for applicant faking behavior, faking does not appear to substantially affect criterion-related validity nor do score corrections improve the accuracy of selection decisions. Such research is often cited as evidence that applicant faking is not a concern for practitioners utilizing personality tests, and studies echoing this sentiment have been influential in both research and practice. Although these findings appear to allay concerns regarding applicant faking, they rest on an important, albeit infrequently investigated, assumption. In order for measures of SD to be useful in research and practice, one must assume that they are reliable andvalidmeasures of the degree towhich an individual fakes his or her responses onapersonality measure. In most studies utilizing SD measures, this assumption has not been tested, and little empirical evidence has supported the linkage between SD measures and actual faking behavior (i.e., observed within-subjects score changes across response settings that vary in their influence on respondents’ motivation to fake). However, in the focal article, the authors state that ‘‘Social Desirability scales, aka Unlikely Virtues scales, detect intentional distortion’’ (p. 26). Furthermore, in a recent simulation examining the effects of SD corrections on validity and hiring decisions (Schmitt & Oswald, 2006), the researchers stated, ‘‘Our analyses implicitly assume that the faking measure is a perfectly construct-valid measure’’ (p. 616). Although some evidence of this may be found in directed faking studies, virtually no studies have linked SD with faking in an applicant setting. Given that much of what we think we know about applicant faking is based on studies using SD as an indicator of faking, further examination of this assumption is in order.

Employee Engagement at the Organizational Level of Analysis

Abstract: Macey and Schneider (2008) frame the construct of employee engagement at the individual level of analysis, but much of the interest in the practitioner and consulting communities (e.g., Buchanan,2004; Jamrog, 2004) and several influential academic studies in this domain (e.g., Harter, Schmidt, & Hayes, 2002; Salanova, Agut, & Peiro, 2005) are framed and conducted at the organizational level. Macey and Schneider leave the level of analysis issue open but do suggest that adding additional levels of analysis to the research repertoireonemployeeengagementwouldbe a fruitful direction for future research. Because research and practice are already moving in this direction (perhaps ahead of solid theory development), we provide below a brief rationale for why it may be valuable to conceptualize engagement at the organizational level of analysis and offer a few suggestions for how researchers may want to proceed.

The Word Is Out: We Need an Active Performance Concept for Modern Workplaces

Abstract: It is high time that somebody summarizes and evaluates the research on engagement to which a number of authors have contributed under different labels. In the following, I shall not differentiate much between engagement, personal initiative, proactivity, taking charge, and voice because these are very similar concepts. The definition that Macey and Schneider (2008) tend to agree with—engagement being related to vigor, dedication, and absorption (Maslach, Schaufeli,& Leiter, 2001)—is similar to other concepts developed in the general domain of active performance. I shall discuss two issues: the necessity of developing an active performance concept of which engagement is one and the function of positive affectivity for engagement.

Leveraging Employee Engagement: The Practical Implications

Abstract: Organizational researchers have long recognized that organizations cannot function through purely contractual relationships with employees. The difficulty of specifying contracts covering all possible contingencies is a common explanation for the reliance on organizations, rather than markets, to carry out various activities (Williamson, 1975). By tying employers and employees together in ongoing employment relationships, organizations are presumed to align the interests of individuals with those of the firm such that employees can be counted on to act more frequently in ways that are consistent with corporate objectives. In the early 20th century, Chester Barnard, EltonMayo, and others similarly emphasized that organizations require cooperation from employees rather than mere compliance. Perfunctory adherence to minimal role requirements is likely to have dysfunctional consequences in most settings. Unionized employees who ‘‘work to rule’’ in contract disputes, for instance, can quickly bring organizations to their knees. As a consequence, fostering high (or even adequate) individual performance necessarily requires attending to employees’ motivations as well as their behaviors. In our view, the engagement construct has captured the attention of managers insofar as it raises the notion of cooperation to a higher level, made all the more important in the context of growing competitive pressures and more rapid change. Although cooperation is required and to some extent, expected of all employees, engagement, as Macey and Schneider (2008) point out, involves going above and beyond the typical in-role performance. In that sense, engagement holds out to organizational leaders the prospect of increasing productivity (i.e., getting more output from a finite set of human capital resources). In an environment of increasing global competition, where organizations are running ‘‘leaner’’ and forced to domorewith less, tapping into the discretionary effort offered by engaged employees becomes all the more imperative for success in the marketplace. In other words, state engagement, as Macey and Schneider define, is essential in competitive environments in which margins are pressured and productivity is critical, insofar as Correspondence concerning this article should be addressed to Mark A. Royal. E-mail: mark.royal@ haygroup.com Address: Hay Group, 111 E. Wacker Drive, Suite 1250, Chicago, IL 60601 Rebecca C. Masson, Mark A. Royal, TomG. Agnew, and Saul Fine, Hay Group. Industrial and Organizational Psychology, 1 (2008), 56–59. Copyright a 2008 Society for Industrial and Organizational Psychology. 1754-9426/08

Is Behavioral Engagement a Distinct and Useful Construct

Abstract: SHARON K. PARKERUniversity of SheffieldANDREW NEALUniversity of QueenslandIt is not surprising that employee engage-ment is a popular idea. A workplace whereenthusiastic team members devote extraeffort to innovation, cooperate with eachother, and effectively adapt to change is anenticing picture for managers and employ-ees alike. It makes sense to describe suchemployees as engaged with their work. Butis it useful to propose that there is a distinctform of a behavior—termed ‘‘behavioralengagement’’ by Macey and Schneider(2008)—that can and should be distin-guished from other forms of behavior? Wesuggest not.Over the past 20 years, there has beena proliferation of partially overlapping con-structs that seek to explain different forms ofwork behavior that are important for organi-zational success. Although the specificbehaviors described by these constructs areimportant, researchers have noted the lackof a framework for integrating the diverserange of performance constructs that nowexist (Rotundo & Sackett, 2002). Macey andSchneider introduce ‘‘behavioral engage-ment’’ as another such construct. Unfortu-nately, we believe this approach will addto the number of performance constructswithout overcoming the barriers to a betterconceptualintegrationamongtheconstructs.Concerns With BehavioralEngagementOurfirstandgeneralconcernisthatthecon-cept of behavioral engagement implies thata particular motivational process (engage-ment) underpins a particular set of behav-iors. This connection is problematic. Anemployee might display innovation, whichthe authors consider a facet of behavioralengagement, not because they feel engagedbut because they fear redundancy and wantto prove their capability. Conversely, anemployee might fail to show innovation,notbecausetheyareunengagedbutbecauseconstraints in the environment inhibit be-havior.Becauseallbehaviorsaremultideter-mined, it is not possible to link a specificformofbehaviorwithaspecificmotivationalstate.This point is illustrated by Macey andSchneider’s example describing some situa-tions where normal task behavior is de-fined as behavioral engagement and otherwhere it is not depending on the level ofdemand in the context (p. 25). The problemhere is that the context determines whetherpsychological engagement is inferred to

What if we took within-person performance variability seriously?

Abstract: This Journal Article is brought to you by the Bond Business School at ePublications@bond. It has been accepted for inclusion in Bond Business School Publications by an authorized administrator of ePublications@bond. For more information, please contact Bond University's Repository Coordinator. Recommended Citation Cynthia D. Fisher. (2008) \"What if we took within-person performance variability seriously?\" ,, .

Triangulation Across Methodologies: All Signs Point to Persistent Stereotyping and Discrimination in Organizations

Abstract: Landy (2008) claims that (a) field-based evidence of discrimination is scarce and (b) laboratory-based research is ‘‘largely useless’’ (p. 379) for understanding real-world organizational contexts. He therefore concludes that we lack sufficient evidence that stereotyping and discrimination influence organizational decision making. We strongly disagree with Landy’s conclusion and show that a large, multimethod, multidisciplinary body of research documents persistent discrimination in organizations. We counter Landy’s first claim by providing several examples of field-based research that document discrimination in real-world organizational contexts but were overlooked entirely by Landy. We then counter Landy’s assertion that laboratory findings do not generalize by highlighting the imperfect assumption upon which Landy bases this claim. We conclude that triangulated evidence—spanning both laboratory and field settings—documents the unfortunate reality that discrimination is alive and well in organizations.

Making Assessment Centers Work the Way They Are Supposed To

Abstract: Despite more than 2 decades of research showing that assessment center ratings vary more by exercise than by dimension, Lance’s (2008) plea to refocus assessment centers on tasks rather than dimensions has gained little traction. Most practitioners (and many academics) have stubbornly resisted a task approach as a blind alley. That apparent contradiction inspired this commentary. The problem lies not just with the recommended action but with misguided approaches to assessment center research and practice. The problem areas include (a) questionable theory and models underlying the experimental tests, (b) misinterpretation and/or misuse of dimensions, (c) misunderstanding the practical uses of assessment centers, and (d) a simplistic and outdated view of assessment center design. Lance’s paper shines a light on these long-standing issues as he leads us to what seem on the surface to be logical conclusions and recommendations. What I hope to show is that the assumptions on which the evidence and logic rest are shaky and cannot support the recommendations. I then offer alternative recommendations for assessment center research and practice. ‘‘Traditional’’ Assessment Center Theory and Models

Mend It, Don't End It: An Alternate View of Assessment Center Construct-Related Validity Evidence

Abstract: The unitarian conceptualization of validity serves as the conceptual and logical basis for the so-called assessment center (AC) ‘‘construct-related validity paradox.’’ Within the unitarian framework, at a theoretical level, if a measurement tool demonstrates criterion-related and content-related validity evidence, as is widely accepted with ACs, then it should also be expected to demonstrate construct-related validity evidence (Binning & Barrett, 1989). And because ACs do not appear to do so, we have the resultant AC construct-related validity paradox. So, accepting the premise that the unitarian view is conceptually and logically sound, what is the explanation for the paradox? Why do AC dimension ratings appear not to ‘‘work’’ in terms of construct-related validity evidence? At a broad conceptual level, we present a view that is contrarian to Lance’s (2008) view of ‘‘why ACs don’t work the way they’re supposed to’’ and subsequently what to do about them. Our contrarian view is based on two key points, namely that the vast majority of the empirical AC research to date—particularly that which serves as the basis for calls for the ‘‘redesign of ACs toward taskor role-based ACs and away from traditional dimensionbased ACs’’ (Lance, 2008, p. 84)—is based on (a) espoused as opposed to actual constructs and (b) flawed analysis resulting from an overemphasis on postexercise dimension ratings as measures of AC dimensions. In our view, ACs in practice appear to be effectively designed to representatively sample from the job content domain and also predict criteria of interest, but they are woefully deficient in their construct explication and development. Consequently, we do not concur with Lance’s interpretation of the extant literature and his conclusions concerning what to do about it. Our position is that the issue is not one of a failure in ‘‘AC theory’’ but rather a failure to engage in appropriate tests of said theory. Until such tests have been undertaken, we think it is premature to abandon Correspondence concerning this article should be addressed to Winfred Arthur, Jr. E-mail: wea@psyc. tamu.edu Address: Department of Psychology, Texas AM Eric Anthony Day, Department of Psychology, The University of Oklahoma; David J. Woehr, Department of Management, The Universityof Tennessee. Industrial and Organizational Psychology, 1 (2008), 105–111. Copyright a 2008 Society for Industrial and Organizational Psychology. 1754-9426/08

What Does Exercise-Based Assessment Really Mean?

Abstract: My commentary addresses Lance’s (2008) recommendation to reorient assessment center (AC) practice away from dimensions toward exercise-based assessment. As exercise-based assessment is dealt with only in general terms in Lance’s article, I aim to delineate what exercise-based assessment really means. Two points are made. First, I argue that taking dimensions away from ACs does not mean that assessee behavior is no longer determined by latent traits because behavior is inherently trait determined. Second, I elaborate on the practical and research implications of exercise-based assessment because these implications are underdeveloped in Lance.

Some New (and Old) Suggestions for Improving Personnel Selection

Abstract: The thoughtful focal article by Highhouse (2008) documents our pervasive, unchanging preferences for demonstrably inferior data combination and assessment methods. Indeed, Grove and Meehl (1996) noted that ‘‘We know of no social science controversy for which the empirical studies are so numerous, varied, and consistent as this one’’ (p. 25). Given the weight of the evidence and the importance of making the best hiring and admissions decisions, this is a topic that warrants serious attention and effort. Highhouse has presented two related but distinct issues. The first is the frequent use of less effective predictors by decision makers (e.g., unstructured interviews). This proclivity seems to be the result of both preference and lack of knowledge. The second issue is a consistent preference for human judgment in data combination over mechanical or statistical methods. Solutions to these problems are not likely to be simple. People appear to be equipped with a set of often useful but sometimes faulty heuristics for reasoning and judging evidence that result in all sorts of mischief (Tversky & Kahneman, 1974). This includes a preference for hands-on experience and reactivity to narratives, emotional appeals, and the judgment of trusted experts. Solutions are further hampered by a real dearth of research on user reactions to selection tools and systems (as noted by Highhouse). However, some suggestions can be made. We need techniques that will increase the appeal of the more effective predictors to address the first issue. For the second issue, we need to either improve acceptance of more effective data combination methods or create selection systems that preserve a strong element of human judgment while addressing limitations in human information processing.

The Construct Validity of the Assessment Center Method and Usefulness of Dimensions as Focal Constructs

Abstract: In this paper, we offer a perspective that differs from that endorsed by Lance (2008). Although we agree with some of Lance’s assertions, particularly with regard to the effectiveness of well-trained assessors and the inappropriateness of the multitrait–multimethod (MTMM) approach for establishing construct validity of assessment center (AC) ratings, we disagree that the evidence he provides supports the conclusion that ACs lack validity to measure intended constructs. In contrast, we posit that the evidence clearly confirms that the AC method does in fact possess validity to assess and develop job-related performance dimensions. We bolster this proposition with four main arguments, which we articulate below.

Lessons Learned in Transitioning Personality Measures From Research to Operational Settings

Abstract: Hough and Oswald have acknowledged the major contribution of the U.S. Army’s Project A to our understanding of personnel selection within the field of industrial–organizational psychology. Results from validation of the Assessment of Background and Life Experiences (ABLE) developed in Project A provided strong evidence of the utility of personality constructs for predicting important aspects of military performance. Since Project A, the U.S. Army Research Institute for theBehavioralandSocial Sciences (ARI) has conducted much research on the use of personality measures for personnel selection and classification decisions. Our commentary will focus primarily on the use of personality measures for highstakes, Army applicant screening. Much of this research has involved either the ABLE or the Assessment of Individual Motivation (AIM), both self-report personality measures assessing constructs that overlap with the Big Five. With respect to item format and transparency, ABLE is similar to many personality measures that are widely used today. AIM uses a forced-choice approach to help reduce concerns regarding fakability. Without question, the ‘‘faking problem’’ has been one of the greatest challenges to the Army’s ability to implement and sustain the operational, large-scale use of self-report personality measures, especially in highstakes testing situations. Our focus on this issue began during Project A when promising findings resulted in the ABLE being seriously considered for use in high-stakes, Army applicant screening. However, due primarily to concerns about its susceptibility to faking and coaching, ABLE was never used operationally by the Army for applicant screening. Today—20 years later—the Army is having some success in using personality measures for making real-life personnel selection and assignment decisions. In the spirit of Hough Correspondence concerning this article should be addressed to Leonard A. White. E-mail: len.white@ cox.net Address: U.S. Army Research Institute for the Behavioral and Social Sciences, 2511 Jefferson Davis Highway, Arlington, VA 22202-3926. Leonard A. White and Mark C. Young, U.S. Army Research Institute for the Behavioral and Social Sciences; Arwen E. Hunter, The George Washington University; Michael G. Rumsey, U.S. Army Research Institute for the Behavioral and Social Sciences. All statements expressed in this commentary are those of the authors and do not necessarily reflect the official opinions of the U.S. Army Research Institute or the Department of the Army. Industrial and Organizational Psychology, 1 (2008), 291–295. Copyright a 2008 No claim to original US government works. 1754-9426/08

Representations of Trait Engagement: Integration, Additions, and Mechanisms

Abstract: Macey and Schneider (2008) proposed (Proposition 11) that trait engagement encompasses at least several personalitybased constructs to include an autotelic personality, trait positive affectivity, proactive personality, and conscientiousness. The first main point of our commentary is that these personality-based constructs have an underlying commonality, not explicated by Macey and Schneider, in that they all embody differences among individuals in their propensity to exercise human agency. The metatheoretical concept of human agency is a key premise of social-cognitive theory (Bandura, 2001). It describes the ability of people to exercise control over their own thoughts and intentions, which enables people to actively shape their present circumstances in ways that facilitate the attainment of subsequent outcomes theydesire.Nevertheless, although human agency is possessed by everyone, there is individual variation in the extent to which it is characteristically exhibited. The notion of human agency offers a basis for understanding why the personality-based constructs identifiedbyMaceyandSchneider have relevance for employee engagement. First, Csikszentmihalyi’s (1990) notion of an autotelicpersonality represents a generalpropensity to mentally transform potential threats into enjoyable challenges. People with an autotelic personality set challenging goals for themselves, become actively involved in endeavors that promote goal accomplishment and skill development, seek developmental feedback, sustain a focus on task performance, and enjoy the state of being mentally involved. Second, trait positive affectivity entails a proclivity for active interaction with one’s environment (Staw, 2004). Such a propensity would seem to directly facilitate agency, which is defined as ‘‘the state of being active, usually in the service of a goal, and of exerting power or influence’’ (VandenBos, 2007, p. 29). Third, the construct ofproactive personality encompasses consistently taking action to change things for the better and persisting to overcome opposition in doing so (Bateman & Crant, 1993). This construct seems virtually tantamount to an agentic orientation, which is ‘‘an emphasis on achieving, doing, succeeding, and making one’s own mark in the world, which may be expressed through such traits as competitiveness and self-focus’’ (VandenBos, 2007, p. 29). Correspondence concerning this article should be addressed to Robert R. Hirschfeld. E-mail: rrhirschfeld@ gmail.com Address: Department of Management and Quantitative Methods, College of Business and Administration, University of Colorado at Colorado Springs, 1420 Austin Bluffs Parkway, Colorado Springs, CO 80933-7150 Robert R. Hirschfeld, Department of Management and Quantitative Methods, College of Business and Administration, University of Colorado at Colorado Springs; Christopher H. Thomas, Department of Management, Northern Illinois University. Industrial and Organizational Psychology, 1 (2008), 63–66. Copyright a 2008 Society for Industrial and Organizational Psychology. 1754-9426/08

Back to Basics of Test Construction and Scoring

Abstract: I want to make two main points in response to Lance’s (2008) paper. First, we should pay more attention to the psychometrics of our simulations, particularly the reliability of exercise scores related to candidate actions. Poor reliability limits convergent validity and undermines the quality of feedback given to candidates. Second, I suggest that a main reason for the troubling multitrait–multimethod (MTMM) results is a mismatch between the inferences to be made based on the scores and the construction of the exercises. Construct validity evidence is poor because the exercises are based on tasks sampled for content rather than chosen or designed for illuminating individual differences on the constructs.