Showing papers in "Personnel Psychology in 1993"

Relations between work group characteristics and effectiveness: implications for designing effective work groups

Abstract: Five common themes were derived from the literature on effective work groups, and then characteristics representing the themes were related to effectivness criteria. Themes included job design, interdependence, composition, context, and process. They contained 19 group characteristics which were assessed by employees and managers. Effectiveness criteria included productivity, employee satisfaction, and manager judgments. Data were collected from 391 employees, 70 managers, and archival records for 80 work groups in a financial organization. Results showed that all three effectiveness criteria were predicted by the characteristics, and nearly all characteristics predicted some of the effectiveness criteria. The job design and process themes were slightly more predictive than the interdependence, composition, and context themes. Implications for designing effective work groups were discussed, and a 54-item measure of the 19 characteristics was presented for future research.

Problems with the use of profile similarity indices in the study of congruence in organizational research.

Abstract: Profile similarity indices (PSIs) have become widely used in studies of congruence (i.e., fit, matching, similarity, agreement) in organizational research. PSIs combine two sets of measures, or profiles, from corresponding entities (e.g., the person and organization, supervisor and subordinate, organization and environment) into a single score intended to represent their overall congruence. Unfortunately, PSIs are conceptually ambiguous, discard information essential to testing congruence hypotheses, conceal the source of the difference between entities, and impose a highly restrictive set of constraints on the coefficients relating the measures comprising the PSI to the outcome. This article shows how polynomial regression analysis may be used to avoid problems with PSIs while capturing the underlying relationships PSIs are intended to represent. Limitations and extensions to the procedure are discussed.

Constraints on the adoption of psychology‐based personnel practices: lessons from organizational innovation

Abstract: Surveys of organizational personnel practices often indicate that techniques advocated by industrial and organizational (I/O) psychologists are used with less frequency than might be expected given their technical merit. This article attempts to explain this phenomenon by viewing the adoption of I/O-type personnel practices as organizational innovations that are subject to the mechanisms and processes described in the innovation-diffusion literature. It is argued that the adoption of I/O-type personnel practices constitutes administrative innovation and that such innovation is not strongly influenced by technical merit. Rather, imitation processes, environmental threat, government regulation, and political influence often dominate highly uncertain adoption processes. Recommendations are made for enhancing the adoption rate for psychology-based personnel innovations.

Using signal‐contingent experience sampling methodology to study work in the field: a discussion and illustration examining task perceptions and mood

Abstract: In this article, the advantages and drawbacks of using Experience Sampling Methodology (ESM), especially signal-contingent ESM, to study psychological variables in the workplace are described It is argued that ESM can be a valuable tool in the field study of work, with the potential to answer different research questions than can be answered by traditional cross-sectional research To illustrate ESM, an application of signal-contingent ESM to the examination of mood and concurrent task perceptions at work is presented Study results indicate that both perceived goal progress and skill on task influence task enjoyment and mood Also described and illustrated is an adaptation of pooled time-series hierarchical regression for analyzing immediate, short-term retrospective (end-of-day), and individual difference variables in a single framework In conjunction, ESM data collection and hierarchical regression analysis provide a useful approach for studies that examine immediate work experiences

Beyond formal experimental design: towards an expanded view of the training evaluation process

Abstract: Textbook treatments of training evaluation typically equate evaluation with the measurement of change and focus on formal experimental design as the mechanism for controlling threats to the inference that the training intervention produced whatever change was observed. This paper notes that two separate questions may be of interest: How much change has occurred? and, Has a target performance level been reached? We show that the evaluation mechanisms needed to answer the two types of questions are markedly different, and discuss circumstances under which an evaluator's interests will focus on one, the other, or both of these questions. We then discuss alternatives to formal design as mechanisms for reducing various threats to validity, and discuss trade-offs between internal validity and statistical conclusion validity.

Detecting moderators with meta‐analysis: an evaluation and comparison of techniques

Abstract: The present study evaluated accuracy levels of seven techniques for ascertaining, after a meta-analysis, whether moderators are present or not: (a) SH-75% rule for uncorrected r, (b) SH-75% rule for corrected r, (c) SH-95% rule for uncorrected r, (d) SH-95% rule for corrected r, (e) the Q statistic; (f) inclusion of 0 in the credibility interval around the corrected r, and (g) the size of the interval. Using Monte Carlo data which were defined by various parameters including sample based artifacts, comparisons of Type I and power determinations were generated. Findings showed that when differences between population correlations were small, power levels for all techniques were relatively low. Overall, SH rules and the Q statistic had greater power but higher Type I error rate than credibility intervals. Because of the high Type I error rate associated with both of the SH-95% techniques and the low power found with the credibility intervals, the SH-75% rules and Q statistic are recommended. Limitations and some practical implications for the findings are discussed.

An application of the theory of neural computation to the prediction of workplace behavior: an illustration and assessment of network analysis

Abstract: Data from two field studies were used to illustrate four applications of neural network analysis, a statistical information technology based on the psychological theories of learning and cognition. Overall results showed that cross-validation multivariate correlations and correct classifications were greater for the neural networks than for the regression model standards of comparison. It was concluded that sufficient evidence exists to warrant further research of the newly developing neural network paradigm for use by personnel scientists and specialists.

A simple and reliable method of scoring the thurstone attitude scales

Abstract: A simple method of scoring the Thurstone Attitude Scales is presented, which does not involve the use of a judging group and yet is found in several samples to be consistently more reliable than the original method of scoring. The scores obtained by the two methods correlate highly (median r= .88), indicating that they are measuring essentially the same thing.

Beyond the mean bias: the effect of warning against faking on biodata item variances

Abstract: We studied the effects of faking biodata test items by randomly warning 214 of 429 applicants for a nurse's assistant position against faking. While the warning mitigated the propensity to fake, the specific warning effects depended on item transparency. For transparent items, warning reduced the extremeness of item means and increased item variances. For nontransparent items, warning did not have an effect on item means and reduced item variances. These faking effects were best predicted when transparency was operationalized in terms of item-specific job desirability in addition to the item-general social desirability. We also demonstrated a psychometric principle: The effect of warning on means at the item level is preserved in scales constructed from those items, but the effect on variances at the item level is masked at the scale level. These results raise new questions regarding the attenuating effects of faking on validity, and regarding the benefit of warning applicants against faking.

Article review checklist: a criterion checklist for reviewing research articles in applied psychology

Abstract: Over the past couple of years, a large group of reviewers and I have been developing a comprehensive checklist of criteria for reviewing research articles. The purpose of this effort was to provide a heuristic device of issues to think about when reviewing an article. As such, we hoped that the checklist might be a useful tool for reviewers, authors, and students. A two-part Delphi-like procedure was followed. In the first part, a preliminary checklist of criteria was developed and circulated to a large number of reviewers. It contained 93 criteria and was divided into 14 categories (e.g., literature review, sample, measures, procedures, analyses , conclusions, etc.). Reviewers were asked to examine the list and think about what they look for when reviewing an article, and then to modify the criteria on the list or add criteria to it. They suggested 860 additional items and 429 modifications to the original criteria. They were also asked to send in any existing unpublished checklists they might have, and several were obtained containing 135 more items. As the criteria were edited and condensed, special effort was made to incorporate all the content and much of the specific wording of the reviewers' suggestions. The resulting checklist had 246 criteria divided into 16 categories. In the second part of the study, the revised checklist was again circulated to the group of reviewers. This time they were asked to rate each criterion in terms of the weighting it should receive when reviewing an article , and they were asked about their background and experience.^ The information was used to develop the final version of the checklist. First, approximately 9% of the criteria were eliminated because they were relatively unimportant (e.g., received low ratings) or were ambiguous (e.g., many ratings left blank). Second, the checklist was simplified by grouping the criteria within each category into clusters of similar criteria. Finally , within each cluster, the criteria were listed in a very gross rank ordering of importance based on the ratings. The final checklist contained 223 criteria, 44 clusters, and 15 categories. Special thanks to the reviewers who not only provided the ideas and data for this checklist, but who also labor tirelessly with little recognition to make the review process work. This information was used to explore differences between reviewers on the weightings they assigned to the criteria. These data will be reported elsewhere.

Effects of previous job exposure and subsequent job status on the functioning of a realistic job preview

Abstract: In a longitudinal field experiment, applicants for a correctional officer position were randomly assigned to two groups. One group was exposed to a videotaped realistic job preview designed to lower expectations, the second group was not exposed to the preview. Participants also completed attitudinal and descriptive survey measures at three points in time. Among the qualified applicants who were offered employment (n = 1,117), 358 accepted positions as correctional officers. The preview resulted in a lower rate of job acceptance among applicants with previous exposure to the job and a higher rate of acceptance among applicants with no previous job exposure (p .05) during a probationary employment period, and significantly higher (p<.05) after the probationary period. Results of the survey measures suggested that persons may have interpreted the realistic information differently depending upon their prior exposure to the job.

Rainforest empiricism and quasi‘-rationality: two approaches to objective biodata

Abstract: TWo approaches to objective biodata, designed to achieve the interpretability and stability of rational approaches, yet minimize socially desirable responding, were explored. The first was a quasi-rational attempt to derive biodata analogs to an existing temperament measure, and then use them as rational scales. The second was a theorybased variant of criterion-referenced keying, termed rainforest empiricism. Both were utilized with two consecutive classes of U.S. Military Academy cadets (n= 2,565) to predict leadership performance over four rating periods. The biodata analogs to the temperament measure added incremental validity over the Academy's current admissions package and had significantly smaller correlations with a social desirability scale than their equivalent temperament scales. Scales developed with the rainforest approach had higher incremental validities and lower social desirability. Both methods demonstrated satisfactory stability upon cross-validation, and provided complementary interpretability. Advantages to each approach, and the implications for their use, are discussed.

Comparability of incumbent and applicant samples for the development of biodata keys: the influence of social desirability

Abstract: An index of Socially Desirable Responding (SDR) was developed to measure the extent of impression management exhibited in applicant and incumbent samples when responding to a biodata form. The sample consisted of 2,262 incumbent sales representatives and 2,726 applicants for sales positions. Greater applicant versus incumbent SDR was observed, but differences varied across a priori item content areas. Impression management was minimal in item categories such as Previous Work Experience and Economic Motivation, but it was more prevalent in categories such as Work Style and Preferences and Self-Evaluations of Prior Sales Success. Using a smaller sample of 810 incumbents and 555 applicants, largely equated for experience, an item-keyed biodata inventory was developed for selection. When regression procedures were used to develop final keys, no comparable items existed across the keys from the two samples. SDR was more highly related to the applicant key than to the incumbent key. Results for option-keyed instruments developed and validated on the same samples were compared with the results associated with the item-keyed instruments, and the conclusions were similar. Implications for the development of biodata forms for selection are discussed.

How definitive are conclusions based on survey data: estimating robustness to nonresponse

Abstract: The problem of nonresponse poses a threat to the external validity of survey conclusions. In this article, procedures are developed to estimate the robustness of survey conclusions to nonresponse by drawing an analogy between studies unavailable for meta-analysis and nonrespondents in a survey. We extended the procedure to assess the effects of systematic differences between respondents and nonrespondents on t- and F-values used to test group differences. Steps to construct robustness tables that are independent of sample size, survey length, and scale width are outlined to determine how definitive conclusions are from survey data.

“extracting” policy from judicial opinions: the dangers of policy capturing in a field setting

Abstract: Policy-capturing techniques have been used to assess factors thought to influence judicial decisions relevant to human resource management. This article explains why the “advice” offered human resource management practitioners and others based on these studies can be incomplete and misleading. Substantive knowledge needed to independently evaluate research involving judicial decision making in field settings is provided. Five dangers associated with the use of policy capturing in the context indicated are identified, discussed, and then illustrated. Alternative research investigating the role of legal inputs in HRM using lawyers as subject matter experts is suggested.

Expanding the context of utility: the strategic impact of personnel selection

Abstract: Others have demonstrated that traditional applications of the Brogden-Cronbach-Gleser (BCG) selection utility formula are deficient in responding to the financial context of managerial decisions (Boudreau, 1983a, 1983b; Cronshaw & Alexander, 1985, 1991). We demonstrate that traditional estimates of selection utility also fail to reflect the strategic context faced by managerial decision makers. We modify the traditional BCG model to yield an estimate of total utility derived from human resources (Utotal) that can be directly compared to firms' strategic need at a particular point in time (Utarget)-Further, we demonstrate that, while strategic need is rarely constant over time, the capacity of a selection system to meet that need is also likely to change as rxy and SDy change over time. Re-examination of what is important to strategic human resource decision makers (selection utility vs. total utility and strategic need) and changing selection system contributions over time yields a more realistic view of how firms benefit from personnel selection.

Item bias indices based on total test score and job performance estimates of ability

Abstract: Researchers (e.g., Ironson, 1982; Tenopyr, 1990) have suggested that item bias investigators equate subgroups on external criteria such as job performance rather than total test scores before considering subgroup passing rates on test items. In a study comparing these two approaches to studies of item bias, we found little evidence of bias using total test score as the estimate of overall examinee ability, but nearly all items were biased in comparisons of white and African-American subgroups on Numerical, Verbal, and Mechanical Reasoning tests and in male-female comparisons on a Mechanical Reasoning test when job performance was used to select “equally able” examinees. However, the use of job performance as the ability index is analogous to performance-based approaches to test bias (Hartigan & Wigdor, 1989; Thorndike, 1971) and directly equivalent to the Darlington (1971) and Cole (1973) test bias definition, the logical inconsistencies of which have been previously described (Hunter & Schmidt, 1976; Peterson & Novick, 1976). We conclude that performance matching as a basis of forming “equal ability” groups is inappropriate.

Use of an assessment center as an aid in management selection

Abstract: An application of assessment center procedures is described for the selection of first level managers from the craft ranks. Information is reported on how the assessment data are used in promotion decisions. An evaluation study was conducted that indicates that the assessment center program has a positive effect on performance of managers at the first level and the pool of potential for higher levels of management. The study results also suggest ways in which assessment data can be used more effectively in promotion decisions.

Log-linear modeling in personnel research

Abstract: Log-linear modeling is a discrete multivariate statistical technique that is designed specifically for analyzing data when both the independent and dependent variables are categorical or nominal. The purpose of this paper is to demonstrate the utility of this technique in personnel research. The paper (a) discusses behavioral areas of application, (b) compares log-linear modeling with chi-square and regression analysis, (c) presents the basic principles and hypotheses of log-linear modeling, and (d) shows how the technique is used.