Showing papers in "Personnel Psychology in 1988"

Transfer of training: a review and directions for future research

Abstract: Transfer of training is of paramount concern for training researchers and practitioners. Despite research efforts, there is a growing concern over the “transfer problem.” The purpose of this paper is to provide a critique of the existing transfer research and to suggest directions for future research investigations. The conditions of transfer include both the generalization of learned material to the job and the maintenance of trained skills over a period of time on the job. The existing research examining the effects of training design, trainee, and work-environment factors on conditions of transfer is reviewed and critiqued. Research gaps identified from the review include the need to (1) test various operationalizations of training design and work-environment factors that have been posited as having an impact on transfer and (2) develop a framework for conducting research on the effects of trainee characteristics on transfer. Needed advancements in the conceptualization and operationalization of the criterion of transfer are also discussed.

An investigation of the determinants of successful assigned mentoring relationships

Abstract: This study examined the influence of protege characteristics, gender composition of the mentoring relationship, the quality of the relationship, and the amount of time the protege spent with the mentor on career and psychosocial benefits gained by the protege. Proteges were assigned to mentors as part of a development program designed to facilitate personal and career development of educators. An instrument designed to assess the extent to which mentors provide career and psychosocial outcomes to proteges was developed. Protege gender, job involvement, and career planning activity was related to attainment of psychosocial outcomes. Implications and future directions for research regarding mentoring are discussed.

A meta-analysis of self-supervisor, self-peer, and peer-supervisor ratings.

Abstract: Reviews of self–supervisor, self–peer, and peer–supervisor ratings have generally concluded that there is at best a modest correlation between different rating sources. Nevertheless, there has been much inconsistency across studies. Accordingly, a meta-analysis was conducted. The results indicated a relatively high correlation between peer and supervisor ratings (ρ= .62) but only a moderate correlation between self-supervisor (ρ= .35) and self-peer ratings (ρ= .36). While rating format (dimensional versus global) and rating scale (trait versus behavioral) had little impact as moderators, job type (managerial/professional versus blue-collar/service) did seem to moderate self-peer and self-supervisor ratings.

Structured Interviewing: Raising the Psychometric Properties of the Employment Interview.

Abstract: A highly structured employment interviewing technique is proposed, which includes the following steps: (1) develop questions based on a job analysis, (2) ask the same questions of each candidate, (3) anchor the rating scales for scoring answers with examples and illustrations, (4) have an interview panel record and rate answers, (5) consistently administer the process to all candidates, and (6) give special attention to job relatedness, fairness, and documentation in accordance with testing guidelines. Examination of psychometric properties for hiring entry-level production employees (n= 149) reveals high interrater reliability (r= .88) and predictive validity (uncorrected r= .34, corrected r= .56), as well as evidence for test fairness and utility. The levels of these properties are comparable to those of a comparison battery of typical employment tests, and correlations with the tests suggest that the interview has a strong cognitive aptitude component. Potential explanations for the effectiveness of this structured interviewing technique are discussed.

Individual and situational correlates of whistle‐blowing

Abstract: Although popular interest in whistle-blowing continues to increase, little is known about why some employees who observe wrongdoing report it, while others do not. In the present study, we analyze archival survey data from individuals who observed wrongdoing in any of 22 organizations. Results are generally consistent with predictions based on a model of whistle-blowing as a type of prosocial behavior. Whistle-blowing was more likely when observers of wrongdoing (1) held professional positions, (2) had more positive reactions to their work, (3) had longer service, (4) were recently recognized for good performance, (5) were male (though race was unrelated to whistle-blowing), (6) were members of larger work groups, and (7) were employed by organizations perceived by others to be responsive to complaints. Implications for research and practice are described.

Work load, tension, and coping : Moderating effects of supervisor support

Abstract: This research investigated the buffering role of social support in the relationship of work load to both tension-anxiety and coping for police radio dispatchers (N= 60). Each dispatcher was observed throughout an entire 8-hour work shift by a trained observer. Objective load consisted of the hourly rate of incoming telephone calls, police radio transmissions, and face-to-face or written communications from superiors, co-workers, and others. At the end of each shift, measures were taken of dispatchers' perceived load, felt tension-anxiety, and coping actions. Hierarchical regression revealed that support from superiors moderated the effects of both objective and perceived loads; no main effects of support were found. Statistically significant interactions confirmed the hypothesized buffering role of support: under high perceived load, dispatchers with high social support engaged in more coping actions and felt less tension-anxiety than did low-support dispatchers. No differences were found when perceived load was low. Support also buffered effects of objective load on tension-anxiety but not on coping.

A Survey of Management Training and Education Practices in U.S. Companies.

Abstract: A survey was systematically developed to assess issues related to management training and education, and it was sent to 1,000 randomly selected U.S. companies with at least 1,000 employees. Based on a 61% response rate, findings were obtained regarding issues such as needs assessment, management training and development approaches, reasons for selecting particular programs, characteristics of participants, how decisions are made regarding who will participate, preparation and follow-up of participants, evaluation of management training programs, future management training trends, and needed training content. Significant findings by company size and industry type also are reported.

The influence of training method and trainee age on the acquisition of computer skills

Abstract: This experiment examined the influence of training method and trainee age on performance during training in the acquisition of computer software skills. A behavioral modeling training method yielded consistently superior computer software mastery compared with a nonmodeling approach. Older trainees exhibited significantly lower performance than did younger trainees in both the modeling and the nonmodeling training conditions. Findings are discussed in terms of strategies for overcoming older worker technological obsolescence.

The Relationship between Pay-for-Performance Perceptions and Pay Satisfaction.

Abstract: A study was conducted with 104 hospital employees to assess the relationship between pay-for-performance perceptions and pay satisfaction. Unlike previous research examining this relationship, a multi-item measure of pay-for-performance perceptions and a multidimensional measure of pay satisfaction were used. As hypothesized, the results indicated a positive relationship between pay-for-performance perceptions and pay-raise satisfaction, pay-level satisfaction, and overall pay satisfaction even after the effects of salary level, salary increases, performance ratings, job tenure, job satisfaction, and promotions were controlled. The importance of gathering perceptual data on characteristics of the pay system believed to have differential relationships with the subdimensions of pay satisfaction are discussed.

Multiple assessment of managerial effectiveness: interrater agreement and consensus in effectiveness models

Abstract: Research has consistently identified poor interrater agreement among multiple assessments of managerial performance. Three alternative sources of dissensus in the effectiveness ratings were examined: rating errors, selective perceptions, and variations in criteria type or weight. As the available empirical evidence and theoretical analysis show, all three causes provide plausible reasons—though in varying degrees—for the low agreement coefficient. However, an empirical study designed to test three specific hypotheses on criterion type and criterion weights found consensus in the effectiveness models of superiors, subordinates, and peers. Consensus among different raters was high on both the role behaviors and on the personal traits of the managers as criteria for effectiveness. While these findings supported Biddle's role theory (1979), disagreement on the relative weights of these criteria was evident. These observations underscore the need for further conceptualization on the preference functions of raters as a primary source of the low convergent validity coefficients among multiple raters. Further research is also desirable on contextual and cognitive factors that may lead to shifts in criterion type and criterion weight, as well as on actual rating error tendencies among different raters.

Validity of self‐assessment: a field evaluation

Abstract: This study evaluated the validity of self-assessment in a natural setting. It also addressed the possibility of improving the utility of self-assessment by cautioning subjects that their responses could be compared with other existing data on them. Subjects were 357 males, aged 18–19 years, who were being screened for a prestigious military course. During the screening, they participated in small groups for nine days of tests, class exercises, and field simulations supervised by veteran commanders. On the ninth day, ratings were collected from commanders, peer group members, and the subjects themselves on dimensions related to eventual success in the training course. The experimental group was told their reports would be compared with those from other sources; the control group was not. Predictive and convergent validities were examined on three criteria: course success, commander ratings, and peer evaluations. Self-assessments from both experimental and control group were valid; however, those of the experimental group did not yield consistently higher validities. Findings are discussed in regard to their practical ramifications.

A comparison of criteria for test validation: a meta‐analytic investigation

Abstract: Meta-analyses of validity coefficients from tests of clerical abilities for five criteria—supervisor ratings, supervisor rankings, work samples, production quantity, and production quality—were conducted, and the resulting expected true validities were compared. Ratings, rankings, work samples, and production quantity all resulted in high test validities. Validities resulting from ratings and quantity-of-production criteria were highly similar across tests. Validities resulting from rankings and work samples were on the average higher than those from ratings and quantity of production. The fifth criterion, quality of production, had low predictability and did not generalize across situations.

An empirical investigation of self‐appraisal‐based performance evaluation

Abstract: This study investigated the effectiveness of a self-appraisal-based performance evaluation system (SABPE) that incorporates self-assessment into traditional supervisory evaluation procedures. Its subject sample consisted of 88 faculty members and their chairpersons at a land-grant state university. Results indicated that (1) there was high congruency between self- and chairperson ratings, (2) both ratings had moderate to high levels of criterion-related validity, and (3) both faculty members and chairpersons reported high SABPE acceptance. The implications of these results for future self-appraisal research are discussed.

Benefit coverage and employee cost: critical factors in explaining compensation satisfaction

Abstract: This research examined the joint effects of benefit coverage and the costs borne by employees on multiple dimensions of compensation satisfaction. Information regarding pay and benefit practices was collected by interviewing personnel specialists and reviewing policy and procedure documents in eight organizations. At a later time, satisfaction data were collected from employees via mail surveys. The results indicated that satisfaction with benefits increased with improved coverage and decreased with higher employee costs. Employees were particularly sensitive to variation in the cost of health insurance. Also, for respondents possessing accurate information about actual coverage levels, the relationship between coverage and satisfaction was more pronounced. For the organizations studied here, salaries and benefits were being administered using compensatory allocation rules. Salary levels were negatively correlated with the levels of benefit coverage, and the various components of the compensation systems covaried in complex ways. The methodological implications associated with this research are emphasized and suggest that the interpretability of much of the past research devoted to employee benefits has been compromised.

An investigation of the factors influencing employees' willingness to accept mobility opportunities

Abstract: This study investigated the influence of 1,076 state employees' career stage, family characteristics, job tenure and specialization, and perceptions of job favorability, movement, and career development opportunities on willingness to accept upward, lateral, and downward mobility opportunities. Results indicated that employees in early career stages with little community tenure who perceived a lack of congruence between their current jobs and their "ideal" jobs and expressed little desire to remain in their current jobs were more willing to accept mobility opportunities. Few differences were noted in regression models across different mobility opportunities. Implications for future research regarding employee mobility are discussed.

A meta‐analysis of the validity of methods for rating training and experience in personnel selection

Abstract: This paper reviews the validity evidence for systematic methods used to evaluate training and experience (T&E) ratings in personnel selection. Meta-analytic summaries of the data indicate that validity varies with the type of T&E evaluation procedure used. The Illinois job element and behavioral consistency methods each demonstrated useful levels of validity (.20 and .45, respectively) with small corrected standard deviations, thus supporting validity generalization. Both the point and task methods yielded low mean validities (.11 and .15, respectively) with larger variability. The authors hypothesized that both the point and task methods were affected by a job experience moderator. Partial support for this hypothesis was found. Moderator analyses suggested that the point method was most valid when the applicant pool had low mean levels of job experience and was least valid with an experienced applicant pool. Additional research is desirable on all T&E methods to decrease the potential impact of second-order sampling error in the meta-analytic results. Further research is also needed to explicate the constructs measured by T&E evaluations. This paper sutnmarizes and evaluates the validity evidence for methods of evaluating the training and experience (T&E) of applicants. T&E methods attempt to predict future job performance through systematic, judgment-based evaluations of information provided by applicants on resumes, applications, or other documents. This paper discusses the rationale for using T&E methods in applicant appraisal, reviews specific approaches to T&E evaluation, and presents a meta-analysis of T&E criterion-related validity coefficients. Meta-analytic results are presented for T&E methods as a whole and separately for specific T&E approaches. The opinions expressed in this article are those of the authors and do not necessarily reflect the opinions and policies of the organizations with which they are affiliated. Correspondence and requests for reprints should be addressed to Michael A. McDaniel, Defense Personnel Security Research and Education Center, 99 Pacific Street, Building 455 Suite E, Monterey, CA 93940.

Setting cutoff scores: legal, psychometric, and professional issues and guidelines

Abstract: Cutoff scores are used in a variety of settings to help managers make personnel decisions. Although cutoff scores are used widely, information about their derivation and appropriate use is scattered in the legal, psychometric, and professional literatures. The purpose of this paper is to analyze critically and to integrate these diverse literatures, to summarize what is known and what is unknown about the use and misuse of cutoff scores. Alternative methods are examined in relation to legal precedents set by the courts, to psychometric principles, and to the various sets of professional guidelines. Directions for future research are suggested, and we conclude with a set of recommendations regarding acceptable professional practice in this area.

Simulation-Based Assessment of Managerial Competence: Reliability and Validity.

Abstract: Prior work based on complexity theory has attempted, with some success, to predict general and managerial performance in complex, uncertain, and fluid task settings. The present paper evaluates a quasi-experimental simulation technique that was specifically designed to measure the impact of individual differences in a number of managerial styles (including a style reflecting cognitive complexity) on executive performance. Twenty simulation-based measures were tested for reliability and validity. On the basis of the data obtained from two separate samples, it was concluded that this quasi-experimental simulation technology may be useful for assessing a number of managerial styles that are not currently tapped by other measurement methods. Research results reported in this paper, as well as results obtained in other concurrent efforts, are summarized.

The role of the traditional research design in underestimating the validity of the employment interview

Abstract: Building upon the suggestions of Arvey and Campion (1982) and Zedeck, Tziner, and Middlestadt (1983), this paper calls into question the traditional approach to validating the employment interview. A general review of the literature reveals that individuals are likely to differ in their ability to provide accurate predictions of employee behavior and that interviewers often differ in their constant tendency to make favorable or unfavorable ratings. A review of published validation studies shows that most researchers fail to consider these tendencies and, therefore, may be inappropriately collapsing data across multiple interviewers. Through analytic argument we conclude that reviews of the existing literature will likely underestimate the usefulness of this widely used technique.

Does pre‐employment drug use predict on‐the‐job suitability?

Abstract: Drug testing is increasingly used in the screening of applicants for employment. Despite the growth of drug testing, there is little research that examines the value of pre-employment drug-use information in the prediction of post-employment suitability. This research, which was based on a sample of 10,188 young adults, examined the criterion-related validity of pre-employment drug-use information. For all drugs examined, the greater the frequency of use and the earlier the age at which the drug was first used, the greater the probability of a person being classified as unsuitable after hire. However, the operational validity of each drug variable was influenced by the base rate of drug use. The low base rates for some drugs make their operational validity of limited value. The operational validity of the marijuana frequency-of-use measure (.07) was approximately equal to that of less frequently used drugs (e.g., stimulants and depressants). No strong moderators of the validity of a drug-composite measure were found.

Influences on managerial pay allocations: a dependency perspective

Abstract: This paper reviews evidence and presents arguments for the view that managerial pay allocations are influenced by the degree of dependence on subordinates and by threats to that dependence. In assessing this view, the issues of managerial discretion in pay allocation and pay as a means of managing dependencies are explored. Major sources of objective and subjective dependence on subordinates are identified. Predictions are made regarding managerial pay allocations under various conditions of dependence and dependency threat. Finally, research and practical implications of the dependency perspective are discussed.

Effect of situational interviews, conventional structured interviews, and training on interview rating agreement: an experimental analysis

Abstract: The present study examines the degree to which agreement in interviewer ratings may be influenced by training, use of structured conventional interviews, or situational interviews. Results of an experimental study conducted among 42 managers who were experienced as interviewers revealed no training effect on rating agreement, but the impact of the situational format on consistency in assessments of applicant suitability was significant. Implications of these findings for usefulness of situational employment interviews and future research investigating the reliability of situational interview formats are discussed.

Some new frontiers in personnel selection research

Abstract: Current research developments in several areas of personnel selection methodology are reviewed. The areas reviewed represent attempts to develop alternatives to traditional paper-and-pencil methods of test development. These areas include computer-interactive tests in the employment situation, new ways of using life-history information, the use of physical tests in selecting individuals for physically demanding jobs, and the development of job-related medical standards for job assignment.

A comparison of holistic versus decomposed rating of Position Analysis Questionnaire work dimensions.

Abstract: Recent studies have attempted to reduce the cost and intrusiveness of the Position Analysis Questionnaire (PAQ) by limiting the amount of information provided to the analyst, with consistently negative results We examined an alternative technique for improving the cost-effectiveness of the PAQ that avoids the need to rate the hundreds of items that constitute the instrument Three groups of raters (professional job analysts, graduate students in industrial psychology who were familiar with the PAQ, and PAQ-unfamiliar undergraduates) made direct holistic ratings of the PAQ dimensions for four familiar jobs The holistic ratings were compared with decomposed PAQ dimension profiles obtained from the item-level ratings of the professional analysts Cronbach accuracy analyses indicated near-zero convergence between the holistic and decomposed dimension ratings, even for the professional PAQ job analysts We conclude that holistic rating of dimensions is not an effective means of reducing the cost of a PAQ job analyses and that it is likely to be similarly ineffective with task- or ability-based instruments

Test validation for scientific understanding: two demonstrations of an approach to studying predictor‐criterion linkages

Abstract: This paper argues that a construct-oriented approach to test validation is likely to enhance scientific understanding of our predictor measures, performance criteria, and links between them. In particular, examining relationships between relatively homogeneous predictors and criteria tapping specific performance areas operationalizes earlier conceptual statements made by Guion and Dunnette about test validation for scientific understanding. Two demonstrations are offered to show how measures of predictor constructs have predictably different patterns of correlations with different criteria. In a study of Navy recruiters (N= 267), individual personality scales had significantly different relationships with three different rating criteria; in a second study, with Army enlisted soldiers (N= 8, 642), cognitive ability and personality construct measures also showed predictable patterns of correlations, with rating criteria measuring three different performance areas. The paper discusses scientific and practical implications of this construct-oriented approach to test validation.

Propositions about grievance settlements: finally, consultation with grievants

Abstract: In the past, research on grievances generally has excluded the views of grievants about the process and its consequences. This study is an analysis of self-reports of 324 grievants concerning the characteristics of a grievance-filing incident and its perceived outcomes. The results raise questions about the veracity of the industrial relations dictum that grievances should be settled at the lowest step in the process. Instead, the nature of the settlement and the type of grievance were perceived to have significant effects on outcome and attitude measures.

Supervisory and employee characteristics as correlates of employee salary increases

Abstract: Previous research on merit pay has focused on the context in which merit decisions take place and on characteristics of the employee. In the present study it was hypothesized that not only do employee characteristics influence merit increases, but so do characteristics of the employee's supervisor. The results from 175 employees in a midwestern manufacturing plant showed that 11% and 24% of the variance in employee salary increases were attributable to supervisory and employee characteristics, respectively. These results suggest that supervisory characteristics should be included in future research involving correlates of employee salary increases and by organizations when auditing the administration of merit pay plans.

Group homogeneity, range restriction, and range enhancement effects on correlations

Abstract: Recently, the issue of range enhancement effects on correlations has been raised as a potential criticism of correction for range restriction in validity generalization. While adding to or deleting from a sample can affect both the variance and correlation in the sample, increases or decreases in individual differences in an intact sample resulting from such factors as history, maturation, training, and experience will not necessarily affect correlations. If such changes in individual differences do occur in concurrent validation studies, the most likely result will be an underestimate of the unrestricted validity.

Issues concerning the use of regression analysis in salary discrimination cases

Abstract: Twenty-nine salary discrimination court cases were examined to determine ways an organization can refute a regression analysis that leads to the inference of discrimination in compensation. It was found to be equally effective for the organization to introduce its own regression, other statistics, or no statistics. The plaintiff won all of the cases when discrimination was proven in promotion. The defendant won all cases when discrimination was not proven in promotion. Defendants also won all cases when plaintiffs inappropriately treated jobs as fungible or when they failed to include important variables. Market variables may be included in regressions if defendants can show they were applied consistently to determine salaries. The implications of the courts' acceptance of some regressions for comparable worth is presented, and a list of recommendations is made for organizations that may be faced with results from regression analysis in court.

Voluntary affirmative action and preferential treatment: legal and research implications

Abstract: This paper provides a review of recent Supreme Court decisions stemming from the implementation of voluntary AAPs that include preferential treatment as a remedy; the evidence and proof required to establish and defend against reverse discrimination claims is identified. Overall, the courts sanctioned the use of preferential treatment if it occured as part of a well-conceived AAP. However, differences were found in the manner in which the courts adjudicated Title VII and constitutional claims of preferential treatment. The discussion examines research on the potential effects of voluntary AAPs on important organizational and societal outcome variables. We conclude that the impact of AAPs is likely to be far-reaching, exerting an effect possibly well beyond the limits considered in court decisions.