Showing papers in "Psychological Assessment in 2000"

Clinical versus mechanical prediction: a meta-analysis.

Abstract: The process of making judgments and decisions requires a method for combining data. To compare the accuracy of clinical and mechanical (formal, statistical) data-combination techniques, we performed a meta-analysis on studies of human health and behavior. On average, mechanical-prediction techniques were about 10% more accurate than clinical predictions. Depending on the specific analysis, mechanical prediction substantially outperformed clinical prediction in 33%-47% of studies examined. Although clinical predictions were often as accurate as mechanical predictions, in only a few studies (6%-16%) were they substantially more accurate. Superiority for mechanical-prediction techniques was consistent, regardless of the judgment task, type of judges, judges' amounts of experience, or the types of data being combined. Clinical predictions performed relatively less well when predictors included clinical interview data. These data indicate that mechanical predictions of human behaviors are equal or superior to clinical prediction methods for a wide range of circumstances.

Development and preliminary validation of a brief broad-spectrum measure of trauma exposure : The Traumatic Life Events Questionnaire

Abstract: This article describes the development and preliminary validation of a brief questionnaire that assesses exposure to a broad range of potentially traumatic events. Items were generated from multiple sources of information. Events were described in behaviorally descriptive terms, consistent with Diagnostic and Statistical Manual of Mental Disorders IV posttraumatic stress disorder stressor criterion A1. When events were endorsed, respondents were asked if they experienced intense fear, helplessness, or horror (stressor criterion A2). In separate studies with college students, Vietnam veterans, battered women, and residents of a substance abuse program, most items possessed adequate to excellent temporal stability. In a study comparing questionnaire and structured-interview inquiries of trauma history, the 2 formats yielded similar rates of disclosure. Preliminary data on positive predictive power are also presented.

Factor analysis and scale revision.

Abstract: This article reviews methodological issues that arise in the application of exploratory factor analysis (EFA) to scale revision and refinement. The authors begin by discussing how the appropriate use of EFA in scale revision is influenced by both the hierarchical nature of psychological constructs and the motivations underlying the revision. Then they specifically address (a) important issues that arise prior to data collection (e.g., selecting an appropriate sample), (b) technical aspects of factor analysis (e.g., determining the number of factors to retain), and (c) procedures used to evaluate the outcome of the scale revision (e.g., determining whether the new measure functions equivalently for different populations).

Development and validation of the Eating Disorder Diagnostic Scale: A brief self-report measure of anorexia, bulimia, and binge-eating disorder.

Abstract: This article describes the development and validation of a brief self-report scale for diagnosing anorexia nervosa, bulimia nervosa, and binge-eating disorder. Study 1 used a panel of eating-disorder experts and provided evidence for the content validity of this scale. Study 2 used data from female participants with and without eating disorders (N = 367) and suggested that the diagnoses from this scale possessed temporal reliability (mean K = .80) and criterion validity (with interview diagnoses; mean K = .83). In support of convergent validity, individuals with eating disorders identified by this scale showed elevations on validated measures of eating disturbances. The overall symptom composite also showed test-retest reliability (r = ,87), internal consistency (mean a = .89), and convergent validity with extant eating-pathology scales. Results implied that this scale was reliable and valid in this investigation and that it may be useful for clinical and research applications. It has been estimated that 10% of female individuals in western countries will suffer from a diagnosable eating disorder (American Psychiatric Association [APA], 1994), making it one of the more prevalent psychiatric problems faced by women. Anorexia nervosa is characterized by (a) extreme emaciation; (b) intense fear of gaining weight or becoming fat despite a low body weight; (c) disturbed perception of weight and shape, an undue influence of weight or shape on self-evaluation, or a denial of the seriousness of the low body weight; and (d) amenorrhea (APA, 1994). This disorder has a lifetime prevalence of almost 1% among females, is refractory to treatment, shows a chronic course, results in serious medical complications, and is associated with psychiatric comorbidity such as mood, anxiety, and personality disorders (Wilson, Heffernan, & Black, 1996). Bulimia nervosa involves (a) recurrent episodes of uncontrollable consumption of large amounts of food, (b) compensatory

Psychopathic traits and conduct problems in community and clinic-referred samples of children: further development of the psychopathy screening device.

Abstract: This study examined the structure of psychopathic traits in 2 samples of children. The nonreferred community sample included 1,136 children recruited from elementary schools in 2 school districts in the southeastern United States. The clinic sample included 160 children referred to an outpatient mental health clinic serving the same geographic region. In both samples, parent and teacher ratings of psychopathic traits were subjected to a principal-axis factor analysis, and the congruence of the factor structure across samples was examined using confirmatory factor analysis. In both samples, 1 dimension that consisted of the callous and unemotional traits that have been hallmarks of most clinical descriptions of psychopathy was isolated. Two other dimensions consisting of narcissistic traits and impulsivity emerged in the community sample. Both the narcissism and impulsivity dimensions were highly related to symptoms of oppositional defiant disorder, conduct disorder, and attention deficit hyperactivity disorder. However, the callous and unemotional traits were only weakly associated with these symptoms after controlling for the other dimensions of psychopathy.

On the sins of short-form development.

Abstract: The empirical short-form literature has been characterized by overly optimistic views of the transfer of validity from parent form to short form and by the weak application of psychometric principles in validating short forms. Reviewers have thus opposed constructing short forms altogether, implying researchers are succumbing to an inappropriate temptation by trying to abbreviate measures. The authors disagree. The authors do not oppose the development of short forms, but they do assert that the validity standards for short forms should be quite high. The authors identify 2 general and 9 specific methodological sins characterizing short-form construction and offer methodological suggestions for the sound development of short forms. They recommend a set of 6 a priori steps researchers should consider and 9 methodological procedures researchers can use to develop valid abbreviated forms of clinical-assessment procedures.

Reactive and proactive aggression: evidence of a two-factor model.

Abstract: This article examines the construct validity of reactive and proactive aggression, as assessed by the teacher-rating scale developed by K. A. Dodge and J. D. Coie (1987). In Study 1 (n = 149 boys), confirmatory factor analyses revealed that a 2-factor model, in which a substantial correlation was observed between the 2 latent factors, presented a better fit than a single-factor model. Study 2 (n = 193 boys) examined the relations presented by the 2 forms of aggression with peer status, leadership, social withdrawal, and victimization by peer. Reactive and proactive aggressive behaviors presented distinct patterns of relations consistent with the theoretical definitions. The results of these studies suggest that the questionnaire measures 2 forms of aggressive behavior that, although being substantially related, have a unique discriminant dimension.

Development and validation of the Stephenson Multigroup Acculturation Scale (SMAS).

Abstract: This article describes the development and psychometric evaluation of the Stephenson Multigroup Acculturation Scale (SMAS). Three studies were conducted to describe its development and refinement, examine its psychometric properties with 436 participants from 5 ethnic groups, and examine the robustness of the factor structure with a new sample. Exploratory factor analyses generated a 2-factor solution: ethnic society immersion and dominant society immersion. Item refinement resulted in a 32-item version of the SMAS. Findings indicated a robust factor structure across groups. Confirmatory factor analysis indicated that the 2-factor model provided a close approximation to the observed data. Studies indicated high reliability and validity indexes. Findings support the role of acculturation as a mediator between ethnic group affiliation and standardized assessment results.

Acute Stress Disorder Scale : A self-report measure of acute stress disorder

Abstract: The Acute Stress Disorder Scale (ASDS) is a self-report inventory that (a) indexes acute stress disorder (ASD) and (b) predicts posttraumatic stress disorder (PTSD). The ASDS is a 19-item inventory that is based on Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV, American Psychiatric Association, 1994) criteria. The ASDS possessed good sensitivity (95%) and specificity (83%) for identifying ASD against the ASD Interview on 99 civilian trauma survivors. Test-retest reliability of the ASDS scores between 2 and 7 days was strong (r = .94). The ASDS predicted 91% of bushfire survivors who developed PTSD and 93% of those who did not; one third of those identified by the ASDS as being at risk did not develop PTSD, however. The ASDS shows promise as a screening instrument to identify acutely traumatized individuals who warrant more thorough assessment for risk of PTSD.

Use of the AUDIT and the DAST-10 to identify alcohol and drug use disorders among adults with a severe and persistent mental illness.

Abstract: State Psychiatric Hospital outpatients (93 men, 69 women) diagnosed with a serious persistent mental illness (SPMI) completed the Alcohol Use Disorders Identification Test (AUDIT) and Drug Abuse Screening Test (DAST-10) by interview as part of a general health/behavior screening instrument. Responses to the AUDIT and DAST-10 were compared with criteria of current diagnosis and occurrence of symptoms in the last year for both alcohol and drug use disorders, respectively. Results showed that for both diagnosis and symptoms, AUDIT cutpoints of 7 and 8 had good sensitivity and specificity, and DAST-10 cutpoint of 2 was identified for both criteria. These and other findings suggest that both instruments have promising clinical utility when used with individuals diagnosed with an SPMI.

An empirical investigation of the factor structure of the AUDIT

Abstract: This study investigated the Alcohol Use Disorders Identification Test's (AUDIT) factor structure and psychometric properties. The factor structure was derived from a sample of 7,035 men and women primary care patients. A principal components analysis identified 2 factors in the AUDIT data and was supported in a confirmatory factor analysis (CFA). The 2 factors were Dependence/Consequences and Alcohol Consumption. The CFA also provided support for a 3-factor model whose factors (Alcohol Consumption, Alcohol Dependence, and Related Consequences) matched those proposed by the AUDIT's developers. Psychometric indexes were determined by use of the baseline and 12-month follow-up data of 301 men and women who entered a clinical trial. The results showed that the 2 factors had good reliability. Validity tests supported the interpretation of what the 2 factors measure, its implications for relationships to other variables, and the comparability of the 2- and 3-factor models.

Use of the Personality Assessment Inventory to assess psychopathy in offender populations.

Abstract: The authors investigated the validity of the Antisocial Features (ANT) scale of the Personality Assessment Inventory (PAI; L. C. Morey, 1991) with respect to assessments of psychopathy in 2 offender samples. Study 1 included 46 forensic psychiatric inpatients who were administered the Screening Version of the Hare Psychopathy Checklist (PCL:SV; S. D. Hart, D. N. Cox, & R. D. Hare, 1995). In Study 2, 55 sex offenders were administered the Hare Psychopathy Checklist--Revised (PCL-R; R. D. Hare, 1991). ANT scores correlated highly with the PCL:SV total score (r = .54) and moderately with the PCL-R total score (r = .40). ANT tapped primarily behavioral symptoms of psychopathy rather than interpersonal and affective symptoms. Also, ANT had low to moderate diagnostic efficiency regarding diagnoses of psychopathy, suggesting that it may be better used as a dimensional rather than categorical measure of this construct.

Structural differences in parent and child reports of children's symptoms of depression and anxiety.

Abstract: Two cohorts of public elementary school children and their parents (assessed 3 years apart) completed child and parent forms of the Children's Depression Inventory (CDI) and the Revised Children's Manifest Anxiety Scale (RCMAS). Assessments were conducted twice, once during the fall (N = 562) and again during the spring (N = 630) of the 6th grade. Factor analyses revealed 3 factors for each measure. Two of the 3 parent CDI factors manifested some degree of congruence with their counterparts from the child CDI. Similarly, 2 of the 3 RCMAS factors were somewhat congruent across informant types. Differences between parent and child factor structures suggest that parents' and children's reports focus on somewhat different aspects of child psychopathology, and they can make qualitatively different contributions to the multiaxial assessment of children.

Validation of a brief measure of posttraumatic stress disorder: the Distressing Event Questionnaire (DEQ).

Abstract: The Distressing Event Questionnaire (DEQ) is a brief instrument for assessing posttraumatic stress disorder (PTSD) according to criteria provided in Diagnostic and Statistical Manual of Mental Disorders (4th ed.). The DEQ possesses high internal consistency and exhibited satisfactory short-term temporal stability in studies with Vietnam War combat veterans and battered women. In a sample of Vietnam War veterans and 4 separate samples of abused women (with histories of incest, rape, intimate partner abuse, or prostitution and abuse), the DEQ exhibited very good discriminative validity when judged against structured interview assessment of PTSD. The DEQ exhibited strong convergent validity with other PTSD measures and other indexes of adjustment and also exhibited strong convergent validity as a measure of PTSD across ethnic groups in both the veteran sample and the combined women's sample.

Summed-score linking using item response theory : Application to Depression measurement

Abstract: An item response theory (IRT) approach to test linking based on summed scores is presented and demonstrated by calibrating a modified 23-item version of the Center for Epidemiologic Studies Depression Scale (CES-D) to the standard 20-item CES-D. Data are from the Depression Patient Outcomes Research Team, II, which used a modified CES-D to measure risk for depression. Responses (N = 1,120) to items on both the original and modified versions were calibrated simultaneously using F. Samejima's (1969, 1997) graded IRT model. The 2 scales were linked on the basis of derived summed-score-to-IRT-score translation tables. The established cut score of 16 on the standard CES-D corresponded most closely to a summed score of 20 on the modified version. The IRT summed-score approach to test linking is a straightforward, valid, and practical method that can be applied in a variety of situations.

The discriminative validity of a phallometric test for pedophilic interests among adolescent sex offenders against children.

Abstract: The authors examined the responses of adolescent sex offenders against children on a phallometric test of pedophilic interests. Participants were 40 adolescent sex offenders against children, 75 young adult sex offenders against children, and 39 young adult comparison participants. The responses of adolescents with female victims resembled those of comparison participants; adolescents with any male victims had larger relative responses to child stimuli than comparison participants. Young adult offenders, regardless of victim sex, had larger relative responses to child stimuli than comparison participants. Using a cut score of 0 (indicating equal or greater arousal to children than to adults), sensitivity was 42% for adolescents with any male victims, and specificity was 92% for the comparison participants. Results suggest phallometric testing can identify pedophilic interests among these adolescent sex offenders.

Validity and utility of computer-based test interpretation.

Abstract: Computers have been important to applied psychology since their introduction, and the application of computerized methods has expanded in recent decades. The application of computerized methods has broadened in both scope and depth. This article explores the most recent uses of computer-based assessment methods and examines their validity. The comparability between computer-administered tests and their pencil-and-paper counterparts is discussed. Basic decision making in psychiatric screening, personality assessment, neuropsychology, and personnel psychology is also investigated. Studies on the accuracy of computerized narrative reports in personality assessment and psychiatric screening are then summarized. Research thus far appears to indicate that computer-generated reports should be viewed as valuable adjuncts to, rather than substitutes for, clinical judgment. Additional studies are needed to support broadened computer-based test usage.

Measurement properties of the Center for Epidemiologic Studies-Depression Scale in a homeless population.

Abstract: The measurement properties of the Center for Epidemiologic Studies--Depression Scale (CES-D; L. S. Radloff, 1977) were evaluated in a probability sample of homeless adults residing in a large and demographically diverse community. The findings from this investigation suggest that the CES-D is a reliable measure of depressive symptoms among homeless adults and that the factor structure of the scale replicates the factor structure found in the general population. Change in the CES-D scores was associated with change in residential status, with participants who had made a transition from homelessness to regular domicile, reporting significantly lower levels of depressive symptomatology. This result indicates the scale's sensitivity to current depressive mood, as affected by significant life events encountered by homeless persons.

Detection of coached general malingering on the MMPI—2.

Abstract: The current study examined the effects of validity-scale coaching on one's ability to feign general psychopathology. College students were coached on malingering strategies and completed the MMPI-2. Their responses were compared with students asked to malinger psychopathology without validity-scale coaching and with psychiatric inpatients completing the MMPI-2 under standard instructions. In accordance with previous research (e.g., J. R. Graham, D. Watts, & R. E. Timbrook, 1991), uncoached malingerers were adequately discriminated from patients by using the Infrequency (F) scale. However, as suggested by previous research (R. Rogers, R. M. Bagby, & D. Chakraborty, 1993), the F scale was not as effective at classifying coached malingerers. It was found that other validity indicators, such as the Infrequency Psychopathology Scale were more accurate at discriminating between coached malingerers and hospitalized patients.

Adolescent offenders and stages of change.

Abstract: Two hundred twenty-five adolescent offenders completed the Stages of Change Scales (SOCS; E. A. McConnaughy, J. O. Prochaska, & W. F. Velicer, 1983), which were designed to measure 4 stages of psychological and behavioral change. The goals of this research were to obtain norms among adolescent offenders for the SOCS, to examine the factor structure of the SOCS, and to investigate the convergent and discriminant validity of the SOCS by examining associations among these scales and the Paulhus Deception Scales (PDS; D. L. Paulhus, 1994, 1999) and Multidimensional Anger Inventory (MAI; J. M. Siegel, 1986). Means and alpha coefficients for the SOCS were similar to those reported in a previous nonnative clinical sample of adult patients. However, relatively large intercorrelations among the 4 scales of the SOCS, and results from factor analyses of SOCS items, suggested that fewer than 4 factors represented the SOCS items among our sample of adolescent offenders. Patterns of correlations with the PDS and with the MAI provided convergent and discriminant validity for the SOCS. Implications of the findings for research and clinical practice are discussed.

Updating to the WAIS-III and WMS-III: considerations for research and clinical practice.

Abstract: The Wechsler Adult Intelligence Scale-Revised (WAIS-R) and the Wechsler Memory Scale-Revised (WMS-R) are the most commonly used intelligence and memory scales in both clinical and neuropsychology. In 1997, updated versions of these instruments (the WAIS-III and WMS-III) were published. Because of the extensive use of the WAIS-R and WMS-R in the field and the body of accumulated research, there is naturally some reluctance by clinicians and researchers to update to the new versions. It is sometimes difficult for clinicians who test individuals on repeated occasions to switch over to the new versions of the scales because of the difficulty of interpreting score discrepancy between the 2 versions. Researchers, especially those conducting longitudinal research, have a similar difficulty in changing measurement devices because of the possible threat of internal validity. This article reviews the substantive revisions of the scales and outlines those issues that users should take into consideration when updating to the new versions.

Child-Adolescent Suicidal Potential Index (CASPI): a screen for risk for early onset suicidal behavior

Abstract: This study's purpose was to develop a reliable and valid self-report questionnaire, the Child-Adolescent Suicidal Potential Index (CASPI), to screen for risk for suicidal behavior in children and adolescents. Four hundred twenty-five child and adolescent psychiatric patients and nonpatients completed the CASPI and other research instruments to rate suicidal and assaultive behavior and symptoms of depression, anxiety, and hopelessness. The 30-item CASPI involves 3 factors (anxious-impulsive depression, suicidal ideation or acts, family distress) that contributed to a unidimensional 2nd-order factor accounting for 59% of the total variance. Internal consistency (alpha) for the total score was .90, and test-retest reliability (ICC) for the total score was .76. Total score distinguished between children and adolescents with different severity of psychopathology and different levels of suicidal and assaultive behavior. Each of the 3 factors had different contributions to discriminating between levels of suicidal status. CASPI total score of 11 distinguished suicidal ideation or acts from nonsuicidal behavior, with sensitivity 70% and specificity 65%. CASPI total score positively correlated with symptom severity of depression, anxiety, and hopelessness.

Comparison of White, African American, Hispanic, and Asian children on the Naglieri Nonverbal Ability Test.

Abstract: This study examined differences between 3 matched samples of White (n = 2,306) and African American (n = 2,306), White (n = 1,176) and Hispanic (n = 1,176), and White (n = 466) and Asian (n = 466) children on the Naglieri Nonverbal Ability Test (NNAT; J. A. Naglieri, 1997a). The groups were selected from 22,620 children included in the NNAT standardization sample and matched on geographic region, socioeconomic status, ethnicity, and type of school setting (public or private). There was only a small difference between the NNAT scores for the White and African American samples (d ratio = .25) and minimal differences between the White and Hispanic (d ratio = .17) and between the White and Asian (d ratio = .02) groups. The NNAT was moderately correlated with achievement for the total sample and correlated similarly with achievement for the White and ethnic minority groups. The median correlation of NNAT with reading was .52 and NNAT with math was .63 across the samples. Results suggest that the NNAT scores have use for fair assessment of White and minority children.

Incremental validity of WISC-III profile elevation, scatter, and shape information for predicting reading and math achievement

Abstract: The use of cognitive subtest profiles to hypothesize about children's learning strengths and weaknesses implicitly assumes that subtest profiles are predictive of academic performance. To test this assumption, Wechsler Intelligence Scale for Children-Third Edition (WISC-III) subtest profiles were decomposed into elevation, scatter, and shape components and sequentially regressed onto reading and math achievement scores for 1,118 nonexceptional and 538 exceptional students. Profile elevation was statistically and practically significant for both exceptional (R = .36-.61) and nonexceptional (R = .72-.75) students. Profile scatter did not aid in the prediction of achievement. Profile shape accounted for an additional 5%-8% of the variance in achievement measures. It was concluded that using WISC-III subtest scatter and shape to predict academic performance was not supported by the accumulated scientific evidence.

Implications of test revisions for research.

Abstract: Test revisions are increasingly common in psychology and neuropsychology in particular. However, such revisions may alter in complex ways the kind of information obtained, and they may assess traits, abilities, and conditions in ways different from earlier versions. This article outlines some of the problems associated with the revision of tests facing clinicians and researchers. Three broad classes of revision are considered. Part 1 considers the aging of tests, part 2 concerns the aging of participants, and part 3 considers changes in test format. Although the article focuses largely on measures of intelligence and personality, the issues addressed in the article apply to other tests and assessment domains as well.

Computers will become increasingly important for psychological assessment: not that there's anything wrong with that!

Abstract: Though one can expect that computer programs will become increasingly important for psychological assessment, current automated assessment programs and statistical-prediction rules are of limited value. Validity has not been clearly established for many automated assessment programs. Statistical-prediction rules are of limited value because they have typically been based on limited information that has not been demonstrated to be optimal and they have almost never been shown to be powerful. Recommendations are made for building and evaluating new computer programs. Finally, comments are made about the ethics of using computers to make judgments.

Structure of the Weinberger Adjustment Inventory Self-Restraint scale and its relation to problem behaviors in adolescence.

Abstract: The authors examined the structure of the Weinberger Adjustment Inventory (WAI) Self-Restraint scale in derivation (n = 1,286) and cross-validation (n = 1,154) samples of mostly African American 6th graders in 3 urban schools. Four models were compared: (a) a 1-factor model; (b) a hierarchical model in which factors representing Impulse Control, Suppression of Aggression, Responsibility, and Consideration of Others were subsumed by a higher order factor; (c) a model that represented these 4 factors as correlated but distinct constructs; and (d) a model that excluded Consideration of Others from the higher order factor. Consistent support was found for the last model based on confirmatory factor analyses and latent-variable analyses examining the relations among self-restraint scales, drug use, delinquency, and aggression. These findings have implications for using the WAI, particularly in studies of adolescent problem behaviors.

Applying artificial neural network models to clinical decision making.

Abstract: Because psychological assessment typically lacks biological gold standards, it traditionally has relied on clinicians' expert knowledge. A more empirically based approach frequently has applied linear models to data to derive meaningful constructs and appropriate measures. Statistical inferences are then used to assess the generality of the findings. This article introduces artificial neural networks (ANNs), flexible nonlinear modeling techniques that test a model's generality by applying its estimates against "future" data. ANNs have potential for overcoming some shortcomings of linear models. The basics of ANNs and their applications to psychological assessment are reviewed. Two examples of clinical decision making are described in which an ANN is compared with linear models, and the complexity of the network performance is examined. Issues salient to psychological assessment are addressed.

The structure and correlates of anxiety sensitivity in older adults.

Abstract: Anxiety sensitivity (AS) has been defined as the fear of symptoms of anxiety and panic, and is most frequently assessed with the Anxiety Sensitivity Index (Peterson & Reiss, 1987). To investigate the nature and structure of AS in an older sample, data were collected from a sample of 322 adults aged 65 to 97, with mean age 75. Confirmatory factor analysis indicated a hierarchical structure with three group factors (physical concerns, mental incapacitation concerns, and social concerns), as well as a general factor, consistent with previous investigations. Results suggest that the nature and structure of the AS trait in older adults are highly similar to those of younger adults.