Convergent and incremental predictive validity of clinician, self-report, and structured interview diagnoses for personality disorders over 5 years.
Summary (2 min read)
Method
- Study participants were drawn from the 668 participants recruited from the multiple CLPS clinical sites.
- Participants who provided written, informed consent underwent diagnostic interviews and completed self-report questionnaires as part of a standardized battery.
- These PD diagnostic assignments were based on the Diagnostic Interview for DSM–IV Personality Disorders (DIPD-IV; Zanarini, Frankenburg, Sickel, & Yong, 1996), reliably administered by trained research personnel.
- Participants used for the current analyses were 320 individuals from the CLPS with available PAF ratings completed by a treating clinician at baseline.
- Baseline SNAP-2 PD scores were significantly greater for the studied group for all four PDs.
Personality Disorder Measures
- The DIPD-IV is a semistructured diagnostic interview for assessing the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM–IV; American Psychiatric Association, 1994) PDs.
- The current report considered only the DIPD-IV scores for the four PDs studied in CLPS.
- The SNAP-2 includes scales assessing the DSM–IV PDs, ranging in length from 19 to 34 items.
Psychosocial Functioning Measures Serving as Independent External Criteria
- Multiple measures of psychosocial functioning served as external outcome criteria.
- Both aspects are crucial for the current purposes, as independent, external criteria provide the only opportunity to discriminate validity among different methods of PD diagnosis.
- To assess psychosocial functioning, CLPS research team interviewers administered the LIFE (Keller et al., 1987), a structured interview assessing functioning in interpersonal relationships and occupational and recreational domains.
- Participants also completed the SAS-SR (Weissman & Bothwell, 1976), a self-report instrument yielding estimates of interpersonal, occupational, and recreational functioning.
- The LIFE and SAS-SR were administered at baseline and repeated at predetermined intervals, including the 5-year follow-up.
Data Analytic Procedures
- The authors first examined the convergent validity of clinicians’ PAF diagnoses with those from a semistructured diagnostic interview (DIPD-IV) and self-report questionnaire (SNAP-2).
- PAF dimensional ratings were compared with those from the DIPD-IV and SNAP-2 (all at baseline) for their ability to predict functional outcomes at the 60-month follow-up (via the LIFE and SAS-SR) using hierarchical regression analyses.
- This was then repeated with the order of entry reversed.
- To account for possible contamination due to shared method variance, the authors conducted these analyses separately using the selfreport criterion and again with the interview-based criterion variable.
- Al- though their use of functional outcomes rather than diagnostic information as criteria attenuates this possibility, the authors nonetheless examined it by performing a parallel set of analyses restricted to a subsample of 110 participants for whom the PAF disagreed with the DIPD-IV at baseline and thus was not required for study inclusion.
Categorical and Dimensional Agreement
- Table 1 provides the agreement between PAF ratings and those from the DIPD-IV and SNAP-2. Categorical agreement between treating clinicians’ diagnoses and the semistructured diagnostic interview ranged from of .21 to .42 , whereas dimensional agreement (Pearson correlations) ranged from .30 to .44 .
- Agreement between clinicians’ ratings and self-report questionnaire was lower than between clinicians’ ratings and semistructured diagnostic interviews, with kappas ranging from .00 (OCPD) to .20 and Pearson correlations ranging from .18 to .28 .
- For context, the authors note that agreement between DIPD-IV and SNAP-2 in the current sample ranged from .25 (OCPD) to .51 for categorical diagnoses and from .57 to .72 for dimensional ratings.
Incremental Predictive Validity
- Tables 2–6 summarize the hierarchical regression analyses.
- In contrast, clinicians’ ratings did not significantly increment the DIPD-IV interview results for either criterion.
- Diagnostic Interview for DSM–IV Personality Disorders; SNAP-2 !.
- Table 4 summarizes regression analyses comparing SNAP-2 ratings with the PAF in predicting functioning assessed by the SAS-SR and LIFE.
Diagnostic Ratings?
- To determine whether clinicians’ familiarity with their patients influenced findings, the authors repeated these analyses using a subsample of clinicians who treated patients for more than 1 year.
- Ninety clinicians had this level of familiarity, and 60-month follow-up data were available for 73 using the LIFE and 62 using the.
Discussion
- In the current study, the authors explicitly compared the value of PD diagnoses provided by clinicians via the PAF with those from a semistructured interview and self-report questionnaire for prospectively predicting psychosocial functioning in a large clinical sample.
- Outcome variables are means of interpersonal, recreation, and work functioning assessed by the SAS-SR and LIFE, respectively.
- Nonetheless, their findings suggest that clinicians’ naturalistic PD diagnoses, even when recorded in a standardized format such as the PAF, do not agree well with other methods.
- PD ratings provides important evidence supporting the utility of self-report instruments for assessing personality pathology.
Limitations
- In the current study, the authors examined a large, carefully diagnosed clinical sample with well-validated criterion measures to provide the first data on the relative validity of clinicians’.
- This sampling strategy excluded other potential participants relevant to the current analyses, such as individuals diagnosed with a PD by the PAF but not according to the DIPD-IV.
- Nonetheless, it would have been ideal to collect clinician ratings of functioning that could have been used as another criterion.
- The current study helps to demonstrate that PD diagnoses, provided by any source or method, do predict subsequent impairment.
- The degree to which participants’ functioning improved through treatment would diminish the predictive utility of a baseline diagnosis.
Conclusions
- Most importantly, their novel findings provide evidence that the latter two methods have greater utility than clinicians’.
- PD diagnoses for predicting psychosocial functioning over 5-year prospective follow-up.
- These findings underscore the advantages of incorporating established semistructured interviews and self-report questionnaires into routine clinical diagnostic practices.
Did you find this useful? Give us your feedback
Citations
126 citations
Cites background from "Convergent and incremental predicti..."
...These can mitigate the issue of feasibility by reducing provider burden without necessarily com promising validity (Samuel et al., 2013; Samuel, Suzuki, & Grif fin, 2016)....
[...]
...These can mitigate the issue of feasibility by reducing provider burden without necessarily compromising validity (Samuel et al., 2013; Samuel, Suzuki, & Griffin, 2016)....
[...]
82 citations
Cites methods from "Convergent and incremental predicti..."
...Although self-reports of PDs do appear to have considerable validity (Samuel et al., 2013), it would be useful to utilize other methods of assessing these traits....
[...]
35 citations
34 citations
Cites background from "Convergent and incremental predicti..."
...For example, self-perceptions differ from informant observations for PDs (Carlson, Vazire, & Oltmanns, 2013); the level of agreement between and predictive validity of self-report, structured interview, and clinician ratings often vary for PDs (Samuel et al., 2013); and clinicians and PD patients diverge in their assessments of variables related to PD (Gritti, Samuel, & Lang, 2016)....
[...]
...…& Oltmanns, 2013); the level of agreement between and predictive validity of self-report, structured interview, and clinician ratings often vary for PDs (Samuel et al., 2013); and clinicians and PD patients diverge in their assessments of variables related to PD (Gritti, Samuel, & Lang, 2016)....
[...]
...Such shortcomings include, but are not limited to, high comorbidity rates, excessive heterogeneity within PDs, and a lack of empirical support for categorical taxonomic structure (see Krueger, Hopwood, Wright, & Markon, 2014)....
[...]
26 citations
Cites result from "Convergent and incremental predicti..."
...Nonetheless, multisource and multiobservation studies on personality taxonomy have been consistent with findings of self-report studies (e.g., Kandler, Bleidorn, Riemann, Angleitner, & Spinath, 2011; Samuel et al., 2013), so these results likely are highly generalizable....
[...]
References
67,017 citations
7,254 citations
1,715 citations
"Convergent and incremental predicti..." refers methods in this paper
...Participants also completed the SAS-SR (Weissman & Bothwell, 1976), a self-report instrument yielding estimates of interpersonal, occupational, and recreational functioning....
[...]
1,521 citations
"Convergent and incremental predicti..." refers methods in this paper
...To assess psychosocial functioning, CLPS research team interviewers administered the LIFE (Keller et al., 1987), a structured interview assessing functioning in interpersonal relationships and occupational and recreational domains....
[...]
Related Papers (5)
Frequently Asked Questions (2)
Q2. What future works have the authors mentioned in the paper "Convergent and incremental predictive validity of clinician, self-report, and structured interview diagnoses for personality disorders over 5 years" ?
Future research that examines the incremental predictive validity of clinicians ’ diagnoses derived from more structured assessments, such as therapists completing the SWAP ( Westen & Shedler, 1999 ), the Personality Disorder Schedule ( Nestadt et al., 2012 ), or even an informant version of an existing PD questionnaire, would help to address this possibility. Future research exploring the validity of other psychiatric diagnoses provided by clinicians in routine practice warrants attention. Rather, their results suggest that clinicians use standardized assessment instruments to inform PD diagnoses.