Principles Underlying the Use of Multiple Informants' Reports
TL;DR: In this review, the authors advance a framework (Operations Triad Model) outlining general principles for using and interpreting informants' reports and provide supportive evidence for this framework and discuss its implications for hypothesis testing, study design, and quantitative review.
Abstract: Researchers use multiple informants' reports to assess and examine behavior. However, informants' reports commonly disagree. Informants' reports often disagree in their perceived levels of a behavior (“low” versus “elevated” mood), and examining multiple reports in a single study often results in inconsistent findings. Although researchers often espouse taking a multi-informant assessment approach, they frequently address informant discrepancies using techniques that treat discrepancies as measurement error. Yet, recent work indicates that researchers in a variety of fields often may be unable to justify treating informant discrepancies as measurement error. In this review, the authors advance a framework (Operations Triad Model) outlining general principles for using and interpreting informants' reports. Using the framework, researchers can test whether or not they can extract meaningful information about behavior from discrepancies among multiple informants' reports. The authors provide supportive evide...
Citations
More filters
TL;DR: An examines methodologies suited to identify such symptom networks and discusses network analysis techniques that may be used to extract clinically and scientifically useful information from such networks (e.g., which symptom is most central in a person's network).
Abstract: In network approaches to psychopathology, disorders result from the causal interplay between symptoms (e.g., worry → insomnia → fatigue), possibly involving feedback loops (e.g., a person may engage in substance abuse to forget the problems that arose due to substance abuse). The present review examines methodologies suited to identify such symptom networks and discusses network analysis techniques that may be used to extract clinically and scientifically useful information from such networks (e.g., which symptom is most central in a person's network). The authors also show how network analysis techniques may be used to construct simulation models that mimic symptom dynamics. Network approaches naturally explain the limited success of traditional research strategies, which are typically based on the idea that symptoms are manifestations of some common underlying factor, while offering promising methodological alternatives. In addition, these techniques may offer possibilities to guide and evaluate therape...
1,824 citations
TL;DR: In this article, the authors meta-analyzed 341 studies published between 1989 and 2014 that reported cross-informant correspondence estimates, and observed low-to-moderate correspondence (mean internalizing: r =.25; mean externalizing: R =.30; mean overall: R.28).
Abstract: Child and adolescent patients may display mental health concerns within some contexts and not others (e.g., home vs. school). Thus, understanding the specific contexts in which patients display concerns may assist mental health professionals in tailoring treatments to patients' needs. Consequently, clinical assessments often include reports from multiple informants who vary in the contexts in which they observe patients' behavior (e.g., patients, parents, teachers). Previous meta-analyses indicate that informants' reports correlate at low-to-moderate magnitudes. However, is it valid to interpret low correspondence among reports as indicating that patients display concerns in some contexts and not others? We meta-analyzed 341 studies published between 1989 and 2014 that reported cross-informant correspondence estimates, and observed low-to-moderate correspondence (mean internalizing: r = .25; mean externalizing: r = .30; mean overall: r = .28). Informant pair, mental health domain, and measurement method moderated magnitudes of correspondence. These robust findings have informed the development of concepts for interpreting multi-informant assessments, allowing researchers to draw specific predictions about the incremental and construct validity of these assessments. In turn, we critically evaluated research on the incremental and construct validity of the multi-informant approach to clinical child and adolescent assessment. In so doing, we identify crucial gaps in knowledge for future research, and provide recommendations for "best practices" in using and interpreting multi-informant assessments in clinical work and research. This article has important implications for developing personalized approaches to clinical assessment, with the goal of informing techniques for tailoring treatments to target the specific contexts where patients display concerns. (PsycINFO Database Record
885 citations
Posted Content•
TL;DR: This article critically evaluated research on the incremental and construct validity of the multi-informant approach to clinical child and adolescent assessment, and identified crucial gaps in knowledge for future research, and provided recommendations for "best practices" in using and interpreting multi-Informant assessments in clinical work and research.
Abstract: Child and adolescent patients may display mental health concerns within some contexts and not others (e.g., home vs. school). Thus, understanding the specific contexts in which patients display concerns may assist mental health professionals in tailoring treatments to patients’ needs. Consequently, clinical assessments often include reports from multiple informants who vary in the contexts in which they observe patients’ behavior (e.g., patients, parents, teachers). Previous meta-analyses indicate that informants’ reports correlate at low-to-moderate magnitudes. However, is it valid to interpret low correspondence among reports as indicating that patients display concerns in some contexts and not others? We meta-analyzed 341 studies published between 1989 and 2014 that reported cross-informant correspondence estimates, and observed low-to-moderate correspondence (mean internalizing: r = .25; mean externalizing: r = .30; mean overall: r = .28). Informant pair, mental health domain, and measurement method moderated magnitudes of correspondence. These robust findings have informed the development of concepts for interpreting multi-informant assessments, allowing researchers to draw specific predictions about the incremental and construct validity of these assessments. In turn, we critically evaluated research on the incremental and construct validity of the multi-informant approach to clinical child and adolescent assessment. In so doing, we identify crucial gaps in knowledge for future research, and provide recommendations for “best practices” in using and interpreting multi-informant assessments in clinical work and research. This article has important implications for developing personalized approaches to clinical assessment, with the goal of informing techniques for tailoring treatments to target the specific contexts where patients display concerns.
815 citations
TL;DR: The current state of the literature regarding psychosocial treatments for schizophrenia is reviewed within the frameworks of the recovery model of mental health and the expanded stress-vulnerability model.
Abstract: The current state of the literature regarding psychosocial treatments for schizophrenia is reviewed within the frameworks of the recovery model of mental health and the expanded stress-vulnerability model. Interventions targeting specific domains of functioning, age groups, stages of illness, and human service system gaps are classified as evidence-based practices or promising practices according to the extent to which their efficacy is currently supported by meta-analyses and individual randomized controlled trials (RCTs). Evidence-based practices include assertive community treatment (ACT), cognitive behavior therapy (CBT) for psychosis, cognitive remediation, family psychoeducation, illness self-management training, social skills training, and supported employment. Promising practices include cognitive adaptive therapy, CBT for posttraumatic stress disorder, first-episode psychosis intervention, healthy lifestyle interventions, integrated treatment for co-occurring disorders, interventions targeting older individuals, peer support services, physical disease management, prodromal stage intervention, social cognition training, supported education, and supported housing. Implications and future directions are discussed.
306 citations
TL;DR: The interpersonal characteristics, risk factors, and consequences of depression in the context of the relevant theories that address the role of interpersonal processes in the onset, maintenance, and chronicity of depression are summarized.
Abstract: Humans have an intrinsic need for social connection; thus, it is crucial to understand depression in an interpersonal context. Interpersonal theories of depression posit that depressed individuals tend to interact with others in a way that elicits rejection, which increases their risk for future depression. In this review, we summarize the interpersonal characteristics, risk factors, and consequences of depression in the context of the relevant theories that address the role of interpersonal processes in the onset, maintenance, and chronicity of depression. Topics reviewed include social skills, behavioral features, communication behaviors, interpersonal feedback seeking, and interpersonal styles as they relate to depression. Treatment implications are discussed in light of the current research on interpersonal processes in depression, and the following future directions are discussed: developing integrative models of depression, improving measurement of interpersonal constructs, examining the association between interpersonal processes in depression and suicide, and tailoring interventions to target interpersonal processes in depression.
294 citations
References
More filters
TL;DR: The existing paradigm for evaluating the evidence base of intervention may need modification to permit a more intricate analysis of the extant research, and increase the sophistication of future research.
Abstract: A consistent result in the evaluation of group-delivered intervention to promote social reciprocity in children with PDDs is that outcome data are inconclusive. Lack of robust evidence of efficacy confounds understanding of these interventions and their value to the field. It is conceivable that the construct of impaired social reciprocity in PDD presents unique circumstances that require special consideration when evaluating the evidence base. Social reciprocity and impairment in social functioning are complex constructs, which require a multi-dimensional, multi-method approach to intervention and measurement of gains. The existing paradigm for evaluating the evidence base of intervention may need modification to permit a more intricate analysis of the extant research, and increase the sophistication of future research.
94 citations
"Principles Underlying the Use of Mu..." refers background or result in this paper
...However, controlled trials often reveal inconsistent findings as to an intervention’s effects (De Los Reyes & Kazdin, 2006, 2008, 2009; Koenig et al. 2009)....
[...]
...Specifically, as mentioned previously, controlled trials often yield inconsistent conclusions regarding the efficacy of treatments examined within these trials, particularly among multiple informants’ outcome reports (De Los Reyes & Kazdin 2006, Koenig et al. 2009)....
[...]
TL;DR: The relations of parents' and teachers' reports of temperament anger-irritability, positive emotionality, and effortful control to children's externalizing and internalizing problems were examined and suggest that there are considerable cross-cultural similarities in the temperament-adjustment associations.
Abstract: The relations of parents' and teachers' reports of temperament anger-irritability, positive emotionality, and effortful control (attention focusing and inhibitory control) to children's externalizing and internalizing problems were examined in Chinese (N = 382) and U.S. (N = 322) samples of school-age children. Results suggested that in both cultures, low effortful control and high anger-irritability were associated with high externalizing problems, although the relations were stronger in the Chinese sample than in the U.S. sample. Low positive emotionality was associated with high internalizing problems in both cultures. However, high positive emotionality was associated with noncomorbid externalizing problems (teachers' reports) in the Chinese sample but not in the U.S. sample. These findings suggest that there are considerable cross-cultural similarities in the temperament-adjustment associations, although some cross-cultural differences might exist. Implications of the findings for the detection and intervention of adjustment problems in Chinese children are discussed.
93 citations
"Principles Underlying the Use of Mu..." refers background in this paper
...P sy ch by U ni ve rs ity o f M ar CP09CH05-DeLosReyes ARI 24 February 2013 10:48...
[...]
...…focus on examining the variance shared among multiple informants’ reports of the same behavior (e.g., Arseneault et al. 2003, Holmbeck et al. 2002, Zhou et al. 2009).1 Third, in controlled trials of treatment efficacy, researchers often select, in advance of a trial, a single outcome indicator…...
[...]
93 citations
"Principles Underlying the Use of Mu..." refers background or methods in this paper
...Yet, recent work indicates that researchers in a variety of fields often may be unable to justify treating informant discrepancies as measurement error....
[...]
...Informants’ reports often disagree in their perceived levels of a behavior (“low” versus “elevated” mood), and examining multiple reports in a single study often results in inconsistent findings....
[...]
...Specifically, variations among multiple informants’ reports of the same behavior can be seen as “error” around a “true score mean” representation of the behavior being assessed (Edgeworth 1888)....
[...]
...Edgeworth (1888) translated the concept of measurement error from the physical sciences for use in the study of mental states, and it ultimately formed a key component of classical test theories of psychological measurement (Borsboom 2005)....
[...]
TL;DR: In this paper, longitudinal multiple-rater data on internalizing problems in twins was used to make a distinction between measurement error and the raters' unique perception of the child's behavior.
Abstract: Genetically informative data can be used to address fundamental questions concerning the measurement of behavior in children. The authors illustrate this with longitudinal multiple-rater data on internalizing problems in twins. Valid information on the behavior of a child is obtained for behavior that multiple raters agree upon and for rater-specific perception of the child's behavior. Rater-disagreement variance varsigma2(rd) accounted for 35% of the individual differences in internalizing behavior. Up to 17% of this varsigma2(rd) was accounted for by rater-specific additive genetic variance varsigma2(Au). Thus, the disagreement should not be considered only to be bias/error but also as representing the unique feature of the relationships between that parent and the child. The longitudinal extension of this model helps to make a distinction between measurement error and the raters' unique perception of the child's behavior. For internalizing behavior, the results show large stability across time, which is accounted for by common additive genetic and common shared environmental factors. Rater-specific shared environmental factors show substantial influence on stability. This could mean that rater bias may be persistent and affect longitudinal studies.
89 citations
"Principles Underlying the Use of Mu..." refers background in this paper
...Specifically, researchers may construct structural models that partition variance reflecting genetic influences on the expression of the assessed behavior, shared environmental influences, and nonshared environmental influences, in addition to variance explained by error (Bartels et al. 2007)....
[...]
...1Recently, researchers in the behavior genetics literature have tested whether unshared variance reflects important information unique to informants, rather than measurement error (Bartels et al. 2007)....
[...]
...Thus, within structural models seeking to explain variance in behavior, behavior genetics researchers may treat informant discrepancies as useful information (Baker et al. 2007, Bartels et al. 2007, Derks et al. 2006, Tackett et al. 2009)....
[...]
TL;DR: Child Behavior Checklist questionnaires were collected for an effective sample of 3,501 Dutch 3-year-old twin pairs to disentangle the child's phenotype from that of the rater, and two contrasting models were fitted to the data.
Abstract: Child Behavior Checklist questionnaires (Achenbach, 1992), filled in separately by mothers and fathers, were collected for an effective sample of 3501 Dutch 3-year-old twin pairs. To disentangle the child's phenotype from that of the rater, two contrasting models were fitted to the data. One model, called a Rater Bias model, is based on the assumption that both parents assess exactly the same behaviors in the child. A weaker alternative of this model, called a Psychometric model, assumes that apart from these common behavioral views, each parent also assesses a unique aspect of the child's behavior. A Psychometric model fitted the data of both Internalizing and Externalizing scales significantly better than a Rater Bias model. This implied that each parent provided unique information from his or her own perspective, apart from the common behavioral view. Using this best fitting model, the etiology of both the Internalizing and Externalizing scales was studied. Common factors (influencing behaviors similarly assessed by both parents) were more important than unique factors (influencing behaviors uniquely assessed by one parent). Common genetic factors explained about 50% of the variance of both scales, indicating a possible inborn vulnerability to childhood psychopathology. Common environmental factors not shared between twins (free of unreliability and error) explained around 14% of both scales, suggesting the importance of pure idiosyncratic experiences even for children as young as 3 years. Common environmental factors shared between twins (unconfounded by rater bias) were only found for the Externalizing scale, explaining 18% of the variance. Rater bias and unreliability, if present in the data, were included in the estimates of the unique factors. Unique genetic, shared, and nonshared environmental factors each explained around 8% of the variance for both scales. These small effects could be detected because of the large sample of twin pairs used.
87 citations
"Principles Underlying the Use of Mu..." refers background in this paper
...At the same time, as alluded to previously, not all literatures treat unshared variance in structural equations models in this way (cf. Holmbeck et al. 2002, Valk et al. 2001)....
[...]