scispace - formally typeset
Search or ask a question
Journal ArticleDOI

The validity of the multi-informant approach to assessing child and adolescent mental health.

27 Apr 2015-Psychological Bulletin (American Psychological Association)-Vol. 141, Iss: 4, pp 858-900
TL;DR: In this article, the authors meta-analyzed 341 studies published between 1989 and 2014 that reported cross-informant correspondence estimates, and observed low-to-moderate correspondence (mean internalizing: r =.25; mean externalizing: R =.30; mean overall: R.28).
Abstract: Child and adolescent patients may display mental health concerns within some contexts and not others (e.g., home vs. school). Thus, understanding the specific contexts in which patients display concerns may assist mental health professionals in tailoring treatments to patients' needs. Consequently, clinical assessments often include reports from multiple informants who vary in the contexts in which they observe patients' behavior (e.g., patients, parents, teachers). Previous meta-analyses indicate that informants' reports correlate at low-to-moderate magnitudes. However, is it valid to interpret low correspondence among reports as indicating that patients display concerns in some contexts and not others? We meta-analyzed 341 studies published between 1989 and 2014 that reported cross-informant correspondence estimates, and observed low-to-moderate correspondence (mean internalizing: r = .25; mean externalizing: r = .30; mean overall: r = .28). Informant pair, mental health domain, and measurement method moderated magnitudes of correspondence. These robust findings have informed the development of concepts for interpreting multi-informant assessments, allowing researchers to draw specific predictions about the incremental and construct validity of these assessments. In turn, we critically evaluated research on the incremental and construct validity of the multi-informant approach to clinical child and adolescent assessment. In so doing, we identify crucial gaps in knowledge for future research, and provide recommendations for "best practices" in using and interpreting multi-informant assessments in clinical work and research. This article has important implications for developing personalized approaches to clinical assessment, with the goal of informing techniques for tailoring treatments to target the specific contexts where patients display concerns. (PsycINFO Database Record
Citations
More filters
Journal ArticleDOI
TL;DR: It is concluded that behavioral parent training, behavioral classroom management, and behavioral peer interventions are well-established treatments for children and adolescents with attention-deficit/hyperactivity disorder.
Abstract: The purpose of this research was to update the series of articles on evidence-based treatment for children and adolescents with attention deficit/hyperactivity disorder that have appeared in this journal (Evans, Owens & Bunford, 2014; Pelham & Fabiano, 2008; Pelham, Wheeler, & Chronis, 1998). We completed a systematic review of the literature published between 2012 and 2016 to establish levels of evidence for psychosocial treatments for these youth. We identified articles using criteria established by the Society of Clinical Child and Adolescent Psychology using keyword searches of abstracts and titles. Articles were classified according to a modified version of the Division 12 task force guidelines that was used in other reviews in this series. The results revealed that findings are becoming increasingly nuanced with variations in levels of evidence related to ages of the children and characteristics of the specific treatment. In addition, we focused our critique on generalization of treatment effects across settings and time and on sample diversity (with regard to ethnicity and levels of parent education) in relation to the population. Children of parents with higher levels of education than average appear to be overrepresented in the literature. Implications for future treatment development and evaluation and for dissemination research are discussed.

595 citations

Journal ArticleDOI
TL;DR: The findings show that dark triad traits are substantially intercorrelated, somewhat more prevalent among men than women, predominantly related to the Big Five personality factor of agreeableness and the HEXACO factor of honesty-humility, and generally associated with various types of negative psychosocial outcomes.
Abstract: The term dark triad refers to the constellation of narcissism, Machiavellianism, and psychopathy. Over the past few years, the concept has gained momentum, with many researchers assuming that the dark triad is a prominent antecedent of transgressive and norm-violating behavior. Our purpose in this meta-analytic review was to evaluate (a) interrelations among narcissism, Machiavellianism, and psychopathy; (b) gender differences in these traits; (c) how these traits are linked to normal personality factors; and (d) the psychosocial correlates of the dark triad. Our findings show that dark triad traits are substantially intercorrelated, somewhat more prevalent among men than women, predominantly related to the Big Five personality factor of agreeableness and the HEXACO factor of honesty-humility, and generally associated with various types of negative psychosocial outcomes. We question whether dark triad traits are sufficiently distinct and argue that the way they are currently measured is too simple to capture the malevolent sides of personality. Because most research in this domain is cross-sectional and based on self-reports, we recommend using a cross-informant approach and prospective, longitudinal research designs for studying the predictive value of dark triad features.

503 citations

Journal ArticleDOI
TL;DR: Overall effect size (ES) and moderator effects were assessed using multilevel modeling to address ES dependency that is common, but typically not modeled, in meta-analyses, and only youth-focused behavioral therapies showed similar and robust effects across youth, parent, and teacher reports.
Abstract: Across 5 decades, hundreds of randomized trials have tested psychological therapies for youth internalizing (anxiety, depression) and externalizing (misconduct, attention deficit and hyperactivity disorder) disorders and problems. Since the last broad-based youth meta-analysis in 1995, the number of trials has almost tripled and data-analytic methods have been refined. We applied these methods to the expanded study pool (447 studies; 30,431 youths), synthesizing 50 years of findings and identifying implications for research and practice. We assessed overall effect size (ES) and moderator effects using multilevel modeling to address ES dependency that is common, but typically not modeled, in meta-analyses. Mean posttreatment ES was 0.46; the probability that a youth in the treatment condition would fare better than a youth in the control condition was 63%. Effects varied according to multiple moderators, including the problem targeted in treatment: Mean ES at posttreatment was strongest for anxiety (0.61), weakest for depression (0.29), and nonsignificant for multiproblem treatment (0.15). ESs differed across control conditions, with "usual care" emerging as a potent comparison condition, and across informants, highlighting the need to obtain and integrate multiple perspectives on outcome. Effects of therapy type varied by informant; only youth-focused behavioral therapies (including cognitive-behavioral therapy) showed similar and robust effects across youth, parent, and teacher reports. Effects did not differ for Caucasian versus minority samples, but more diverse samples are needed. The findings underscore the benefits of psychological treatments as well as the need for improved therapies and more representative, informative, and rigorous intervention science. (PsycINFO Database Record

495 citations

Journal ArticleDOI
TL;DR: This update of Clark and Watson (1995) provides a synopsis of major points of an earlier article and discusses issues in scale construction that have become more salient as clinical and personality assessment has progressed over the past quarter-century.
Abstract: In this update of Clark and Watson (1995), we provide a synopsis of major points of our earlier article and discuss issues in scale construction that have become more salient as clinical and personality assessment has progressed over the past quarter-century. It remains true that the primary goal of scale development is to create valid measures of underlying constructs and that Loevinger's theoretical scheme provides a powerful model for scale development. We still discuss practical issues to help developers maximize their measures' construct validity, reiterating the importance of (a) clear conceptualization of target constructs, (b) an overinclusive initial item pool, (c) paying careful attention to item wording, (d) testing the item pool against closely related constructs, (e) choosing validation samples thoughtfully, and (f) emphasizing unidimensionality over internal consistency. We have added (g) consideration of the hierarchical structures of personality and psychopathology in scale development, discussion of (h) codeveloping scales in the context of these structures, (i) "orphan," and "interstitial" constructs, which do not fit neatly within these structures, (j) problems with "conglomerate" constructs, and (k) developing alternative versions of measures, including short forms, translations, informant versions, and age-based adaptations. Finally, we have expanded our discussions of (l) item-response theory and of external validity, emphasizing (m) convergent and discriminant validity, (n) incremental validity, and (o) cross-method analyses, such as questionnaires and interviews. We conclude by reaffirming that all mature sciences are built on the bedrock of sound measurement and that psychology must redouble its efforts to develop reliable and valid measures. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

310 citations

Journal ArticleDOI
TL;DR: A series of recommendations for facilitating cross-study comparisons and leveraging multifaceted approaches to studying emotion regulation processes within a developmental psychopathology framework are provided.
Abstract: In response to rapidly growing rates of comorbidity among psychiatric disorders, clinical scientists have become interested in identifying transdiagnostic processes that can help explain dysfunction across diagnostic categories (e.g., Kring & Sloan, 2009). One factor that has received a great deal of attention is that of emotion regulation, namely, the ability to modulate the intensity and/or duration of emotional states (e.g., Cicchetti, Ackerman, & Izard, 1995; Gross, 1998). Recent theoretical and empirical work has begun to emphasize the role that emotion regulation plays in the temporal comorbidity between internalizing and externalizing conditions (e.g., Aldao & De Los Reyes, 2015; De Los Reyes & Aldao, 2015; Drabick & Kendall, 2010; Jarrett & Ollendick, 2008; Patrick & Hajcak, 2016). However, close inspection of this work reveals two very pertinent areas of growth: (a) this literature is characterized by mixed findings that are likely explained, in part, by methodological heterogeneity; and (b) emotion regulation tends to be studied in relatively narrow terms. To address these issues, we provide a series of recommendations for facilitating cross-study comparisons and leveraging multifaceted approaches to studying emotion regulation processes within a developmental psychopathology framework. We hope that our perspective can enhance the organization and growth of this very important area of inquiry, and ultimately result in more effective prevention and treatment programs.

282 citations

References
More filters
Journal ArticleDOI
04 Sep 2003-BMJ
TL;DR: A new quantity is developed, I 2, which the authors believe gives a better measure of the consistency between trials in a meta-analysis, which is susceptible to the number of trials included in the meta- analysis.
Abstract: Cochrane Reviews have recently started including the quantity I 2 to help readers assess the consistency of the results of studies in meta-analyses. What does this new quantity mean, and why is assessment of heterogeneity so important to clinical practice? Systematic reviews and meta-analyses can provide convincing and reliable evidence relevant to many aspects of medicine and health care.1 Their value is especially clear when the results of the studies they include show clinically important effects of similar magnitude. However, the conclusions are less clear when the included studies have differing results. In an attempt to establish whether studies are consistent, reports of meta-analyses commonly present a statistical test of heterogeneity. The test seeks to determine whether there are genuine differences underlying the results of the studies (heterogeneity), or whether the variation in findings is compatible with chance alone (homogeneity). However, the test is susceptible to the number of trials included in the meta-analysis. We have developed a new quantity, I 2, which we believe gives a better measure of the consistency between trials in a meta-analysis. Assessment of the consistency of effects across studies is an essential part of meta-analysis. Unless we know how consistent the results of studies are, we cannot determine the generalisability of the findings of the meta-analysis. Indeed, several hierarchical systems for grading evidence state that the results of studies must be consistent or homogeneous to obtain the highest grading.2–4 Tests for heterogeneity are commonly used to decide on methods for combining studies and for concluding consistency or inconsistency of findings.5 6 But what does the test achieve in practice, and how should the resulting P values be interpreted? A test for heterogeneity examines the null hypothesis that all studies are evaluating the same effect. The usual test statistic …

45,105 citations

Journal ArticleDOI
TL;DR: This transmutability of the validation matrix argues for the comparisons within the heteromethod block as the most generally relevant validation data, and illustrates the potential interchangeability of trait and method components.
Abstract: Content Memory (Learning Ability) As Comprehension 82 Vocabulary Cs .30 ( ) .23 .31 ( ) .31 .31 .35 ( ) .29 .48 .35 .38 ( ) .30 .40 .47 .58 .48 ( ) As judged against these latter values, comprehension (.48) and vocabulary (.47), but not memory (.31), show some specific validity. This transmutability of the validation matrix argues for the comparisons within the heteromethod block as the most generally relevant validation data, and illustrates the potential interchangeability of trait and method components. Some of the correlations in Chi's (1937) prodigious study of halo effect in ratings are appropriate to a multitrait-multimethod matrix in which each rater might be regarded as representing a different method. While the published report does not make these available in detail because it employs averaged values, it is apparent from a comparison of his Tables IV and VIII that the ratings generally failed to meet the requirement that ratings of the same trait by different raters should correlate higher than ratings of different traits by the same rater. Validity is shown to the extent that of the correlations in the heteromethod block, those in the validity diagonal are higher than the average heteromethod-heterotrait values. A conspicuously unsuccessful multitrait-multimethod matrix is provided by Campbell (1953, 1956) for rating of the leadership behavior of officers by themselves and by their subordinates. Only one of 11 variables (Recognition Behavior) met the requirement of providing a validity diagonal value higher than any of the heterotrait-heteromethod values, that validity being .29. For none of the variables were the validities higher than heterotrait-monomethod values. A study of attitudes toward authority and nonauthority figures by Burwen and Campbell (1957) contains a complex multitrait-multimethod matrix, one symmetrical excerpt from which is shown in Table 6. Method variance was strong for most of the procedures in this study. Where validity was found, it was primarily at the level of validity diagonal values higher than heterotrait-heteromethod values. As illustrated in Table 6, attitude toward father showed this kind of validity, as did attitude toward peers to a lesser degree. Attitude toward boss showed no validity. There was no evidence of a generalized attitude toward authority which would include father and boss, although such values as the VALIDATION BY THE MULTITRAIT-MULTIMETHOD MATRIX

15,795 citations

Journal ArticleDOI
13 Jan 1996-BMJ
TL;DR: Evidence Based Medicine (IBM) as discussed by the authors is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients, which is a hot topic for clinicians, public health practitioners, purchasers, planners and the public.
Abstract: It's about integrating individual clinical expertise and the best external evidence Evidence based medicine, whose philosophical origins extend back to mid-19th century Paris and earlier, remains a hot topic for clinicians, public health practitioners, purchasers, planners, and the public. There are now frequent workshops in how to practice and teach it (one sponsored by the BMJ will be held in London on 24 April); undergraduate1 and postgraduate2 training programmes are incorporating it3 (or pondering how to do so); British centres for evidence based practice have been established or planned in adult medicine, child health, surgery, pathology, pharmacotherapy, nursing, general practice, and dentistry; the Cochrane Collaboration and Britain's Centre for Review and Dissemination in York are providing systematic reviews of the effects of health care; new evidence based practice journals are being launched; and it has become a common topic in the lay media. But enthusiasm has been mixed with some negative reaction.4 5 6 Criticism has ranged from evidence based medicine being old hat to it being a dangerous innovation, perpetrated by the arrogant to serve cost cutters and suppress clinical freedom. As evidence based medicine continues to evolve and adapt, now is a useful time to refine the discussion of what it is and what it is not. Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The …

12,134 citations

Journal ArticleDOI
TL;DR: The QUADAS-2 tool will allow for more transparent rating of bias and applicability of primary diagnostic accuracy studies.
Abstract: In 2003, the QUADAS tool for systematic reviews of diagnostic accuracy studies was developed. Experience, anecdotal reports, and feedback suggested areas for improvement; therefore, QUADAS-2 was developed. This tool comprises 4 domains: patient selection, index test, reference standard, and flow and timing. Each domain is assessed in terms of risk of bias, and the first 3 domains are also assessed in terms of concerns regarding applicability. Signalling questions are included to help judge risk of bias. The QUADAS-2 tool is applied in 4 phases: summarize the review question, tailor the tool and produce review-specific guidance, construct a flow diagram for the primary study, and judge bias and applicability. This tool will allow for more transparent rating of bias and applicability of primary diagnostic accuracy studies.

8,370 citations