Journal ArticleDOI
Sources of unreliability in depression ratings.
Kenneth A. Kobak,Brianne Brown,Ian R. Sharp,Hollie Levy-Mack,Kurrie Wells,Felice Ockun,Janet B. W. Williams +6 more
Reads0
Chats0
TLDR
Experienced and uncalibrated raters should focus on establishing common conventions, whereas experienced and calibrated ratersShould focus on fine tuning judgment calls on different thresholds of symptoms, and calibration training seems to improve reliability over experience alone.Abstract:
Background Good interrater reliability is essential to minimize error variance and improve study power. Reasons why raters differ in scoring the same patient include information variance (different information obtained because of asking different questions), observation variance (the same information is obtained, but raters differ in what they notice and remember), interpretation variance (differences in the significance attached to what is observed), criterion variance (different criteria used to score items), and subject variance (true differences in the subject). We videotaped and transcribed 30 pairs of interviews to examine the most common sources of rater unreliability. Method Thirty patients who experienced depression were independently interviewed by 2 different raters on the same day. Raters provided rationales for their scoring, and independent assessors reviewed the rationales, the interview transcripts, and the videotapes to code the main reason for each discrepancy. One third of the interviews were conducted by raters who had not administered the Hamilton Depression Rating Scale before; one third, by raters who were experienced but not calibrated; and one third, by experienced and calibrated raters. Results Experienced and calibrated raters had the highest interrater reliability (intraclass correlation [ICC]; r = 0.93) followed by inexperienced raters (r = 0.77) and experienced but uncalibrated raters (r = 0.55). The most common reason for disagreement was interpretation variance (39%), followed by information variance (30%), criterion variance (27%), and observation variance (4%). Experienced and calibrated raters had significantly less criterion variance than the other cohorts (P = 0.001). Conclusions Reasons for disagreement varied by level of experience and calibration. Experienced and uncalibrated raters should focus on establishing common conventions, whereas experienced and calibrated raters should focus on fine tuning judgment calls on different thresholds of symptoms. Calibration training seems to improve reliability over experience alone. Experienced raters without cohort calibration had lower reliability than inexperienced raters.read more
Citations
More filters
Journal ArticleDOI
Is it valid to measure suicidal ideation by depression rating scales
Martin Desseilles,Martin Desseilles,Martin Desseilles,Nader Perroud,Sébastien Guillaume,Isabelle Jaussent,Catherine Genty,Alain Malafosse,Philippe Courtet +8 more
TL;DR: The findings suggest that the use of a single suicide item or a dimensional factor derived from a depression scale might be a valid approach to assess the suicidal ideations.
Journal ArticleDOI
Feasibility and Validation of a Computer-Automated Columbia-Suicide Severity Rating Scale Using Interactive Voice Response Technology
James C. Mundt,John H. Greist,Alan J. Gelenberg,David J. Katzelnick,James W. Jefferson,Jack G. Modell +5 more
TL;DR: Data obtained from relatively small patient samples recruited from a single investigational site over a relatively short follow- up period support the feasibility and validity of the eC-SSRS for prospective monitoring of suicidality for use in clinical trials or clinical care, but further research with larger samples, other patient populations, and longer follow-up periods is needed.
Journal ArticleDOI
Placebo-related effects in clinical trials in schizophrenia: what is driving this phenomenon and what can be done to minimize it?
TL;DR: Key issues related to the effect of placebo observed in schizophrenia clinical trials are clarified and potential solutions to them are identified.
Journal ArticleDOI
The Computerized Adaptive Diagnostic Test for Major Depressive Disorder (CAD-MDD): A Screening Tool for Depression
Robert D. Gibbons,Giles Hooker,Matthew Finkelman,David J. Weiss,Paul A. Pilkonis,Ellen Frank,Tara L. Moore,David J. Kupfer +7 more
TL;DR: High sensitivity and reasonable specificity for a clinician-based DSM-IV diagnosis of depression can be obtained using an average of 4 adaptively administered self-report items in less than 1 minute, relative to the currently used PHQ-9.
Journal ArticleDOI
Inter-rater agreement in evaluation of disability: systematic review of reproducibility studies
Jürgen Barth,Jürgen Barth,Wout de Boer,Jason W. Busse,Jan L. Hoving,Jan L. Hoving,Sarah Kedzia,Rachel Couban,Katrin Fischer,David Yoh von Allmen,Jerry Spanjer,Regina Kunz +11 more
TL;DR: Research on the reliability of medical evaluations of disability for work is limited and indicates high variation in judgments among assessing professionals, which suggests standardising the evaluation process could improve reliability.
References
More filters
Journal ArticleDOI
A rating scale for depression
TL;DR: The present scale has been devised for use only on patients already diagnosed as suffering from affective disorder of depressive type, used for quantifying the results of an interview, and its value depends entirely on the skill of the interviewer in eliciting the necessary information.
Journal ArticleDOI
A structured interview guide for the Hamilton Depression Rating Scale.
TL;DR: A test-retest reliability study conducted on a series of psychiatric inpatients demonstrated that the use of the SIGH-D results in a substantially improved level of agreement for most of the HDRS items.
Journal ArticleDOI
Standardizing the Hamilton Depression Rating Scale: past, present, and future.
TL;DR: The Hamilton Depression Rating Scale (HAM-D) has become the most widely used depression severity rating scale in the world, and a fitting tribute to Per Bech, who has contributed so much to the assessment of depression severity.
Journal ArticleDOI
The GRID-HAMD : standardization of the Hamilton Depression Rating Scale
Janet B. W. Williams,Kenneth A. Kobak,Per Bech,Nina Engelhardt,Kenneth R. Evans,Joshua D. Lipsitz,Jason T. Olin,Jay Pearson,Amir H Kalali +8 more
TL;DR: The data suggest that the GRID-HAMD is an improvement over the original Guy version as well as the SIGH-D in its incorporation of innovative features and preservation of high reliability and validity.
Journal ArticleDOI
Comparison of the standard and structured interview guide for the Hamilton Depression Rating Scale in depressed geriatric inpatients.
Paul J. Moberg,Lawrence W Lazarus,Raquelle I. Mesholam,Warren B. Bilker,Ismael Lee Chuy,Ida Neyman,Virginia Markvart +6 more
TL;DR: The interrater reliability of the standard Hamilton Depression Rating Scale (Ham-D) and a structured interview guide for the Ham-D (the SIGH-D were compared in a sample of 20 elderly inpatients with major depression.