scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Sample size and optimal designs for reliability studies

TL;DR: A functional approximation to earlier exact results is shown to have excellent agreement with the exact results and one can use it easily without intensive numerical computation.
Abstract: A method is developed to calculate the required number of subjects k in a reliability study, where reliability is measured using the intraclass correlation rho. The method is based on a functional approximation to earlier exact results. The approximation is shown to have excellent agreement with the exact results and one can use it easily without intensive numerical computation. Optimal design configurations are also discussed; for reliability values of about 40 per cent or higher, use of two or three observations per subject will minimize the total number of observations required.
Citations
More filters
Journal ArticleDOI
TL;DR: In this review, the basics of classic reliability theory are addressed in the context of choosing and interpreting an ICC and how the SEM and its variants can be used to construct confidence intervals for individual scores and to determine the minimal difference needed to be exhibited for one to be confident that a true change in performance of an individual has occurred.
Abstract: Reliability, the consistency of a test or measurement, is frequently quantified in the movement sciences literature. A common metric is the intraclass correlation coefficient (ICC). In addition, the SEM, which can be calculated from the ICC, is also frequently reported in reliability studies. However, there are several versions of the ICC, and confusion exists in the movement sciences regarding which ICC to use. Further, the utility of the SEM is not fully appreciated. In this review, the basics of classic reliability theory are addressed in the context of choosing and interpreting an ICC. The primary distinction between ICC equations is argued to be one concerning the inclusion (equations 2,1 and 2,k) or exclusion (equations 3,1 and 3,k) of systematic error in the denominator of the ICC equation. Inferential tests of mean differences, which are performed in the process of deriving the necessary variance components for the calculation of ICC values, are useful to determine if systematic error is present. If so, the measurement schedule should be modified (removing trials where learning and/or fatigue effects are present) to remove systematic error, and ICC equations that only consider random error may be safely used. The use of ICC values is discussed in the context of estimating the effects of measurement error on sample size, statistical power, and correlation attenuation. Finally, calculation and application of the SEM are discussed. It is shown how the SEM and its variants can be used to construct confidence intervals for individual scores and to determine the minimal difference needed to be exhibited for one to be confident that a true change in performance of an individual has occurred.

3,992 citations


Cites background from "Sample size and optimal designs for..."

  • ...The reader is referred to other studies for further discussion (16, 35, 52)....

    [...]

Journal ArticleDOI
TL;DR: The issue of statistical testing of kappa is considered, including the use of confidence intervals, and appropriate sample sizes for reliability studies using kappa are tabulated.
Abstract: Purpose. This article examines and illustrates the use and interpretation of the kappa statistic in musculoskeletal research. Summary of Key Points. The reliability of clinicians' ratings is an important consideration in areas such as diagnosis and the interpretation of examination findings. Often, these ratings lie on a nominal or an ordinal scale. For such data, the kappa coefficient is an appropriate measure of reliability. Kappa is defined, in both weighted and unweighted forms, and its use is illustrated with examples from musculoskeletal research. Factors that can influence the magnitude of kappa (prevalence, bias, and nonindependent ratings) are discussed, and ways of evaluating the magnitude of an obtained kappa are considered. The issue of statistical testing of kappa is considered, including the use of confidence intervals, and appropriate sample sizes for reliability studies using kappa are tabulated. Conclusions. The article concludes with recommendations for the use and interpretation of kappa.

3,427 citations

Journal ArticleDOI
TL;DR: In this paper, the authors developed guidelines for reporting reliability and agreement studies in interrater and intra-arater reliability and agreements, and proposed 15 issues that should be addressed when reporting such studies.

1,605 citations

Book
07 Jun 2000
TL;DR: In this paper, the authors developed and tested a questionnaire for clinical trials and found that the questionnaire scores and measures were validated, reliability, and sensitivity, respectively, in terms of Validity, Reliability, Sensitivity.
Abstract: Principles of Measurement Scales. DEVELOPING AND TESTING QUESTIONNAIRES. Scores and Measurements: Validity, Reliability, Sensitivity. Multi-item Scales. Factor Analysis. Item Response Theory and Differential Item Functioning. Questionnaire Development and Scoring. ANALYSIS OF QoL DATA. Cross-sectional Analysis. Exploring Longitudinal Data. Modelling Longitudinal Data. Missing Data. Quality-adjusted Survival. PRACTICAL ASPECTS AND CLINICAL INTERPRETATION. Clinical Trials. Sample Sizes. Practical and Reporting Issues. Clinical Interpretation. Appendices. References. Index.

1,542 citations

Book
01 Aug 2011
TL;DR: This chapter discusses the development of a measurement instrument, field testing - item reduction and data structure, and systematic reviews of measurement properties Index.
Abstract: 1. Introduction 2. Concepts, theories and models, and types of measurements 3. The development of a measurement instrument 4. Field testing - item reduction and data structure 5. Reliability 6. Validity 7. Responsiveness 8. Interpretation 9. Systematic reviews of measurement properties Index.

1,262 citations

Trending Questions (1)
What is the optimal sample size for DA?

The optimal sample size for DA is not mentioned in the provided information.