The widespread use of standardized health surveys is predicated on the largely untested assumption that scales constructed from those surveys will satisfy minimum psychometric requirements across diverse population groups. Data from the Medical Outcomes Study (MOS) were used to evaluate data completeness and quality, test scaling assumptions, and estimate internal-consistency reliability for the eight scales constructed from the MOS SF-36 Health Survey. Analyses were conducted among 3,445 patients and were replicated across 24 subgroups differing in sociodemographic characteristics, diagnosis, and disease severity. For each scale, item-completion rates were high across all groups (88% to 95%), but tended to be somewhat lower among the elderly, those with less than a high school education, and those in poverty. On average, surveys were complete enough to compute scales scores for more than 96% of the sample. Across patient groups, all scales passed tests for item-internal consistency (97% passed) and item-discriminant validity (92% passed). Reliability coefficients ranged from a low of 0.65 to a high of 0.94 across scales (median = 0.85) and varied somewhat across patient subgroups. Floor effects were negligible except for the two role disability scales. Noteworthy ceiling effects were observed for both role disability scales and the social functioning scale. These findings support the use of the SF-36 survey across the diverse populations studied and identify population groups in which use of standardized health status measures may or may not be problematic.

https://www.jstor.org/stable/pdfplus/3766189.pdf

The MOS 36-item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups.

List of figures List of tables List of boxes Introduction Part I. Effect Sizes and the Interpretation of Results: 1. Introduction to effect sizes 2. Interpreting effects Part II. The Analysis of Statistical Power: 3. Power analysis and the detection of effects 4. The painful lessons of power research Part III. Meta-Analysis: 5. Drawing conclusions using meta-analysis 6. Minimizing bias in meta-analysis Last word: thirty recommendations for researchers Appendices: 1. Minimum sample sizes 2. Alternative methods for meta-analysis Bibliography Index.

The essential guide to effect sizes : statistical power, meta-analysis, and the interpretation of research results

OBJECTIVE: Cross-sectional studies are often used in psychiatric research as a basis of longitudinal inferences about developmental or disease processes. While the limitations of such usage are often acknowledged, these are often understated. The authors describe how such inferences are often, and sometimes seriously, misleading. METHOD: Why and how these inferences mislead are here demonstrated on an intuitive level, by using simulated data inspired by real problems in psychiatric research. >RESULTS: Four factors with major roles in the relationship between cross-sectional studies and longitudinal inferences are selection of time scale, type of developmental process studied, reliability of measurement, and clarity of terminology. The authors suggest how to recognize inferential errors when they occur, describe how to protect against such errors in future research, and delineate the circumstances in which only longitudinal studies can answer crucial questions. CONCLUSIONS: The simple conclusion is that on...

https://ajp.psychiatryonline.org/doi/pdfplus/10.1176/appi.ajp.157.2.163

How can we learn about developmental processes from cross-sectional studies, or can we?

Statistical power is a topic of importance to any researcher using statistical inference testing. Studies with low levels of statistical power usually result in inconclusive findings, even though the researcher may have expended much time and effort gathering the data for analysis. A survey of the statistical power of articles employing statistical inference testing published in leading MIS journals shows that their statistical power is, on average, substantially below accepted norms. The consequence of this low power is that MIS researchers typically have a 40 percent chance of not detecting the phenomenon under study, even though it, in fact, may exist.Fortunately, there are several techniques, beyond expanding the sample size (which often may be impossible) that researchers can use to improve the power of their studies. Some are as easy as using a different but more powerful statistical test, while others require developing more elaborate sampling plans or a more careful construction of the research design. Attention tot he statistical power of a study is one key ingredient in assuring the success of the study. This article should serve as a useful guide for MIS researcher sin the planning, execution, and interpretation of inferential statistical analyses.

The problem of statistical power in MIS research

Rater bias is a substantial source of error in psychological research Bias distorts observed effect sizes beyond the expected level of attenuation due to intrarater error, and the impact of bias is not accurately estimated using conventional methods of correction for attenuation Using a model based on multivariate generalizability theory, this article illustrates how bias affects research results The model identifies 4 types of bias that may affect findings in research using observer ratings, including the biases traditionally termed leniency and halo errors The impact of bias depends on which of 4 classes of rating design is used, and formulas are derived for correcting observed effect sizes for attenuation (due to bias variance) and inflation (due to bias covariance) in each of these classes The rater bias model suggests procedures for researchers seeking to minimize adverse impact of bias on study findings

https://psycnet.apa.org/journals/met/5/1/64.pdf

J. P. Sutcliffe

Papers

On the relationship of reliability to statistical power.