scispace - formally typeset
Open accessJournal ArticleDOI: 10.6339/JDS.201604_14(2).0002

Comparing Pre-Post Change Across Groups: Guidelines for Choosing between Difference Scores, ANCOVA, and Residual Change Scores

04 Mar 2021-Journal of data science (中華資料採礦協會)-Vol. 14, Iss: 2, pp 205-229
Abstract: Psychological researchers often investigate differences between groups in the amount of change from pre-test to post-test. For example, researchers treating a group of depressed students may wish to compare the amount of improvement from pre-intervention to post-intervention for males and females, or the researchers may randomly assign participants to groups and compare their improvement across two treatments. In the first case, there are likely to be pre-test differences between the groups, whereas in the second case no pre-test group differences are expected. Three of the most popular methods for comparing independent groups with regard to the amount of change are difference scores, ANCOVA, and residualized change scores. Although the choice between these methods is sometimes clear, in most instances this is not the case. In this research, a simulation study was used to determine the effect of many common issues on the bias, Type I error rates, and power of the difference score, ANCOVA, and residual change score. These issues included: sample size, reliability of the measure, floor/ceiling effects, effect size, and baseline group differences. Results from the study are used to provide specific recommendations with regards to applying each of these three methods.

... read more


16 results found

Journal ArticleDOI: 10.1037/APL0000273
Abstract: Culturally savvy organizations recognize that selecting and developing people who can be effective in a global workforce is important in today's business environment. Nevertheless, many companies struggle to identify and develop talent who are happy and successful working and living outside their home country. We examine 1 factor that may foster success in a host country-minority status in 1's home country-as a predictor of change in acculturation over time. Specifically, we draw on the conservation of resources model to suggest that international students who have been a member of more minority groups in their home country have unique experiences working with dissimilar others that offer advantages when acculturating to new cultures and novel situations. Then, change in host country acculturation is explored as a mechanism to explain how minority status in the home country relates to intentions to leave the host country and psychological well-being 6 months after entry. Two moderators (cultural intelligence, perceived diversity climate of the host institution) of these relationships are also examined. Results revealed that the relationship between minority status in the home country and change in host country acculturation was positive and stronger for those with higher cultural intelligence. Further, the relationship between change in host country acculturation and psychological well-being was positive when perceived diversity climate of the host institution was high, but was not significant when perceived diversity climate was low. (PsycINFO Database Record

... read more

Topics: Acculturation (62%), Cultural diversity (56%), Diversity (politics) (52%) ... read more

21 Citations

Journal ArticleDOI: 10.1080/00461520.2021.1939700
Abstract: It is often hypothesized that prior knowledge strongly predicts learning performance. It can affect learning positively mediated through some processes and negatively mediated through others. We ex...

... read more

16 Citations

Journal ArticleDOI: 10.1111/PSYP.13762
21 Jan 2021-Psychophysiology
Abstract: In studies of event-related brain potentials (ERPs), difference scores between conditions in a task are frequently used to isolate neural activity for use as a dependent or independent variable. Adequate score reliability is a prerequisite for studies examining relationships between ERPs and external correlates, but there is no extensive treatment on the suitability of the various available approaches to estimating difference score reliability that focus on ERP research. In the present study, we provide formulas from classical test theory and generalizability theory for estimating the internal consistency of subtraction-based and residualized difference scores. These formulas are then applied to error-related negativity (ERN) and reward positivity (RewP) difference scores from the same sample of 117 participants. Analyses demonstrate that ERN difference scores can be reliable, which supports their use in studies of individual differences. However, RewP difference scores yielded poor reliability due to the high correlation between the constituent reward and non-reward ERPs. Findings emphasize that difference score reliability largely depends on the internal consistency of constituent scores and the correlation between those scores. Furthermore, generalizability theory yields more suitable estimates of internal consistency for subtraction-based difference scores than classical test theory. We conclude that ERP difference scores can show adequate reliability and be useful for isolating neural activity in studies of individual differences.

... read more

11 Citations

Open accessJournal ArticleDOI: 10.3758/S13428-020-01407-2
Qimin Liu1, Lijuan Wang2Institutions (2)
Abstract: Ceiling and floor effects are often observed in social and behavioral science. The current study examines ceiling/floor effects in the context of the t-test and ANOVA, two frequently used statistical methods in experimental studies. Our literature review indicated that most researchers treated ceiling or floor data as if these data were true values, and that some researchers used statistical methods such as discarding ceiling or floor data in conducting the t-test and ANOVA. The current study evaluates the performance of these conventional methods for t-test and ANOVA with ceiling or floor data. Our evaluation also includes censored regression with regard to its capacity for handling ceiling/floor data. Furthermore, we propose an easy-to-use method that handles ceiling or floor data in t-tests and ANOVA by using properties of truncated normal distributions. Simulation studies were conducted to compare the performance of the methods in handling ceiling or floor data for t-test and ANOVA. Overall, the proposed method showed greater accuracy in effect size estimation and better-controlled Type I error rates over other evaluated methods. We developed an easy-to-use software package and web applications to help researchers implement the proposed method. Recommendations and future directions are discussed.

... read more

Topics: Floor effect (66%), Ceiling effect (60%), Ceiling (cloud) (56%)

11 Citations

Journal ArticleDOI: 10.1007/S10826-016-0577-4
Abstract: The study explored changes in young adults’ mental health, attachment, and separation from parents during a seven-week wilderness therapy program. Utilizing a longitudinal one-group design, the study examined outcomes of 157 young adults in one wilderness therapy program. From pre to post treatment, participants reported significant improvement in mental health symptoms and interpersonal relationships, as well as increases in the belief that others can be depended upon. Participants reported less resentment and anger towards mothers from pre to post treatment, but an increase in their needs for approval from fathers. The study details a link between young adults’ attachment, independence from parents, and improvement in mental health, suggesting that treatment which targets these links may provide more effective intervention.

... read more

Topics: Wilderness therapy (63%), Mental health (58%)

10 Citations


26 results found

Open accessJournal Article
01 Jan 2014-MSOR connections
Abstract: Copyright (©) 1999–2012 R Foundation for Statistical Computing. Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the R Core Team.

... read more

Topics: R Programming Language (78%)

229,202 Citations

Journal ArticleDOI: 10.1037/H0029382
Lee J. Cronbach1, Lita FurbyInstitutions (1)
Abstract: Procedures previously recommended by various authors for the estimation of "change" scores, "residual" or "basefree" measures of change, and other kinds of difference scores are examined. A procedure proposed by Lord is extended to obtain more precise estimates, and an alternative to the Tucker-Damarin-Messick procedure is offered. A consideration of the purposes for which change measures have been sought in the past leads to a series of recommended procedures which solve research and personnel-decision problems without estimation of change scores for individuals.

... read more

Topics: Residual (50%)

2,350 Citations

Journal ArticleDOI: 10.2307/271083
Abstract: Change scores have been widely criticized for their purported unreliability and for their sensitivity to regression toward the mean. These objections are shown to be unfounded under a plausible regression model for the nonequivalent control group design. This model leads to inferences that are intuitively correct, as judged by changes in means over time, while the conventional model leads to inferences that are intuitively false. Moreover, the conventional model implies that regression toward the mean within groups leads to regression toward the mean between groups, an implausible result for naturally occurring groups. Nevertheless, the conventional model may be more appropriate when there is a true causal effect of the pretest on the posttest, or when cases are assigned to groups on the basis of their pretest scores.

... read more

Topics: Regression toward the mean (57%), Regression analysis (57%), Standardized coefficient (56%) ... read more

995 Citations

Journal ArticleDOI: 10.1037/H0025105
Frederic M. Lord1Institutions (1)
Abstract: It is common practice in behavioral research, and in other areas, to apply the analysis of covariance in the investigation of preexisting natural groups. The research worker is usually interested in some criterion variable (y) and would like to make allowances for the fact that his groups are not matched on some important independent variable or "control" variable (x). The situation is such that observed differences in the dependent variable might logically be caused by differences in the independent variable, and the research worker wishes to rule out this possibility. It is widely recognized that ideally the research worker should assign cases or individuals at random to the groups that are to be studied by analysis of covariance. In behavioral research and in many other areas, such random assignment is usually difficult or impossible—as, for example, in a com-

... read more

581 Citations

No. of citations received by the Paper in previous years