Showing papers by "Keith F. Widaman published in 2023"

Toward a statistical validation of brain signatures as robust measures of behavioral substrates

Abstract: The “brain signature of cognition” concept has garnered interest as a data‐driven, exploratory approach to better understand key brain regions involved in specific cognitive functions, with the potential to maximally characterize brain substrates of behavioral outcomes. Previously we presented a method for computing signatures of episodic memory. However, to be a robust brain measure, the signature approach requires a rigorous validation of model performance across a variety of cohorts. Here we report validation results and provide an example of extending it to a second behavioral domain. In each of two discovery data cohorts, we derived regional brain gray matter thickness associations for two domains: neuropsychological and everyday cognition memory. We computed regional association to outcome in 40 randomly selected discovery subsets of size 400 in each cohort. We generated spatial overlap frequency maps and defined high‐frequency regions as “consensus” signature masks. Using separate validation datasets, we evaluated replicability of cohort‐based consensus model fits and explanatory power by comparing signature model fits with each other and with competing theory‐based models. Spatial replications produced convergent consensus signature regions. Consensus signature model fits were highly correlated in 50 random subsets of each validation cohort, indicating high replicability. In comparisons over each full cohort, signature models outperformed other models. In this validation study, we produced signature models that replicated model fits to outcome and outperformed other commonly used measures. Signatures in two memory domains suggested strongly shared brain substrates. Robust brain signatures may therefore be achievable, yielding reliable and useful measures for modeling substrates of behavioral domains.

Mid‐ to late‐life physical and recreational activities: Associations with late‐life cognition

Abstract: Physical and cognitively stimulating recreational activities are lifestyle factors that may modify risk of late‐life cognitive decline. We sought to examine the role of retrospective ratings of mid‐life (age 40) physical and recreational activity engagement – and change in these activities from age 40 to study entry – in predicting late‐life cognition. We hypothesized that higher self‐reported levels of physical and recreational activities at age 40, and less decline in these activities over time, would be associated with higher levels of late‐life cognition and less rapid cognitive decline.

A Note on Statistical Hypothesis Testing: Probabilifying Modus Tollens Invalidates Its Force? Not True!

Abstract: The import or force of the result of a statistical test has long been portrayed as consistent with deductive reasoning. The simplest form of deductive argument has a first premise with conditional form, such as p→ q, which means that “if p is true, then q must be true.” Given the first premise, one can either affirm or deny the antecedent clause ( p) or affirm or deny the consequent claim ( q). This leads to four forms of deductive argument, two of which are valid forms of reasoning and two of which are invalid. The typical conclusion is that only a single form of argument—denying the consequent, also known as modus tollens—is a reasonable analog of decisions based on statistical hypothesis testing. Now, statistical evidence is never certain, but is associated with a probability (i.e., a p-level). Some have argued that modus tollens, when probabilified, loses its force and leads to ridiculous, nonsensical conclusions. Their argument is based on specious problem setup. This note is intended to correct this error and restore the position of modus tollens as a valid form of deductive inference in statistical matters, even when it is probabilified.

Computerized adaptive test strategies for the matrix reasoning subtest of the Wechsler Adult Intelligence Scale, 4th Edition (WAIS-IV).

Abstract: OBJECTIVE Most neuropsychological tests were developed without the benefit of modern psychometric theory. We used item response theory (IRT) methods to determine whether a widely used test - the 26-item Matrix Reasoning subtest of the WAIS-IV - might be used more efficiently if it were administered using computerized adaptive testing (CAT). METHOD Data on the Matrix Reasoning subtest from 2197 participants enrolled in the National Neuropsychology Network (NNN) were analyzed using a two-parameter logistic (2PL) IRT model. Simulated CAT results were generated to examine optimal short forms using fixed-length CATs of 3, 6, and 12 items and scores were compared to the original full subtest score. CAT models further explored how many items were needed to achieve a selected precision of measurement (standard error ≤ .40). RESULTS The fixed-length CATs of 3, 6, and 12 items correlated well with full-length test results (with r = .90, .97 and .99, respectively). To achieve a standard error of .40 (approximate reliability = .84) only 3-7 items had to be administered for a large percentage of individuals. CONCLUSIONS This proof-of-concept investigation suggests that the widely used Matrix Reasoning subtest of the WAIS-IV might be shortened by more than 70% in most examinees while maintaining acceptable measurement precision. If similar savings could be realized in other tests, the accessibility of neuropsychological assessment might be markedly enhanced, and more efficient time use could lead to broader subdomain assessment.

Effects of early-life environment and adulthood SES on cognitive change in a multiethnic cohort.

Abstract: OBJECTIVES Early-life socioeconomic status (SES) and adversity are associated with late-life cognition and risk of dementia. We examined the association between early-life SES and adversity and late-life cross-sectional cognitive outcomes as well as global cognitive decline, hypothesizing that adulthood SES would mediate these associations. METHODS Our sample (N = 837) was a racially and ethnically diverse cohort of non-Hispanic/Latino White (48%), Black (27%), and Hispanic/Latino (19%) participants from Northern California. Participant addresses were geocoded to the level of the census tract, and US Census Tract 2010 variables (e.g., percent with high school diploma) were extracted and combined to create a neighborhood SES composite. We used multilevel latent variable models to estimate early-life (e.g., parental education, whether participant ever went hungry) and adult (participant's education, main occupation) SES factors and their associations with cross-sectional and longitudinal cognitive outcomes of episodic memory, semantic memory, executive function, and spatial ability. RESULTS Child and adult factors were strongly related to domain-specific cognitive intercepts (0.20-0.48 SD per SD of SES factor); in contrast, SES factors were not related to global cognitive change (0.001-0.01 SD per year per SD of SES factor). Adulthood SES mediated a large percentage (68-75%) of the total early-life effect on cognition. CONCLUSIONS Early-life sociocontextual factors are more strongly associated with cross-sectional late-life cognitive performance compared to cognitive change; this effect is largely mediated through associations with adulthood SES.