Many psychologists do not realize that exploratory use of the popular multiway analysis of variance harbors a multiple-comparison problem. In the case of two factors, three separate null hypotheses are subject to test (i.e., two main effects and one interaction). Consequently, the probability of at least one Type I error (if all null hypotheses are true) is 14 % rather than 5 %, if the three tests are independent. We explain the multiple-comparison problem and demonstrate that researchers almost never correct for it. To mitigate the problem, we describe four remedies: the omnibus F test, control of the familywise error rate, control of the false discovery rate, and preregistration of the hypotheses.

/pdf/hidden-multiplicity-in-exploratory-multiway-anova-prevalence-5232rl2agt.pdf

Hidden multiplicity in exploratory multiway ANOVA: Prevalence and remedies.

Learning about hypothesis evaluation using the Bayes factor could enhance psychological research. In contrast to null-hypothesis significance testing it renders the evidence in favor of each of the hypotheses under consideration (it can be used to quantify support for the null-hypothesis) instead of a dichotomous reject/do-not-reject decision; it can straightforwardly be used for the evaluation of multiple hypotheses without having to bother about the proper manner to account for multiple testing; and it allows continuous reevaluation of hypotheses after additional data have been collected (Bayesian updating). This tutorial addresses researchers considering to evaluate their hypotheses by means of the Bayes factor. The focus is completely applied and each topic discussed is illustrated using Bayes factors for the evaluation of hypotheses in the context of an ANOVA model, obtained using the R package bain. Readers can execute all the analyses presented while reading this tutorial if they download bain and the R-codes used. It will be elaborated in a completely nontechnical manner: what the Bayes factor is, how it can be obtained, how Bayes factors should be interpreted, and what can be done with Bayes factors. After reading this tutorial and executing the associated code, researchers will be able to use their own data for the evaluation of hypotheses by means of the Bayes factor, not only in the context of ANOVA models, but also in the context of other statistical models. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

/pdf/a-tutorial-on-testing-hypotheses-using-the-bayes-factor-4tng593674.pdf

A tutorial on testing hypotheses using the Bayes factor.

http://www.jiac-j.com/article/S1341321X22001192/pdf

Epidemiology of SARS-CoV-2 infection in nursing facilities and the impact of their clusters in a Japanese core city

Background Few reports have evaluated the effect of the SARS-CoV-2 variant and vaccination on the clinical and imaging features of COVID-19. Purpose To evaluate and compare the effect of vaccination and variant prevalence on the clinical and imaging features of infections by the SARS-CoV-2. Materials and Methods Consecutive adults hospitalized for confirmed COVID-19 at three centers (two academic medical centers and one community hospital) and registered in a nationwide open data repository for COVID-19 between August 2021 and March 2022 were retrospectively included. All patients had available chest radiographs or CT images. Patients were divided into two groups according to predominant variant type over the study period. Differences between clinical and imaging features were analyzed with use of the Pearson χ2 test, Fisher exact test, or the independent t test. Multivariable logistic regression analyses were used to evaluate the effect of variant predominance and vaccination status on imaging features of pneumonia and clinical severity. Results Of the 2180 patients (mean age, 57 years ± 21; 1171 women), 1022 patients (47%) were treated during the Delta variant predominant period and 1158 (53%) during the Omicron period. The Omicron variant prevalence was associated with lower pneumonia severity based on CT scores (odds ratio [OR], 0.71 [95% CI: 0.51, 0.99; P = .04]) and lower clinical severity based on intensive care unit (ICU) admission or in-hospital death (OR, 0.43 [95% CI: 0.24, 0.77; P = .004]) than the Delta variant prevalence. Vaccination was associated with the lowest odds of severe pneumonia based on CT scores (OR, 0.05 [95% CI: 0.03, 0.13; P < .001]) and clinical severity based on ICU admission or in-hospital death (OR, 0.15 [95% CI: 0.07, 0.31; P < .001]) relative to no vaccination. Conclusion The SARS-CoV-2 Omicron variant prevalence and vaccination were associated with better clinical outcomes and lower severe pneumonia risk relative to Delta variant prevalence. © RSNA, 2022 Supplemental material is available for this article. See also the editorial by Little in this issue.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9527969

SARS-CoV-2 Variants Infection in Relationship to Imaging-based Pneumonia and Clinical Outcomes

Many empirical researchers do not realize that the common multiway analysis of variance (ANOVA) harbors a multiple comparison problem. In the case of two factors, three separate null hypotheses are subject to test (i.e., two main effects and one interaction). Consequently, the probability of at least one Type I error (if all null hypotheses are true) is 14% rather than 5% if the three tests are independent. We explain the multiple comparison problem and demonstrate that researchers almost never correct for it. We describe one of several correction procedures (i.e., sequential Bonferroni), and show that its application alters at least one of the substantive conclusions in 45 out of 60 articles considered. An additional method to mitigate the multiplicity in multiway ANOVA is preregistration of hypotheses.

Hidden Multiplicity in Multiway ANOVA: Prevalence, Consequences, and Remedies

The objective of this study was to investigate the incidence and risk factors associated with COVID-19 vaccine breakthrough infections. We included all persons ≥18 years that had been fully vaccinated against COVID-19 for ≥14 days, between 1 February 2021 and 5 December 2021, in Belgium. The incidence of breakthrough infections (laboratory confirmed SARS-CoV-2-infections) was determined. Factors associated with breakthrough infections were analyzed using COX proportional hazard models. Among 8,062,600 fully vaccinated adults, we identified 373,070 breakthrough infections with an incidence of 11.2 (95%CI 11.2–11.3)/100 person years. Vaccination with Ad26.COV2.S (HR1.54, 95%CI 1.52–1.56) or ChAdOx1 (HR1.68, 95%CI 1.66–1.69) was associated with a higher risk of a breakthrough infection compared to BNT162b2, while mRNA-1273 was associated with a lower risk (HR0.68, 95%CI 0.67–0.69). A prior COVID-19-infection was protective against a breakthrough infection (HR0.23, 95%CI 0.23–0.24), as was an mRNA booster (HR0.44, 95%CI 0.43–0.45). During a breakthrough infection, those who had a prior COVID-19 infection were less likely to have COVID-19 symptoms of almost all types than naïve persons. We identified risk factors associated with breakthrough infections, such as vaccination with adenoviral-vector vaccines, which could help inform future decisions on booster vaccination strategies. A prior COVID-19 infection lowered the risk of breakthrough infections and of having symptoms, highlighting the protective effect of hybrid immunity.

/pdf/incidence-and-risk-factors-of-covid-19-vaccine-breakthrough-3pv94e09.pdf

Incidence and Risk Factors of COVID-19 Vaccine Breakthrough Infections: A Prospective Cohort Study in Belgium

In Belgium, the first COVID-19 death was reported on 10 March 2020. Nursing home (NH) residents are particularly vulnerable for COVID-19, making it essential to follow-up the spread of COVID-19 in this setting. This manuscript describes the methodology of surveillance and epidemiology of COVID-19 cases, hospitalizations and deaths in Belgian NHs.A COVID-19 surveillance in all Belgian NHs (n = 1542) was set up by the regional health authorities and Sciensano. Aggregated data on possible/confirmed COVID-19 cases and hospitalizations and case-based data on deaths were reported by NHs at least once a week. The study period covered April-December 2020. Weekly incidence/prevalence data were calculated per 1000 residents or staff members.This surveillance has been launched within 14 days after the first COVID-19 death in Belgium. Automatic data cleaning was installed using different validation rules. More than 99% of NHs participated at least once, with a median weekly participation rate of 95%. The cumulative incidence of possible/confirmed COVID-19 cases among residents was 206/1000 in the first wave and 367/1000 in the second wave. Most NHs (82%) reported cases in both waves and 74% registered ≥10 possible/confirmed cases among residents at one point in time. In 51% of NHs, at least 10% of staff was absent due to COVID-19 at one point. Between 11 March 2020 and 3 January 2021, 11,329 COVID-19 deaths among NH residents were reported, comprising 57% of all COVID-19 deaths in Belgium in that period.This surveillance was crucial in mapping COVID-19 in this vulnerable setting and guiding public health interventions, despite limitations of aggregated data and necessary changes in protocol over time. Belgian NHs were severely hit by COVID-19 with many fatal cases. The measure of not allowing visitors, implemented in the beginning of the pandemic, could not avoid the spread of SARS-CoV-2 in the NHs during the first wave. The virus was probably often introduced by staff. Once the virus was introduced, it was difficult to prevent healthcare-associated outbreaks. Although, in contrast to the first wave, personal protective equipment was available in the second wave, again a high number of cases were reported. 

/pdf/covid-19-cases-hospitalizations-and-deaths-in-belgian-2uzymlhx.pdf

COVID-19 cases, hospitalizations and deaths in Belgian nursing homes: results of a surveillance conducted between April and December 2020

Experimental studies are usually designed with specific expectations about the results in mind. However, most researchers apply some form of omnibus test to test for any differences, with follow up tests like pairwise comparisons or simple effects analyses for further investigation of the effects. The power to find full support for the theory with such an exploratory approach which is usually based on multiple testing is, however, rather disappointing. With the simulations in this paper we showed that many of the common choices in hypothesis testing led to a severely underpowered form of theory evaluation. Furthermore, some less commonly used approaches were presented and a comparison of results in terms of power to find support for the theory was made. We concluded that confirmatory methods are required in the context of theory evaluation and that the scientific literature would benefit from a clearer distinction between confirmatory and exploratory findings. Also, we emphasis the importance of reporting all tests, significant or not, including the appropriate sample statistics like means and standard deviations. Another recommendation is related to the fact that researchers, when they discuss the conclusions of their own study, seem to underestimate the role of sampling variability. The execution of more replication studies in combination with proper reporting of all results provides insight in between study variability and the amount of chance findings.

/pdf/confirmatory-methods-or-huge-samples-are-required-to-obtain-3iadu8lcqd.pdf

Confirmatory Methods, or Huge Samples, Are Required to Obtain Power for the Evaluation of Theories

Introduction The pathogenesis of COVID-19 depends on the interplay between host characteristics, viral characteristics and contextual factors. Here, we compare COVID-19 disease severity between hospitalized patients in Belgium infected with the SARS-CoV-2 variant B.1.1.7 and those infected with previously circulating strains. Methods The study is conducted within a causal framework to study the severity of SARS-CoV-2 variants by merging surveillance registries in Belgium. Infection with SARS-CoV-2 B.1.1.7 (‘exposed’) was compared to infection with previously circulating strains (‘unexposed’) in terms of the manifestation of severe COVID-19, intensive care unit (ICU) admission, or in-hospital mortality. The exposed and unexposed group were matched based on the hospital and the mean ICU occupancy rate during the patient’s hospital stay. Other variables identified as confounders in a Directed Acyclic Graph (DAG) were adjusted for using regression analysis. Sensitivity analyses were performed to assess the influence of selection bias, vaccination rollout, and unmeasured confounding. Results We observed no difference between the exposed and unexposed group in severe COVID-19 disease or in-hospital mortality (RR = 1.15, 95% CI [0.93–1.38] and RR = 0.92, 95% CI [0.62–1.23], respectively). The estimated standardized risk to be admitted in ICU was significantly higher (RR = 1.36, 95% CI [1.03–1.68]) when infected with the B.1.1.7 variant. An age-stratified analysis showed that among the younger age group (≤65 years), the SARS-CoV-2 variant B.1.1.7 was significantly associated with both severe COVID-19 progression and ICU admission. Conclusion This matched observational cohort study did not find an overall increased risk of severe COVID-19 or death associated with B.1.1.7 infection among patients already hospitalized. There was a significant increased risk to be transferred to ICU when infected with the B.1.1.7 variant, especially among the younger age group. However, potential selection biases advocate for more systematic sequencing of samples from hospitalized COVID-19 patients.

Freek Haarhuis

Papers

Incidence and Risk Factors of COVID-19 Vaccine Breakthrough Infections: A Prospective Cohort Study in Belgium

COVID-19 cases, hospitalizations and deaths in Belgian nursing homes: results of a surveillance conducted between April and December 2020

Confirmatory Methods, or Huge Samples, Are Required to Obtain Power for the Evaluation of Theories

Severity of infection with the SARS-CoV-2 B.1.1.7 lineage among hospitalized COVID-19 patients in Belgium