scispace - formally typeset
Open accessJournal ArticleDOI: 10.1016/J.IJPSYCHO.2021.02.016

Making ERP research more transparent: Guidelines for preregistration

04 Mar 2021-International Journal of Psychophysiology (Elsevier)-Vol. 164, pp 52-63
Abstract: A combination of confirmation bias, hindsight bias, and pressure to publish may prompt the (unconscious) exploration of various methodological options and reporting only the ones that lead to a (statistically) significant outcome. This undisclosed analytic flexibility is particularly relevant in EEG research, where a myriad of preprocessing and analysis pipelines can be used to extract information from complex multidimensional data. One solution to limit confirmation and hindsight bias by disclosing analytic choices is preregistration: researchers write a time-stamped, publicly accessible research plan with hypotheses, data collection plan, and the intended preprocessing and statistical analyses before the start of a research project. In this manuscript, we present an overview of the problems associated with undisclosed analytic flexibility, discuss why and how EEG researchers would benefit from adopting preregistration, provide guidelines and examples on how to preregister data preprocessing and analysis steps in typical ERP studies, and conclude by discussing possibilities and limitations of this open science practice.

... read more

Topics: Hindsight bias (53%)
Citations
  More

7 results found


Open accessJournal ArticleDOI: 10.1016/J.IJPSYCHO.2021.04.004
Abstract: Event-related brain potentials (ERPs) represent direct measures of neural activity that are leveraged to understand cognitive, affective, sensory, and motor processes. Every ERP researcher encounters the obstacle of determining whether measurements are precise or psychometrically reliable enough for an intended purpose. In this primer, we review three types of measurements metrics: data quality, group-level internal consistency, and subject-level internal consistency. Data quality estimates characterize the precision of ERP scores but provide no inherent information about whether scores are precise enough for examining individual differences. Group-level internal consistency characterizes the ratio of between-person differences to the precision of those scores, and provides a single internal consistency estimate for an entire group of participants that risks masking low internal consistency for some individuals. Subject-level internal consistency considers the precision of an ERP score for a person relative to between-person differences for a group, and an estimate is yielded for each individual. We apply each metric to published error-related negativity (ERN) and reward positivity (RewP) data and demonstrate how failing to consider data quality and internal consistency can undermine statistical inferences. We conclude with general comments on how these estimates may be used to improve measurement quality and methodological transparency. Subject-level internal consistency computation is implemented within the ERP Reliability Analysis (ERA) Toolbox.

... read more

12 Citations


Open accessJournal ArticleDOI: 10.1177/03616843211037613
Sofia Persson1, Madeleine Pownall2Institutions (2)
Abstract: Feminist scholars have long been concerned with claims of hardwired brain sex differences emanating from neuroscience and evolutionary psychology. Past criticisms of these claims have rightfully qu...

... read more

Topics: Evolutionary psychology (52%)

4 Citations


Open accessJournal ArticleDOI: 10.1098/RSOS.200805
Abstract: Current attempts at methodological reform in sciences come in response to an overall lack of rigor in methodological and scientific practices in experimental sciences. However, most methodological reform attempts suffer from similar mistakes and over-generalizations to the ones they aim to address. We argue that this can be attributed in part to lack of formalism and first principles. Considering the costs of allowing false claims to become canonized, we argue for formal statistical rigor and scientific nuance in methodological reform. To attain this rigor and nuance, we propose a five-step formal approach for solving methodological problems. To illustrate the use and benefits of such formalism, we present a formal statistical analysis of three popular claims in the metascientific literature: (i) that reproducibility is the cornerstone of science; (ii) that data must not be used twice in any analysis; and (iii) that exploratory projects imply poor statistical practice. We show how our formal approach can inform and shape debates about such methodological claims.

... read more

2 Citations


Open accessPosted ContentDOI: 10.1098/RSOS.210155
Abstract: In recent years, open science practices have become increasingly popular in psychology and related sciences. These practices aim to increase rigour and transparency in science as a potential respon...

... read more

Topics: Transparency (behavior) (70%), Rigour (51%)

2 Citations


Open accessJournal ArticleDOI: 10.1016/J.IJPSYCHO.2021.06.002
Abstract: Previous studies have shown that electrophysiological measures of error processing are affected in patients at risk or diagnosed with internalizing disorders, hence, suggesting that error processing could be a suitable biomarker for internalizing disorders. In this narrative review, we will evaluate studies that address the role of event-related potential (ERP) measures of error-processing in externalizing disorders and discuss to what extend these can be considered a biomarker for externalizing disorders. Currently, there is evidence for the notion that electrophysiological indices of error processing such as the error-related negativity (ERN) and error positivity (Pe) are reduced in individuals with substance use disorders, attention-deficit/hyperactivity disorder, and in forensic populations. However, it remains unclear whether this is also the case for other understudied disorders such as behavioral addiction. Furthermore, to fully understand how these deficits affect day to day behavior, we encourage research to focus on testing current theories and hypotheses of ERN and Pe. In addition, we argue that within an externalizing disorder, individual differences in error processing deficits may be related to prognosis and gender of the patient, methodological issues and presence of comorbidity. Next, we review studies that have related treatment trajectories with ERP measures of error processing, and we discuss the prospect of improving error processing as a treatment option. We conclude that ERP measures of error processing are candidate biomarkers for externalizing disorders, albeit we strongly urge researchers to continue looking into the predictive value of these measures in the etiology and treatment outcome through multi-method and longitudinal designs.

... read more

Topics: Error-related negativity (52%)

1 Citations


References
  More

183 results found


Journal ArticleDOI: 10.1037/0033-2909.86.3.638
Robert Rosenthal1Institutions (1)
Abstract: For any given research area, one cannot tell how many studies have been conducted but never reported. The extreme view of the "file drawer problem" is that journals are filled with the 5% of the studies that show Type I errors, while the file drawers are filled with the 95% of the studies that show nonsignificant results. Quantitative procedures for computing the tolerance for filed and future null results are reported and illustrated, and the implications are discussed. (15 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)

... read more

Topics: Poison control (51%)

6,443 Citations


Open accessJournal ArticleDOI: 10.1155/2011/156869
Abstract: This paper describes FieldTrip, an open source software package that we developed for the analysis of MEG, EEG, and other electrophysiological data. The software is implemented as a MATLAB toolbox and includes a complete set of consistent and user-friendly high-level functions that allow experimental neuroscientists to analyze experimental data. It includes algorithms for simple and advanced analysis, such as time-frequency analysis using multitapers, source reconstruction using dipoles, distributed sources and beamformers, connectivity analysis, and nonparametric statistical permutation tests at the channel and source level. The implementation as toolbox allows the user to perform elaborate and structured analyses of large data sets using the MATLAB command line and batch scripting. Furthermore, users and developers can easily extend the functionality and implement new algorithms. The modular design facilitates the reuse in other software packages.

... read more

Topics: Software (55%), EEGLAB (53%)

6,162 Citations


Journal ArticleDOI: 10.1037/0033-2909.108.3.480
Ziva Kunda1Institutions (1)
Abstract: It is proposed that motivation may affect reasoning through reliance on a biased set of cognitive processes—that is, strategies for accessing, constructing, and evaluating beliefs. The motivation to be accurate enhances use of those beliefs and strategies that are considered most appropriate, whereas the motivation to arrive at particular conclusions enhances use of those that are considered most likely to yield the desired conclusion. There is considerable evidence that people are more likely to arrive at conclusions that they want to arrive at, but their ability to do so is constrained by their ability to construct seemingly reasonable justifications for these conclusions. These ideas can account for a wide variety of research concerned with motivated reasoning. The notion that goals or motives affect reasoning has a long and controversial history in social psychology. The propositions that motives may affect perceptions (Erdelyi, 1974), attitudes (Festinger, 1957), and attributions (Heider, 1958) have been put forth by some psychologists and challenged by others. Although early researchers and theorists took it for granted that motivation may cause people to make self-serving attributions and permit them to believe what they want to believe because they want to believe it, this view, and the research used to uphold it, came under concentrated criticism in the 1970s. The major and most damaging criticism of the motivational view was that all research purported to demonstrate motivated reasoning could be reinterpreted in entirely cognitive, nonmotivational terms (Miller & Ross, 1975; Nisbett & Ross, 1980). Thus people could draw self-serving conclusions not because they wanted to but because these conclusions seemed more plausible, given their prior beliefs and expectancies. Because both cognitive and motivational accounts could be generated for any empirical study, some theorists argued that the hot versus cold cognition controversy could not be solved, at least in the attribution paradigm (Ross & Fletcher, 1985; Tetlock & Levi, 1982). One reason for the persistence of this controversy lies in the failure of researchers to explore the mechanisms underlying motivated reasoning. Recently, several authors have attempted to rectify this neglect (Kruglanski & Freund, 1983; Kunda, 1987; Pyszczynski & Greenberg, 1987; Sorrentino & Higgins, 1986). All these authors share a view of motivation as having its effects through cognitive processes: People rely on cognitive processes and representations to arrive at their desired conclusions, but motivation plays a role in determining which of these will be used on a given occasion.

... read more

Topics: Motivated reasoning (65%), Motivated tactician (61%), Attitude polarization (52%) ... read more

5,890 Citations


Open accessJournal ArticleDOI: 10.1109/72.761722
Aapo Hyvärinen1Institutions (1)
Abstract: Independent component analysis (ICA) is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible. We use a combination of two different approaches for linear ICA: Comon's information theoretic approach and the projection pursuit approach. Using maximum entropy approximations of differential entropy, we introduce a family of new contrast functions for ICA. These contrast functions enable both the estimation of the whole decomposition by minimizing mutual information, and estimation of individual independent components as projection pursuit directions. The statistical properties of the estimators based on such contrast functions are analyzed under the assumption of the linear mixture model, and it is shown how to choose contrast functions that are robust and/or of minimum variance. Finally, we introduce simple fixed-point algorithms for practical optimization of the contrast functions.

... read more

Topics: Projection pursuit (62%), Independent component analysis (60%), Multidimensional systems (56%) ... read more

5,716 Citations


Journal ArticleDOI: 10.1016/J.JNEUMETH.2007.03.024
Eric Maris1, Robert Oostenveld1Institutions (1)
Abstract: In this paper, we show how ElectroEncephaloGraphic (EEG) and MagnetoEncephaloGraphic (MEG) data can be analyzed statistically using nonparametric techniques Nonparametric statistical tests offer complete freedom to the user with respect to the test statistic by means of which the experimental conditions are compared This freedom provides a straightforward way to solve the multiple comparisons problem (MCP) and it allows to incorporate biophysically motivated constraints in the test statistic, which may drastically increase the sensitivity of the statistical test The paper is written for two audiences: (1) empirical neuroscientists looking for the most appropriate data analysis method, and (2) methodologists interested in the theoretical concepts behind nonparametric statistical tests For the empirical neuroscientist, a large part of the paper is written in a tutorial-like fashion, enabling neuroscientists to construct their own statistical test, maximizing the sensitivity to the expected effect And for the methodologist, it is explained why the nonparametric test is formally correct This means that we formulate a null hypothesis (identical probability distribution in the different experimental conditions) and show that the nonparametric test controls the false alarm rate under this null hypothesis

... read more

Topics: Test statistic (63%), p-value (61%), Pearson's chi-squared test (61%) ... read more

5,084 Citations


Performance
Metrics
No. of citations received by the Paper in previous years
YearCitations
20217