scispace - formally typeset
Open accessPosted ContentDOI: 10.31234/OSF.IO/SEYQ7

Making the black box transparent: A template and tutorial for (pre-)registration of studies using Experience Sampling Methods (ESM)

02 Mar 2021-Vol. 4, Iss: 1, pp 251524592092468
Abstract: A growing interest in understanding complex and dynamic psychological processes as they occur in everyday life has led to an increase in studies using ambulatory assessment techniques, including th

... read more


11 results found

Open accessJournal ArticleDOI: 10.1016/J.JPSYCHORES.2020.110211
Abstract: Objective One of the promises of the experience sampling methodology (ESM) is that a statistical analysis of an individual's emotions, cognitions and behaviors in everyday-life could be used to identify relevant treatment targets. A requisite for clinical implementation is that outcomes of such person-specific time-series analyses are not wholly contingent on the researcher performing them. Methods To evaluate this, we crowdsourced the analysis of one individual patient's ESM data to 12 prominent research teams, asking them what symptom(s) they would advise the treating clinician to target in subsequent treatment. Results Variation was evident at different stages of the analysis, from preprocessing steps (e.g., variable selection, clustering, handling of missing data) to the type of statistics and rationale for selecting targets. Most teams did include a type of vector autoregressive model, examining relations between symptoms over time. Although most teams were confident their selected targets would provide useful information to the clinician, not one recommendation was similar: both the number (0–16) and nature of selected targets varied widely. Conclusion This study makes transparent that the selection of treatment targets based on personalized models using ESM data is currently highly conditional on subjective analytical choices and highlights key conceptual and methodological issues that need to be addressed in moving towards clinical implementation.

... read more

33 Citations

Journal ArticleDOI: 10.1016/J.JPSYCHORES.2020.110191
Sigert Ariens1, Eva Ceulemans1, Janne Adolf1Institutions (1)
Abstract: Time series analysis of intensive longitudinal data provides the psychological literature with a powerful tool for assessing how psychological processes evolve through time. Recent applications in the field of psychosomatic research have provided insights into the dynamical nature of the relationship between somatic symptoms, physiological measures, and emotional states. These promising results highlight the intrinsic value of employing time series analysis, although application comes with some important challenges. This paper aims to present an approachable, non-technical overview of the state of the art on these challenges and the solutions that have been proposed, with emphasis on application towards psychosomatic hypotheses. Specifically, we elaborate on issues related to measurement intervals, the number and nature of the variables used in the analysis, modeling stable and changing processes, concurrent relationships, and extending time series analysis to incorporate the data of multiple individuals. We also briefly discuss some general modeling issues, such as lag-specification, sample size and time series length, and the role of measurement errors. We hope to arm applied researchers with an overview from which to select appropriate techniques from the ever growing variety of time series analysis approaches.

... read more

10 Citations

Open accessJournal ArticleDOI: 10.1037/MET0000294
Abstract: With the advent of online and app-based studies, researchers in psychology are making increasing use of repeated subjective reports. The new methods open up opportunities to study behavior in the field and to map causal processes, but they also pose new challenges. Recent work has added initial elevation bias to the list of common pitfalls; here, higher negative states (i.e., thoughts and feelings) are reported on the first day of assessment than on later days. This article showcases a new approach to addressing this and other measurement reactivity biases. Specifically, we employed a planned missingness design in a daily diary study of more than 1,300 individuals who were assessed over a period of up to 70 days to estimate and adjust for measurement reactivity biases. We found that day of first item presentation, item order, and item number were associated with only negligible bias: Items were not answered differently depending on when and where they were shown. Initial elevation bias may thus be more limited than has previously been reported or it may act only at the level of the survey, not at the item level. We encourage researchers to make design choices that will allow them to routinely assess measurement reactivity biases in their studies. Specifically, we advocate the routine randomization of item display and order, as well as of the timing and frequency of measurement. Randomized planned missingness makes it possible to empirically gauge how fatigue, familiarity, and learning interact to bias responses.

... read more

6 Citations

Open accessJournal ArticleDOI: 10.1111/JORA.12657
Julie Janssens1, Robin Achterhof1, Ginette Lafit1, Eva Bamps1  +6 moreInstitutions (1)
Abstract: COVID-19 lockdown measures have profoundly impacted adolescent' daily life, with research suggesting an increase in irritability, stress, loneliness, and family conflict. A potential protective factor is parent-child relationship quality; however, no studies have investigated this. We used data from SIGMA, a longitudinal, experience sampling cohort study, in which N = 173 adolescents aged 11 to 20 were tested before and during COVID-19. Multilevel analyses showed decreased daily-life irritability and increased loneliness from pre- to mid-pandemic. Daily-life stress levels were unchanged. Relationship quality was negatively associated with irritability and loneliness and buffered against the increase in loneliness. Effect sizes were small and do not support a strong effect of the first lockdown on irritability, stress, loneliness, and family conflict in adolescents.

... read more

Topics: Loneliness (62%), Irritability (59%)

2 Citations

Open accessJournal ArticleDOI: 10.3389/FPSYT.2021.697127
Abstract: Adolescence is a vulnerable period for psychopathology development, and certain parenting styles are consistent and robust predictors of a broad range of mental health outcomes. The mechanisms through which maladaptive parenting styles affect the development of psychopathology are assumed to be largely social in nature. Yet, the social mechanisms linking parenting to psychopathology are unexplored at arguably the most important level of functioning: daily life. This study aims to identify the associations between three parenting styles, and the experience of daily-life social interactions. Furthermore, we aim to explore the extent to which these parenting styles and altered daily-life social experiences are associated with psychopathology. In this study, we recruited a sample of N = 1,913 adolescents (63.3% girls; mean age = 13.7, age range = 11 to 20) as part of the first wave of the longitudinal cohort study "SIGMA". Parenting styles (psychological control, responsiveness, and autonomy support) and psychopathology symptoms were assessed using a retrospective questionnaire battery. The experienced quality of social interactions in different types of company was assessed using the experience sampling method, ten times per day for 6 days. Direct associations between parenting styles and general quality of daily-life social experiences were tested using a three-level linear model, revealing significant associations between social experiences and different parenting styles. When interaction effects were added to this model, we found that maternal responsiveness and paternal psychological control mainly related to altered qualities of social interactions with parents, while paternal autonomy support was associated with better experiences of non-family social interactions. Finally, an exploratory path analysis highlighted how both paternal autonomy support and altered quality of non-family interactions are uniquely associated with psychopathology levels. These findings demonstrate the general and pervasive effects of maladaptive parenting styles, as parenting seems to broadly affect adolescents' interactions with different types of social partners in everyday life. Moreover, they illustrate a potential mediated relationship in which altered daily-life social interactions could drive the development of psychopathology. A stronger focus may be required on the role of altered day-to-day social experiences in the prevention and potentially, the treatment, of adolescent psychopathology.

... read more

Topics: Parenting styles (70%), Social relation (58%), Psychopathology (56%)

1 Citations


41 results found

Journal ArticleDOI: 10.1016/J.TREE.2008.10.008
Abstract: How should ecologists and evolutionary biologists analyze nonnormal data that involve random effects? Nonnormal data such as counts or proportions often defy classical statistical procedures. Generalized linear mixed models (GLMMs) provide a more flexible approach for analyzing nonnormal data when random effects are present. The explosion of research on GLMMs in the last decade has generated considerable uncertainty for practitioners in ecology and evolution. Despite the availability of accurate techniques for estimating GLMM parameters in simple cases, complex GLMMs are challenging to fit and statistical inference such as hypothesis testing remains difficult. We review the use (and misuse) of GLMMs in ecology and evolution, discuss estimation and inference and summarize 'best-practice' data analysis procedures for scientists facing this challenge.

... read more

6,380 Citations

Open accessJournal ArticleDOI: 10.1016/J.JML.2012.11.001
Abstract: Linear mixed-effects models (LMEMs) have become increasingly prominent in psycholinguistics and related areas. However, many researchers do not seem to appreciate how random effects structures affect the generalizability of an analysis. Here, we argue that researchers using LMEMs for confirmatory hypothesis testing should minimally adhere to the standards that have been in place for many decades. Through theoretical arguments and Monte Carlo simulation, we show that LMEMs generalize best when they include the maximal random effects structure justified by the design. The generalization performance of LMEMs including data-driven random effects structures strongly depends upon modeling criteria and sample size, yielding reasonable results on moderately-sized samples when conservative criteria are used, but with little or no power advantage over maximal models. Finally, random-intercepts-only LMEMs used on within-subjects and/or within-items data from populations where subjects and/or items vary in their sensitivity to experimental manipulations always generalize worse than separate F1 and F2 tests, and in many cases, even worse than F1 alone. Maximal LMEMs should be the ‘gold standard’ for confirmatory hypothesis testing in psycholinguistics and beyond.

... read more

5,453 Citations

Open accessJournal ArticleDOI: 10.1038/NRN3475
Abstract: A study with low statistical power has a reduced chance of detecting a true effect, but it is less well appreciated that low power also reduces the likelihood that a statistically significant result reflects a true effect. Here, we show that the average statistical power of studies in the neurosciences is very low. The consequences of this include overestimates of effect size and low reproducibility of results. There are also ethical dimensions to this problem, as unreliable research is inefficient and wasteful. Improving reproducibility in neuroscience is a key priority and requires attention to well-established but often ignored methodological principles.

... read more

Topics: Reproducibility Project (50%)

4,720 Citations

Journal ArticleDOI: 10.1027/1614-2241.1.3.86
Cora J. M. Maas1, Joop J. Hox1Institutions (1)
Abstract: An important problem in multilevel modeling is what constitutes a sufficient sample size for accurate estimation. In multilevel analysis, the major restriction is often the higher-level sample size. In this paper, a simulation study is used to determine the influence of different sample sizes at the group level on the accuracy of the estimates (regression coefficients and variances) and their standard errors. In addition, the influence of other factors, such as the lowest-level sample size and different variance distributions between the levels (different intraclass correlations), is examined. The results show that only a small sample size at level two (meaning a sample of 50 or less) leads to biased estimates of the second-level standard errors. In all of the other simulated conditions the estimates of the regression coefficients, the variance components, and the standard errors are unbiased and accurate.

... read more

Topics: Sample size determination (68%), Standard error (59%), Marginal model (58%) ... read more

2,559 Citations

No. of citations received by the Paper in previous years