Using large, publicly available datasets to study adolescent development: Opportunities and challenges
TL;DR: The opportunities and challenges associated with large, longitudinal phenotypically rich data sets available for reuse, provide an overview of particularly valuable resources available to the field, and recommend best practices to improve the rigor and transparency of analyses conducted on large, secondary data sets.
Abstract: Adolescence is a period of rapid change, with cognitive, mental wellbeing, environmental biological factors interacting to shape lifelong outcomes. Large, longitudinal phenotypically rich data sets available for reuse (secondary data) have revolutionized the way we study adolescence, allowing the field to examine these unfolding processes across hundreds or even thousands of individuals. Here, we outline the opportunities and challenges associated with such secondary data sets, provide an overview of particularly valuable resources available to the field, and recommend best practices to improve the rigor and transparency of analyses conducted on large, secondary data sets.
Citations
More filters
TL;DR: The Adolescent Brain Cognitive Development (ABCD) Study as discussed by the authors has provided immense opportunities for researchers across disciplines since its first data release in 2018, but the size and scope of the study also present a number of hurdles, which range from becoming familiar with the study design and data structure to employing rigorous and reproducible analyses.
11 citations
TL;DR: In this paper , the authors investigate qualitative differences between small-scale and large-scale studies, which are not due to their size per se, and demonstrate how these qualitative differences might pose considerable challenges to the interpretation of research results.
8 citations
TL;DR: In this article , the authors discuss three open science practices (pre- and post-registration, registered reports, and data management) and the opportunities they bring to facilitate enhanced credibility in longitudinal studies.
Abstract: Longitudinal studies provide unique opportunities to study dynamic developmental processes over time and are often afforded a high degree of credibility. Transparency facilitates evaluation of credibility, yet, research practices that can increase transparency, that is, open science practices, do not appear to be widely implemented in longitudinal developmental research. In the current article I discuss three open science practices (pre- and post-registration, Registered Reports, and data management) and the opportunities they bring to facilitate enhanced credibility in longitudinal studies. Drawing on my own experiences of conducting longitudinal developmental research on adolescent mental health, I provide practical examples of how these open science practices can be implemented. Using open science practices in longitudinal research is also accompanied by challenges, and I specifically discuss the issue of evidencing prior knowledge of data in Registered Reports and some potential solutions to this challenge.
3 citations
TL;DR: A brief overview of methodological approaches and research designs that bridge brain and behavioral research on learning can be found in this article , where the authors argue that ultimately these methods and designs may help to unravel questions such as why learning interventions work, what learning computations change across development, and how learning difficulties are distinct between individuals.
Abstract: Abstract The brain undergoes profound development across childhood and adolescence, including continuous changes in brain morphology, connectivity, and functioning that are, in part, dependent on one’s experiences. These neurobiological changes are accompanied by significant changes in children’s and adolescents’ cognitive learning. By drawing from studies in the domains of reading, reinforcement learning, and learning difficulties, we present a brief overview of methodological approaches and research designs that bridge brain- and behavioral research on learning. We argue that ultimately these methods and designs may help to unravel questions such as why learning interventions work, what learning computations change across development, and how learning difficulties are distinct between individuals.
3 citations
TL;DR: A brief overview of methodological approaches and research designs that bridge brain and behavioral research on learning can be found in this article , where the authors argue that ultimately these methods and designs may help to unravel questions such as why learning interventions work, what learning computations change across development, and how learning difficulties are distinct between individuals.
Abstract: Abstract The brain undergoes profound development across childhood and adolescence, including continuous changes in brain morphology, connectivity, and functioning that are, in part, dependent on one’s experiences. These neurobiological changes are accompanied by significant changes in children’s and adolescents’ cognitive learning. By drawing from studies in the domains of reading, reinforcement learning, and learning difficulties, we present a brief overview of methodological approaches and research designs that bridge brain- and behavioral research on learning. We argue that ultimately these methods and designs may help to unravel questions such as why learning interventions work, what learning computations change across development, and how learning difficulties are distinct between individuals.
3 citations
References
More filters
TL;DR: For example, this article found a strong relationship between the breadth of exposure to abuse or household dysfunction during childhood and multiple risk factors for several of the leading causes of death in adults.
12,712 citations
TL;DR: It is shown that despite empirical psychologists’ nominal endorsement of a low rate of false-positive findings, flexibility in data collection, analysis, and reporting dramatically increases actual false- positive rates, and a simple, low-cost, and straightforwardly effective disclosure-based solution is suggested.
Abstract: In this article, we accomplish two things. First, we show that despite empirical psychologists' nominal endorsement of a low rate of false-positive findings (≤ .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We present computer simulations and a pair of actual experiments that demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis. Second, we suggest a simple, low-cost, and straightforwardly effective disclosure-based solution to this problem. The solution involves six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process.
4,727 citations
TL;DR: A straightforward guide to understanding, selecting, calculating, and interpreting effect sizes for many types of data and to methods for calculating effect size confidence intervals and power analysis is provided.
Abstract: The Publication Manual of the American Psychological Association (American Psychological Association, 2001, American Psychological Association, 2010) calls for the reporting of effect sizes and their confidence intervals. Estimates of effect size are useful for determining the practical or theoretical importance of an effect, the relative contributions of factors, and the power of an analysis. We surveyed articles published in 2009 and 2010 in the Journal of Experimental Psychology: General, noting the statistical analyses reported and the associated reporting of effect size estimates. Effect sizes were reported for fewer than half of the analyses; no article reported a confidence interval for an effect size. The most often reported analysis was analysis of variance, and almost half of these reports were not accompanied by effect sizes. Partial η2 was the most commonly reported effect size estimate for analysis of variance. For t tests, 2/3 of the articles did not report an associated effect size estimate; Cohen's d was the most often reported. We provide a straightforward guide to understanding, selecting, calculating, and interpreting effect sizes for many types of data and to methods for calculating effect size confidence intervals and power analysis.
3,117 citations
TL;DR: In this article, the authors show that identifying conditions invoked in previous applications of regression discontinuity methods are often overly strong and that treatment effects can be nonparametrically identified under an RD design by a weak functional form restriction.
Abstract: Ž. THE REGRESSION DISCONTINUITY RD data design is a quasi-experimental design with the defining characteristic that the probability of receiving treatment changes discontinuously as a function of one or more underlying variables. This data design arises frequently in economic and other applications but is only infrequently exploited as a source of identifying information in evaluating effects of a treatment. In the first application and discussion of the RD method, Thistlethwaite and Campbell Ž. 1960 study the effect of student scholarships on career aspirations, using the fact that awards are only made if a test score exceeds a threshold. More recently, Van der Klaauw Ž. 1997 estimates the effect of financial aid offers on students’ decisions to attend a particular college, taking into account administrative rules that set the aid amount partly on the basis of a discontinuous function of the students’ grade point average and SAT Ž. score. Angrist and Lavy 1999 estimate the effect of class size on student test scores, taking advantage of a rule stipulating that another classroom be added when the average Ž. class size exceeds a threshold level. Finally, Black 1999 uses an RD approach to estimate parents’ willingness to pay for higher quality schools by comparing housing prices near geographic school attendance boundaries. Regression discontinuity methods have potentially broad applicability in economic research, because geographic boundaries or rules governing programs often create discontinuities in the treatment assignment mechanism that can be exploited under the method. Although there have been several discussions and applications of RD methods in the literature, important questions still remain concerning sources of identification and ways of estimating treatment effects under minimal parametric restrictions. Here, we show that identifying conditions invoked in previous applications of RD methods are often overly strong and that treatment effects can be nonparametrically identified under an RD design by a weak functional form restriction. The restriction is unusual in that it requires imposing continuity assumptions in order to take advantage of the known discontinuity in the treatment assignment mechanism. We also propose a way of nonparametrically estimating treatment effects and offer an interpretation of the Wald estimator as an RD estimator.
2,577 citations
TL;DR: A hypothesis is proposed that experience during a sensitive period modifies the architecture of a circuit in fundamental ways, causing certain patterns of connectivity to become highly stable and, therefore, energetically preferred.
Abstract: Experience exerts a profound influence on the brain and, therefore, on behavior. When the effect of experience on the brain is particularly strong during a limited period in development, this period is referred to as a sensitive period. Such periods allow experience to instruct neural circuits to process or represent information in a way that is adaptive for the individual. When experience provides information that is essential for normal development and alters performance permanently, such sensitive periods are referred to as critical periods. Although sensitive periods are reflected in behavior, they are actually a property of neural circuits. Mechanisms of plasticity at the circuit level are discussed that have been shown to operate during sensitive periods. A hypothesis is proposed that experience during a sensitive period modifies the architecture of a circuit in fundamental ways, causing certain patterns of connectivity to become highly stable and, therefore, energetically preferred. Plasticity that occurs beyond the end of a sensitive period, which is substantial in many circuits, alters connectivity patterns within the architectural constraints established during the sensitive period. Preferences in a circuit that result from experience during sensitive periods are illustrated graphically as changes in a ''stability landscape,'' a metaphor that represents the relative contributions of genetic and experiential influences in shaping the information processing capabilities of a neural circuit. By understanding sensitive periods at the circuit level, as well as understanding the relationship between circuit properties and behavior, we gain a deeper insight into the critical role that experience plays in shaping the development of the brain and behavior.
1,355 citations