Showing papers by "John Concato published in 2022"

Real-World Evidence - Where Are We Now?

Abstract: M than 5 years after the passage of the 21st Century Cures Act of 2016, the terms “realworld data” (RWD) and “real-world evidence” (RWE) are being used inconsistently and sometimes interchangeably. This imprecision has complicated efforts to assess the impact of such data and evidence and hindered attempts to track their use. The Food and Drug Administration (FDA), in its Framework for FDA’s Real-World Evidence Program,1 defined RWD as “data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources” and defined RWE as “clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of RWD,” regardless of the type of study design. But there are two widespread misconceptions about these terms. The first is the notion that RWD and RWE were brand-new concepts in 2016. In reality, sources of data and types of study designs haven’t fundamentally changed, but electronic access to more detailed clinical data is evolving, and such information’s reliability and relevance to research are improving. The availability of more robust data on clinical factors that affect health outcomes also provides opportunities for exploring various statistical methods in lieu of randomization. Including “RWD” or “RWE” in the description of a study, however, doesn’t tell us exactly where the data came from or what kind of study architecture is involved. Providing more specifics about data sources and study design can reduce confusion over RWD and RWE.2 The second misconception is that a simple dichotomy between randomized, controlled trials (RCTs) and observational studies delineates the entire landscape of study design.3 Although randomization of treatment assignment is a key strength of RCTs, not all clinical trials are randomized; rather, their defining feature is assignment of treatment according to an investigational protocol. For example, in single-group trials, investigators assign participants to receive an intervention without randomization — and face challenges similar to those in observational studies in determining whether differences in clinical outcomes between the protocol-driven group and a comparator (“control”) group represent actual treatment effects. Correcting these misconceptions requires recognition that the degree of reliance on RWD varies with the type of study design and that, by definition, RCTs that incorporate RWD generate RWE (see diagram). This conceptualization confirms that even when strict eligibility criteria may limit the generalizability of trial results, trial participants exist, and their outcomes occur, in the “real world” — despite perceptions that generation of RWE occurs only outside clinical trials. Also, although the terms “clinical trials” and “observational studies” have clear meanings when used properly, the terms “interventional studies” and “noninterventional studies” have advantages in describing whether the treatment of interest was administered according to a study protocol. These conceptual distinctions were less pertinent when causal inferences regarding therapeutic effectiveness relied mainly on interventional studies with primary data collected in traditional RCTs. Increasingly, however, RCTs incorporate RWD, and when randomization isn’t feasible for ethical or other reasons, externally controlled trials include a comparator group derived entirely from a source of secondary data (“external” to the treatment group). Conversely, noninterventional studies that analyze primary data collected from registries are being conducted more often. Notwithstanding confusion regarding these terms and concepts, we at the FDA continue to evaluate RWD and RWE as we consider regulatory decisions. Indeed, the agency published four related draft guidance documents in 2021.4 FDA guidance on data from electronic health records and medical claims databases includes recommendations on how to select relevant data sources and define and validate study variables; other guidance provides recommendations on designing or using an existing registry to support regulatory decision making. A guidance document on data standards advises sponsors to document a rationale for changes

Penetrance and Pleiotropy of Polygenic Risk Scores for Schizophrenia, Bipolar Disorder, and Depression Among Adults in the US Veterans Affairs Health Care System

Abstract: This cross-sectional study benchmarks the penetrance of current neuropsychiatric polygenic risk scores in the Veterans Health Administration health care system, and explores associations between polygenic risk scores and broad categories of human disease via phenome-wide association studies.

Targeted learning: Towards a future informed by real-world evidence

Abstract: The 21st Century Cures Act of 2016 includes a provision for the U.S. Food and Drug Administration (FDA) to evaluate the potential use of real-world evidence (RWE) to support new indications for use for previously approved drugs, and to satisfy post-approval study requirements. Extracting reliable evidence from real-world data (RWD) is often complicated by a lack of treatment randomization, potential intercurrent events, and informative loss to follow up. Targeted Learning (TL) is a sub-field of statistics that provides a rigorous framework to help address these challenges. The TL Roadmap offers a step-by-step guide to generating valid evidence and assessing its reliability. Following these steps produces an extensive amount of information for assessing whether the study provides reliable scientific evidence in support regulatory decision making. This paper presents two case studies that illustrate the utility of following the roadmap. We use targeted minimum loss-based estimation combined with super learning to estimate causal effects. We also compared these findings with those obtained from an unadjusted analysis, propensity score matching, and inverse probability weighting. Non-parametric sensitivity analyses illuminate how departures from (untestable) causal assumptions would affect point estimates and confidence interval bounds that would impact the substantive conclusion drawn from the study. TL's thorough approach to learning from data provides transparency, allowing trust in RWE to be earned whenever it is warranted.

Real-World Evidence Prediction of a Phase IV Oncology Trial: Comparative Degarelix vs Leuprolide Safety

Abstract: Abstract Background Medical and regulatory communities are increasingly interested in the utility of real-world evidence (RWE) for answering questions pertaining to drug safety and effectiveness, but concerns about validity remain. A principled approach to conducting RWE studies may alleviate concerns and increase confidence in findings. This study sought to predict the findings from the PRONOUNCE trial using a principled approach to generating RWE. Methods This propensity score–matched observational cohort study used 3 claims databases to compare the occurrence of major adverse cardiovascular events among initiators of degarelix vs leuprolide. Patients were included if they had a history of prostate cancer and atherosclerotic cardiovascular disease. Patients were excluded if they did not have continuous database enrollment in the year before treatment initiation, were exposed to androgen deprivation therapy or experienced an acute cardiovascular event within 30 days before treatment initiation, or had a history or risk factors of QT prolongation. Results There were 12 448 leuprolide and 1969 degarelix study-eligible patients before matching, with 1887 in each arm after propensity score matching. The results for major adverse cardiovascular events comparing degarelix with leuprolide in the observational analysis (hazard ratio = 1.35, 95% confidence interval = 0.94 to 1.93) was consistent with the subsequently released PRONOUNCE result (hazard ratio = 1.28, 95% confidence interval = 0.59 to 2.79). Conclusions This study successfully predicted the result of a comparative cardiovascular safety trial in the oncology setting. Although the findings are encouraging, limitations of measuring cancer stage and tumor progression are representative of challenges in attempting to generalize whether claims-based RWE can be used as actionable evidence.

Association of Gulf War Illness with Characteristics in Deployed vs. Non-Deployed Gulf War Era Veterans in the Cooperative Studies Program 2006/Million Veteran Program 029 Cohort: A Cross-Sectional Analysis

Abstract: Gulf War Illness (GWI), a chronic multisymptom illness with a complex and uncertain etiology and pathophysiology, is highly prevalent among veterans deployed to the 1990–1991 GW. We examined how GWI phenotypes varied by demographic and military characteristics among GW-era veterans. Data were from the VA’s Cooperative Studies Program 2006/Million Veteran Program (MVP) 029 cohort, Genomics of GWI. From June 2018 to March 2019, 109,976 MVP enrollees (out of a total of over 676,000) were contacted to participate in the 1990–1991 GW-era Survey. Of 109,976 eligible participants, 45,169 (41.1%) responded to the 2018–2019 survey, 35,902 respondents met study inclusion criteria, 13,107 deployed to the GW theater. GWI phenotypes were derived from Kansas (KS) and Centers for Disease Control and Prevention (CDC) GWI definitions: (a) KS Symptoms (KS Sym+), (b) KS GWI (met symptom criteria and without exclusionary health conditions) [KS GWI: Sym+/Dx−], (c) CDC GWI and (d) CDC GWI Severe. The prevalence of each phenotype was 67.1% KS Sym+, 21.5% KS Sym+/Dx−, 81.1% CDC GWI, and 18.6% CDC GWI severe. These findings affirm the persistent presence of GWI among GW veterans providing a foundation for further exploration of biological and environmental underpinnings of this condition.

Penetrance and Pleiotropy of Polygenic Risk Scores for Schizophrenia, Bipolar Disorder, and Depression in the VA Health Care System

Abstract: Background: Serious mental illnesses, including schizophrenia, bipolar disorder and depression are heritable, highly multifactorial disorders and major causes of disability worldwide. Polygenic risk scores (PRS) aggregate variants identified from genome-wide association studies (GWAS) into individual-level estimates of liability and are a promising tool for clinical risk stratification. Methods: By leveraging the VA's extensive electronic health record (EHR) and a cohort of 9,378 individuals with confirmed diagnoses of schizophrenia or bipolar I disorder, we validated automated case-control assignments based on ICD-9/10 codes, and benchmarked the performance of current PRS for schizophrenia, bipolar disorder, and major depression in 400,000 Million Veteran Program (MVP) participants. We explored broader relationships between PRS and 1,650 disease categories via phenome-wide association studies (PheWAS). Finally, we applied genomic structural equation modeling (gSEM) to derive novel PRS indexing common and disorder-specific latent genetic factors. Results: Among 3,953 and 5,425 individuals with diagnoses of schizophrenia or bipolar disorder type I that were confirmed by structured clinical interviews, 95% were correctly identified using ICD-9/10 codes (2 or more). Current PRS were robustly associated with case status in European (p<10-254) and African (p<10-5) participants and were higher among more frequently hospitalized patients (p<10-4). PheWAS confirmed previous associations among higher neuropsychiatric PRS and elevated risk for psychiatric and physical health problems and extended these findings to African Americans. Conclusions: Using diagnoses confirmed by in-person structured clinical interviews and current neuropsychiatric PRS, we demonstrated the validity of an EHR-based phenotyping approach in US veterans, highlighting the potential of PRS for disentangling biological and mediated pleiotropy.

Evaluating and improving real-world evidence with Targeted Learning

Abstract: Purpose: The Targeted Learning roadmap provides a systematic guide for generating and evaluating real-world evidence (RWE). From a regulatory perspective, RWE arises from diverse sources such as randomized controlled trials that make use of real-world data, observational studies, and other study designs. This paper illustrates a principled approach to assessing the validity and interpretability of RWE. Methods: We applied the roadmap to a published observational study of the dose-response association between ritodrine hydrochloride and pulmonary edema among women pregnant with twins in Japan. The goal was to identify barriers to causal effect estimation beyond unmeasured confounding reported by the study's authors, and to explore potential options for overcoming the barriers that robustify results. Results: Following the roadmap raised issues that led us to formulate alternative causal questions that produced more reliable, interpretable RWE. The process revealed a lack of information in the available data to identify a causal dose-response curve. However, under explicit assumptions the effect of treatment with any amount of ritodrine versus none, albeit a less ambitious parameter, can be estimated from data. Conclusion: Before RWE can be used in support of clinical and regulatory decision-making, its quality and reliability must be systematically evaluated. The TL roadmap prescribes how to carry out a thorough, transparent, and realistic assessment of RWE. We recommend this approach be a routine part of any decision-making process.