The TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) Statement includes a 22-item checklist, which aims to improve the reporting of studies developing, validating, or updating a prediction model, whether for diagnostic or prognostic purposes. The TRIPOD Statement aims to improve the transparency of the reporting of a prediction model study regardless of the study methods used. This explanation and elaboration document describes the rationale; clarifies the meaning of each item; and discusses why transparent reporting is important, with a view to assessing risk of bias and clinical usefulness of the prediction model. Each checklist item of the TRIPOD Statement is explained in detail and accompanied by published examples of good reporting. The document also provides a valuable reference of issues to consider when designing, conducting, and analyzing prediction model studies. To aid the editorial process and help peer reviewers and, ultimately, readers and systematic reviewers of prediction model studies, it is recommended that authors include a completed checklist in their submission. The TRIPOD checklist can also be downloaded from www.tripod-statement.org.

/pdf/transparent-reporting-of-a-multivariable-prediction-model-3n8wy14hp9.pdf

Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration.

Prevalence, Impact, and Predictive Value of Detecting Subclinical Coronary and Carotid Atherosclerosis in Asymptomatic Adults: The BioImage Study

Importance Precise stratification of cardiovascular risk in patients with coronary heart disease (CHD) is needed to inform treatment decisions. Objective To derive and validate a score to predict risk of cardiovascular outcomes among patients with CHD, using large-scale analysis of circulating proteins. Design, Setting, and Participants Prospective cohort study of participants with stable CHD. For the derivation cohort (Heart and Soul study), outpatients from San Francisco were enrolled from 2000 through 2002 and followed up through November 2011 (≤11.1 years). For the validation cohort (HUNT3, a Norwegian population-based study), participants were enrolled from 2006 through 2008 and followed up through April 2012 (5.6 years). Exposures Using modified aptamers, 1130 proteins were measured in plasma samples. Main Outcomes and Measures A 9-protein risk score was derived and validated for 4-year probability of myocardial infarction, stroke, heart failure, and all-cause death. Tests, including the C statistic, were used to assess performance of the 9-protein risk score, which was compared with the Framingham secondary event model, refit to the cohorts in this study. Within-person change in the 9-protein risk score was evaluated in the Heart and Soul study from paired samples collected 4.8 years apart. Results From the derivation cohort, 938 samples were analyzed, participants’ median age at enrollment was 67.0 years, and 82% were men. From the validation cohort, 971 samples were analyzed, participants’ median age at enrollment was 70.2 years, and 72% were men. In the derivation cohort, C statistics were 0.66 for refit Framingham, 0.74 for 9-protein, and 0.75 for refit Framingham plus 9-protein models. In the validation cohort, C statistics were 0.64 for refit Framingham, 0.70 for 9-protein, and 0.71 for refit Framingham plus 9-protein models. Adding the 9-protein risk score to the refit Framingham model increased the C statistic by 0.09 (95% CI, 0.06-0.12) in the derivation cohort, and in the validation cohort, the C statistic was increased by 0.05 (95% CI, 0.02-0.09). Compared with the refit Framingham model, the integrated discrimination index for the 9-protein model was 0.12 (95% CI, 0.08-0.16) in the derivation cohort and 0.08 (95% CI, 0.05-0.10) in the validation cohort. In analysis of paired samples among 139 participants with cardiovascular events after the second sample, absolute within-person annualized risk increased more for the 9-protein model (median, 1.86% [95% CI, 1.15%-2.54%]) than for the refit Framingham model (median, 1.00% [95% CI, 0.87%-1.19%]) (P = .002), while among 375 participants without cardiovascular events, both scores changed less and similarly (P = .30). Conclusions and Relevance Among patients with stable CHD, a risk score based on 9 proteins performed better than the refit Framingham secondary event risk score in predicting cardiovascular events, but still provided only modest discriminative accuracy. Further research is needed to assess whether the score is more accurate in a lower-risk population.

/pdf/development-and-validation-of-a-protein-based-risk-score-for-4uw0eu2lt7.pdf

Development and Validation of a Protein-Based Risk Score for Cardiovascular Outcomes Among Patients With Stable Coronary Heart Disease

The area under the receiver operating characteristic curve is often used as a summary index of the diagnostic ability in evaluating biomarkers when the clinical outcome (truth) is binary. When the clinical outcome is right-censored survival time, the C index, motivated as an extension of area under the receiver operating characteristic curve, has been proposed by Harrell as a measure of concordance between a predictive biomarker and the right-censored survival outcome. In this work, we investigate methods for statistical comparison of two diagnostic or predictive systems, of which they could either be two biomarkers or two fixed algorithms, in terms of their C indices. We adopt a U-statistics-based C estimator that is asymptotically normal and develop a nonparametric analytical approach to estimate the variance of the C estimator and the covariance of two C estimators. A z-score test is then constructed to compare the two C indices. We validate our one-shot nonparametric method via simulation studies in terms of the type I error rate and power. We also compare our one-shot method with resampling methods including the jackknife and the bootstrap. Simulation results show that the proposed one-shot method provides almost unbiased variance estimations and has satisfactory type I error control and power. Finally, we illustrate the use of the proposed method with an example from the Framingham Heart Study.

/pdf/comparing-two-correlated-c-indices-with-right-censored-ad7el5mox4.pdf

Comparing two correlated C indices with right-censored survival outcome: a one-shot nonparametric approach

Cardiovascular disease (CVD) is among the leading causes of death and disability worldwide. Since its beginning, the Framingham study has been a leader in identifying CVD risk factors. Clinical trials have demonstrated that when the modifiable risk factors are treated and corrected, the chances of CVD occurring can be reduced. The Framingham study also recognized that CVD risk factors are multifactorial and interact over time to produce CVD. In response, Framingham investigators developed the Framingham Risk Functions (also called Framingham Risk Scores) to evaluate the chance or likelihood of developing CVD in individuals. These functions are multivariate functions (algorithms) that combine the information in CVD risk factors such as sex, age, systolic blood pressure, total cholesterol, high-density lipoprotein cholesterol, smoking behavior, and diabetes status to produce an estimate (or risk) of developing CVD or a component of CVD (such as coronary heart disease, stroke, peripheral vascular disease, or heart failure) over a fixed time, for example, the next 10 years. These estimates of CVD risk are often major inputs in recommending drug treatments such as cholesterol-lowering drugs.

Cardiovascular Disease Risk Assessment: Insights from Framingham.

Cardiovascular risk prediction functions offer an important diagnostic tool for clinicians and patients themselves. They are usually constructed with the use of parametric or semi-parametric survival regression models. It is essential to be able to evaluate the performance of these models, preferably with summaries that offer natural and intuitive interpretations. The concept of discrimination, popular in the logistic regression context, has been extended to survival analysis. However, the extension is not unique. In this paper, we define discrimination in survival analysis as the model's ability to separate those with longer event-free survival from those with shorter event-free survival within some time horizon of interest. This definition remains consistent with that used in logistic regression, in the sense that it assesses how well the model-based predictions match the observed data. Practical and conceptual examples and numerical simulations are employed to examine four C statistics proposed in the literature to evaluate the performance of survival models. We observe that they differ in the numerical values and aspects of discrimination that they capture. We conclude that the index proposed by Harrell is the most appropriate to capture discrimination described by the above definition. We suggest researchers report which C statistic they are using, provide a rationale for their selection, and be aware that comparing different indices across studies may not be meaningful.

/pdf/quantifying-discrimination-of-framingham-risk-functions-with-1t6bjfu5m3.pdf

Linye Song

Papers

Quantifying discrimination of Framingham risk functions with different survival C statistics