Multivariable regression models are powerful tools that are used frequently in studies of clinical outcomes. These models can use a mixture of categorical and continuous variables and can handle partially observed (censored) responses. However, uncritical application of modelling techniques can result in models that poorly fit the dataset at hand, or, even more likely, inaccurately predict outcomes on new subjects. One must know how to measure qualities of a model's fit in order to avoid poorly fitted or overfitted models. Measurement of predictive accuracy can be difficult for survival time data in the presence of censoring. We discuss an easily interpretable index of predictive discrimination as well as methods for assessing calibration of predicted survival probabilities. Both types of predictive accuracy should be unbiasedly validated using bootstrapping or cross-validation, before using predictions in a new data series. We discuss some of the hazards of poorly fitted and overfitted regression models and present one modelling strategy that avoids many of the problems discussed. The methods described are applicable to all regression models, but are particularly needed for binary, ordinal, and time-to-event outcomes. Methods are illustrated with a survival analysis in prostate cancer using Cox regression.

/pdf/multivariable-prognostic-models-issues-in-developing-models-49wjg4tzp4.pdf

Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors

Prognostic/Clinical Prediction Models: Multivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors

SUMMARY Multivariable regression models are powerful tools that are used frequently in studies of clinical outcomes. These models can use a mixture of categorical and continuous variables and can handle partially observed (censored) responses. However, uncritical application of modelling techniques can result in models that poorly fit the dataset at hand, or, even more likely, inaccurately predict outcomes on new subjects. One must know how to measure qualities of a model's fit in order to avoid poorly fitted or overfitted models. Measurement of predictive accuracy can be difficult for survival time data in the presence of censoring. We discuss an easily interpretable index of predictive discrimination as well as methods for assessing calibration of predicted survival probabilities. Both types of predictive accuracy should be unbiasedly validated using bootstrapping or cross-validation, before using predictions in a new data series. We discuss some of the hazards of poorly fitted and overfitted regression models and present one modelling strategy that avoids many of the problems discussed. The methods described are applicable to all regression models, but are particularly needed for binary, ordinal, and time-to-event outcomes. Methods are illustrated with a survival analysis in prostate cancer using Cox regression. Accurate estimation of patient prognosis is important for many reasons. First, prognostic estimates can be used to inform the patient about likely outcomes of her disease. Second, the physician can use estimates of prognosis as a guide for ordering additional tests and selecting appropriate therapies. Third, prognostic assessments are useful in the evaluation of technologies; prognostic estimates derived both with and without using the results of a given test can be compared to measure the incremental prognostic information provided by that test over what is provided by prior information.' Fourth, a researcher may want to estimate the effect of a single factor (for example, treatment given) on prognosis in an observational study in which many uncontrolled confounding factors are also measured. Here the simultaneous effects of the uncontrolled variables must be controlled (held constant mathematically if using a regression model) so that the effect of the factor of interest can be more purely estimated. An analysis of how variables (especially continuous ones) affect the patient outcomes of interest is necessary to

/pdf/tutorial-in-biostatistics-multivariable-prognostic-models-4auy6032o6.pdf

Tutorial in biostatistics multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors

Regression models are frequently used to develop diagnostic, prognostic, and health resource utilization models in clinical, health services, outcomes, pharmacoeconomic, and epidemiologic research, and in a multitude of non-health-related areas. Regression models are also used to adjust for patient heterogeneity in randomized clinical trials, to obtain tests that are more powerful and valid than unadjusted treatment comparisons.

/pdf/regression-modeling-strategies-1jyn0f4r32.pdf

Regression Modeling Strategies

https://journal.chestnet.org/article/S0012-3692(16)52804-9/pdf

The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults.

We describe the use of cubic splines in regression models to represent the relationship between the response variable and a vector of covariates. This simple method can help prevent the problems that result from inappropriate linearity assumptions. We compare restricted cubic spline regression to non-parametric procedures for characterizing the relationship between age and survival in the Stanford Heart Transplant data. We also provide an illustrative example in cancer therapeutics.

Flexible regression models with cubic splines

STUDY OBJECTIVE To document the long-term prognosis of patients with non-Hodgkin lymphoma treated between 1971 and 1975 and to determine how the prognosis varies by histologic subtype and stage. SETTING Three cancer referral centers in the United States and one center in Italy. PATIENTS A consecutive sample of 1153 previously untreated patients with non-Hodgkin lymphoma. At the time of analysis, 71% of the patients had died and the median follow-up for patients still alive was 11.2 years. MEASUREMENTS AND MAIN RESULTS The 10-year survival proportions were 45% (CI, 40% to 50%); 26% (CI, 22% to 30%); and 23% (CI, 18% to 30%) for patients with low-, intermediate-, and high-grade lymphomas, respectively. Patients with intermediate- and high-grade lymphomas were curable, but this was not apparent for patients with advanced stage low-grade lymphomas. For the low-grade follicular small cleaved and follicular mixed lymphomas, the Ann Arbor staging system distinguished the prognosis of patients with stage I disease from those with more extensive involvement. For the diffuse large cell and immunoblastic lymphomas, the Ann Arbor staging system distinguished long-term prognosis for patients with stage I disease from patients with stage II disease and those with more disseminated involvement. CONCLUSIONS The probability of long-term survival for unselected patients with non-Hodgkin lymphoma can be substantial. Long-term prognosis depends on the histologic subtype of the tumor and the extent of dissemination. The Working Formulation for non-Hodgkin lymphomas is a simple and useful nomenclature for selecting treatment and reporting results. The Ann Arbor staging system is a useful but imperfect prognostic indicator.

The Non-Hodgkin Lymphoma Pathologic Classification Project. Long-term follow-up of 1153 patients with non-Hodgkin lymphomas.

Demonstrating therapeutic equivalence of two treatments is the goal of many clinical trials. For instance, when the toxicity of an effective standard treatment is of concern, much effort is devoted to developing new therapies that would be both as effective and less toxic. In this paper we review the special characteristics of these trials and describe sequential monitoring of equivalence studies using repeated confidence intervals. We show how sequential monitoring may be of particular value in this setting and critically discuss the choice of some important design parameters. We also provide tables for use when planning a sequential equivalence trial.

https://www.jstor.org/stable/pdfplus/2531438.pdf

Planning and monitoring of equivalence studies.

For life-threatening diseases, ethical considerations preclude the inclusion of an untreated control group in the investigation of a new therapeutic agent when a standard therapy exists. In these cases, active controlled studies are conducted, and may be planned to demonstrate either superiority or equivalence/non-inferiority of the new drug over the standard therapy (active control). In the non-inferiority study, an important aspect is the ability to detect an inferior drug (assay sensitivity). It has been suggested that assay sensitivity for a non-inferiority study should be deduced from historical data, specifically placebo controlled studies with the standard therapy. The assessment of assay sensitivity may also be important in a superiority trial that fails to demonstrate a statistically significant difference between treatments, and the sponsor attempts to determine whether there is lack of inferiority as an alternative hypothesis for regulatory approval. This paper describes two methods of putative placebo analysis for assessing assay sensitivity in active controlled trials. One approach imputes a point estimate for the odds ratio (95 per cent confidence interval) for a new drug (T) compared to a placebo control (P). A Bayesian approach calculates the posterior probability that T is superior to P, or, that T is at least k per cent as good as the active control (A) and A is more effective than P. These methods are applied in two clinical/regulatory settings: a phase III trial comparing docetaxel (Taxotere) to doxorubicin in metastatic breast cancer patients, and a phase III programme with two trials comparing enoxaparin (Lovenox) plus aspirin to unfractionated heparin plus aspirin in patients with unstable angina or non-Q-wave myocardial infarction. The methodologies presented in this paper were used in securing regulatory approval for docetaxel in the treatment of locally advanced or metastatic breast cancer after failure of prior chemotherapy, and for enoxaparin in the treatment of acute coronary syndrome.

Sylvain Durrleman

Papers

Flexible regression models with cubic splines

The Non-Hodgkin Lymphoma Pathologic Classification Project. Long-term follow-up of 1153 patients with non-Hodgkin lymphomas.

Planning and monitoring of equivalence studies.

The use of putative placebo in active control trials: two applications in a regulatory setting.

Planning and monitoring of equivalence studies