4. Modelling Survival Data in Medical Research

Informative diagnostic tools are vital to the development of useful mixed-effects models. The Visual Predictive Check (VPC) is a popular tool for evaluating the performance of population PK and PKPD models. Ideally, a VPC will diagnose both the fixed and random effects in a mixed-effects model. In many cases, this can be done by comparing different percentiles of the observed data to percentiles of simulated data, generally grouped together within bins of an independent variable. However, the diagnostic value of a VPC can be hampered by binning across a large variability in dose and/or influential covariates. VPCs can also be misleading if applied to data following adaptive designs such as dose adjustments. The prediction-corrected VPC (pcVPC) offers a solution to these problems while retaining the visual interpretation of the traditional VPC. In a pcVPC, the variability coming from binning across independent variables is removed by normalizing the observed and simulated dependent variable based on the typical population prediction for the median independent variable in the bin. The principal benefit with the pcVPC has been explored by application to both simulated and real examples of PK and PKPD models. The investigated examples demonstrate that pcVPCs have an enhanced ability to diagnose model misspecification especially with respect to random effects models in a range of situations. The pcVPC was in contrast to traditional VPCs shown to be readily applicable to data from studies with a priori and/or a posteriori dose adaptations.

Prediction-Corrected Visual Predictive Checks for Diagnosing Nonlinear Mixed-Effects Models

: Scientific learning is an iterative process employing Criticism and Estimation. Correspondingly the formulated model factors into two complimentary parts - a predictive part allowing model criticism, and a Bayes posterior part allowing estimation. Implications for significance tests, the theory of precise measurement, and for ridge estimates are considered. Predictive checking functions for transformation, serial correlation, bad values, and their relation with Bayesian options are considered. Robustness is seen from a Bayesian viewpoint and examples are given. For the bad value problem a comparison with M estimators is made. (Author)

Sampling and Bayes' inference in scientific modelling and robustness

Conclusions from clinical trial results that are derived from model-based analyses rely on the model adequately describing the underlying system. The traditionally used diagnostics intended to provide information about model adequacy have seldom discussed shortcomings. Without an understanding of the properties of these diagnostics, development and use of new diagnostics, and additional information pertaining to the diagnostics, there is risk that adequate models will be rejected and inadequate models accepted. Thus, a diagnosis of available diagnostics is desirable.

Diagnosing Model Diagnostics

A growing number of population pharmacokinetic analyses of therapeutic monoclonal antibodies (mAbs) have been published in the scientific literature. The aims of this article are to summarize the findings from these studies and to relate the findings to the general pharmacokinetic and structural characteristics of therapeutic mAbs. A two-compartment model was used in the majority of the population analyses to describe the disposition of the mAb. Population estimates of the volumes of distribution in the central (V1) and peripheral (V2) compartments were typically small, with median (range) values of 3.1 (2.4–5.5) L and 2.8 (1.3–6.8) L, respectively. The estimated between-subject variability in the V1 was usually moderate, with a median (range) coefficient of variation (CV) of 26% (12–84%). Between-subject variability in other distribution-related parameters such as the V2 and intercompartmental clearance were often not estimated. Although the pharmacokinetic models used most frequently in the population analyses were models with linear clearance, other models with nonlinear, or parallel linear and nonlinear clearance pathways were also applied, as many therapeutic mAbs are eliminated via saturable target-mediated mechanisms. Population estimates of the maximum elimination rate (Vmax) and the mAb concentration at which elimination was at half maximum for Michaelis-Menten-type elimination pathways varied considerably among the different therapeutic mAbs. However, estimates of the total clearance (CL) of mAbs with linear clearance characteristics and of the clearance of mAbs via the linear clearance pathway (CLL) with parallel linear and nonlinear clearance were quite similar for the different mAbs and typically ranged from 0.2 to 0.5 L/day, which is relatively close to the estimated clearance of endogenous IgG of 0.21 L/day. The between-subject variability in the Vmax, CL and CLL was moderate to high, with estimated CVs ranging from 15% to 65%. Measures of body size were the covariates most commonly identified as influencing the pharmacokinetics of therapeutic mAbs. In summary, many features of the population pharmacokinetics of currently used therapeutic mAbs are similar, despite differences in their pharmacological targets and studied patient populations.

Population pharmacokinetics of therapeutic monoclonal antibodies.

https://www.hal.inserm.fr/inserm-00274332/document

Computing normalised prediction distribution errors to evaluate nonlinear mixed-effect models: The npde add-on package for R

Purpose
The aim of this study is to define and illustrate metrics for the external evaluation of a population model.

https://www.hal.inserm.fr/inserm-00189557/document

Metrics for External Model Evaluation with an Application to the Population Pharmacokinetics of Gliclazide

AIMS: A descriptive survey of published population pharmacokinetic and/or pharmacodynamic (PK/PD) analyses from 2002 to 2004 was conducted and an evaluation made of how model building was performed and reported. METHODS: We selected 324 articles in Pubmed using defined keywords. A data abstraction form (DAF) was then built comprising two parts: general characteristics including article identification, context of the analysis, description of clinical studies from which the data arose, and model building, including description of the processes of modelling. The papers were examined by two readers, who extracted the relevant information and transmitted it directly to a MySQL database, from which descriptive statistical analysis was performed. RESULTS: Most published papers concerned patients with severe pathology and therapeutic classes suffering from narrow therapeutic index and/or high PK/PD variability. Most of the time, modelling was performed for descriptive purposes, with rich rather than sparse data and using NONMEM software. PK and PD models were rarely complex (one or two compartments for PK; E(max) for PD models). Covariate testing was frequently performed and essentially based on the likelihood ratio test. Based on a minimal list of items that should systematically be found in a population PK-PD analysis, it was found that only 39% and 8.5% of the PK and PD analyses, respectively, published from 2002 to 2004 provided sufficient detail to support the model-building methodology. CONCLUSIONS: This survey allowed an efficient description of recent published population analyses, but also revealed deficiencies in reporting information on model building.

https://bpspubs.onlinelibrary.wiley.com/doi/pdf/10.1111/j.1365-2125.2007.02975.x

Overview of model-building strategies in population PK/PD analyses: 2002-2004 literature survey.

To evaluate by simulation the statistical properties of normalized prediction distribution errors (NPDE), prediction discrepancies (pd), standardized prediction errors (SPE), numerical predictive check (NPC) and decorrelated NPC (NPC(dec)) for the external evaluation of a population pharmacokinetic analysis, and to illustrate the use of NPDE for the evaluation of covariate models. We assume that a model M(B) has been built using a building dataset B, and that a separate validation dataset, V is available. Our null hypothesis H(0) is that the data in V can be described by M(B). We use several methods to test this hypothesis: NPDE, pd, SPE, NPC and NPC(dec). First, we evaluated by simulation the type I error under H(0) of different tests applied to the four methods. We also propose and evaluate a single global test combining normality, mean and variance tests applied to NPDE, pd and SPE. We perform tests on NPC and NPC(dec), after a decorrelation. M(B) was a one compartment model with first order absorption (without covariate), previously developed from two phase II and one phase III studies of the antidiabetic drug, gliclazide. We simulated 500 external datasets according to the design of a phase III study. Second, we investigated the application of NPDE to covariate models. We propose two approaches: the first approach uses correlation tests or mean comparisons to test the relationship between NPDE and covariates; the second evaluates NPDE split by category for discrete covariates or quantiles for continuous covariates. We generated several validation datasets under H(0) and under alternative assumptions with a model without covariate, with one continuous covariate (weight), or one categorical covariate (sex). We calculated the powers of the different tests using simulations, where the covariates of the phase III study were used. The simulations under H(0) show a high type I error for the different tests applied to SPE and an increased type I error for pd. The different tests present a type I error close to 5% for the global test appied to NPDE. We find a type I error higher than 5% for the test applied to classical NPC but this test becomes close to 5% for NPC(dec). For covariate models, when model and validation dataset are consistent, type I error of the tests are close to 5% for both effects. When validation datasets and models are not consistent, the tests detect the correlation between NPDE and the covariate. We recommend to use NPDE over SPE for external model evaluation, since they do not depend on an approximation of the model and have good statistical properties. NPDE represent a better approach than NPC, since in order to perform tests on NPC, a decorrelation step must be applied before. NPDE, in this illustration, is also a good tool to evaluate model with or without covariates.

https://www.hal.inserm.fr/inserm-00448559/document

Evaluation of different tests based on observations for external model evaluation of population analyses

Model evaluation is an important part of model building, and has been the subject of regulatory guidelines in drug development. In the present paper, we illustrate the use of some recently proposed metrics on several simulated datasets. These metrics include Visual Predictive Checks (VPC), prediction discrepancies (pd) and normalised prediction distribution errors (npde). We illustrate them using simulated datasets. Prediction bands around selected percentiles can be obtained through repeated simulations under the model being tested, and their addition to VPC plots or plots of pd and npde versus time and predictions are useful to highlight model deficiencies. Tests for some of the metrics are also available and can be used as a complement to graphs. Résumé : L’évaluation est une partie importante de la construction de modèles, faisant l’objet de recommendations de la part des autorités régulant la mise sur le marché de nouveaux médicaments. Dans ce papier, nous effectuons une courte revue de métriques récemment proposées, en particulier les VPC (Visual Predictive Check), les discordances de prédictions (pd) et les erreurs de prédiction sur la distribution (npde). Nous illustrons ces métriques sur quelques exemples simulés. Nous montrons comment il est possible de construire des bandes de prédiction autour de la courbe des médianes (ou d’autres percentiles) des données simulées. Ces bandes de prédiction sont un outil visuel particulièrement efficace pour détecter des zones où le modèle peut être amélioré. La distribution de certaines métriques est connue et permet de proposer des tests pour compléter les graphes diagnostiques.

http://journal-sfds.fr/article/download/45/36

Karl Brendel

Papers

Computing normalised prediction distribution errors to evaluate nonlinear mixed-effect models: The npde add-on package for R

Metrics for External Model Evaluation with an Application to the Population Pharmacokinetics of Gliclazide

Overview of model-building strategies in population PK/PD analyses: 2002-2004 literature survey.

Evaluation of different tests based on observations for external model evaluation of population analyses

Model evaluation in nonlinear mixed effect models, with applications to pharmacokinetics