Predicting acute ovarian failure in female survivors of childhood cancer: a cohort study in the Childhood Cancer Survivor Study (CCSS) and the St Jude Lifetime Cohort (SJLIFE).

TL;DR: Both acute ovarian failure risk prediction models performed well and could help clinical discussions regarding the need for fertility preservation interventions in girls and young women newly diagnosed with cancer.

Abstract: Summary Background Cancer treatment can cause gonadal impairment. Acute ovarian failure is defined as the permanent loss of ovarian function within 5 years of cancer diagnosis. We aimed to develop and validate risk prediction tools to provide accurate clinical guidance for paediatric patients with cancer. Methods In this cohort study, prediction models of acute ovarian failure risk were developed using eligible female US and Canadian participants in the Childhood Cancer Survivor Study (CCSS) cohort and validated in the St Jude Lifetime Cohort (SJLIFE) Study. 5-year survivors from the CCSS cohort were included if they were at least 18 years old at their most recent follow-up and had complete treatment exposure and adequate menstrual history (including age at menarche, current menstrual status, age at last menstruation, and menopausal aetiology) information available. Participants in the SJLIFE cohort were at least 10-year survivors. Participants were excluded from the prediction analysis if they had an ovarian hormone deficiency, had missing exposure information, or had indeterminate ovarian status. The outcome of acute ovarian failure was defined as permanent loss of ovarian function within 5 years of cancer diagnosis or no menarche after cancer treatment by the age of 18 years. Logistic regression, random forest, and support vector machines were used as candidate methods to develop the risk prediction models in the CCSS cohort. Prediction performance was evaluated internally (in the CCSS cohort) and externally (in the SJLIFE cohort) using the areas under the receiver operating characteristic curve (AUC) and the precision-recall curve (average precision [AP; average positive predictive value]). Findings Data from the CCSS cohort were collected for participants followed up between Nov 3, 1992, and Nov 25, 2016, and from the SJLIFE cohort for participants followed up between Oct 17, 2007, and April 16, 2012. Of 11 336 female CCSS participants, 5886 (51·9%) met all inclusion criteria for analysis. 1644 participants were identified from the SJLIFE cohort, of whom 875 (53·2%) were eligible for analysis. 353 (6·0%) of analysed CCSS participants and 50 (5·7%) of analysed SJLIFE participants had acute ovarian failure. The overall median follow-up for the CCSS cohort was 23·9 years (IQR 20·4–27·9), and for SJLIFE it was 23·9 years (19·0–30·0). The three candidate methods (logistic regression, random forest, and support vector machines) yielded similar results, and a prescribed dose model with abdominal and pelvic radiation doses and an ovarian dose model with ovarian radiation dosimetry using logistic regression were selected. Common predictors in both models were history of haematopoietic stem-cell transplantation, cumulative alkylating drug dose, and an interaction between age at cancer diagnosis and haematopoietic stem-cell transplant. External validation of the model in the SJLIFE cohort produced an estimated AUC of 0·94 (95% CI 0·90–0·98) and AP of 0·68 (95% CI 0·53–0·81) for the ovarian dose model, and AUC of 0·96 (0·94–0·97) and AP of 0·46 (0·34–0·61) for the prescribed dose model. Based on these models, an online risk calculator has been developed for clinical use. Interpretation Both acute ovarian failure risk prediction models performed well. The ovarian dose model is preferred if ovarian radiation dosimetry is available. The models, along with the online risk calculator, could help clinical discussions regarding the need for fertility preservation interventions in girls and young women newly diagnosed with cancer. Funding Canadian Institutes of Health Research, Women and Children's Health Research Institute, National Cancer Institute, and American Lebanese Syrian Associated Charities.

Summary

Introduction

• Due to advancements in cancer treatment and supportive care, there are almost 500,000 survivors of childhood cancer in the United States, 1 and between 300,000 and 500,000 childhood cancer survivors in Europe.
• AOF occurs when an individual permanently stops menstruating within five years of their cancer diagnosis, or fails to progress through puberty or to achieve menarche by 18 years of age following cancer treatment.
• Obtaining an accurate risk estimate is important as available fertility preservation technologies such as ovarian tissue and oocyte cryopreservation are expensive and invasive.
• The operative risk may be elevated in children who are immunocompromised or have abnormal blood counts, increasing their risk for infection or bleeding.

Study design and participants

• The CCSS, a multi-institutional longitudinal cohort study of 24,362 survivors of childhood cancer from North America, was the primary source of data.
• Originally established in 1994, its cohort characteristics, eligibility criteria, and study design features have been documented elsewhere.
• Survivors were eligible for the AOF prediction analysis if they were female, had complete treatment exposure data, were at least 18 years of age at their latest follow-up, and provided menstrual history information including age at menarche, current menstrual status, age at last menstrual period, and etiology of menopause (surgical vs. nonsurgical) for those who were currently menopausal.
• 4, 20 Participants were eligible for SJLIFE if they had been diagnosed and treated for a paediatric cancer at St. Jude Children's Research Hospital after 1962 and were at least ten-year survivors.
• Ethics approval for the study was obtained from the University of Alberta Health Research Ethics Board (PRO00067066).

Procedures

• Self-reported demographic and outcome information was obtained from CCSS participants through baseline and follow-up questionnaires, and treatment exposure information was abstracted from medical and radiation records.
• 8 Classification of ovarian status, the primary outcome, was assigned to CCSS participants using an established definition, 6 or manually by endocrinologists (SM-M, CAS) for ambiguous cases, based on responses to menstrual history questions on baseline and/or follow-up questionnaires.
• Individuals who had not experienced menarche were classified with AOF if menarche was not achieved by age 18.
• As ovarian radiation dose information is not universally available, the authors also considered a prescribed dose model where the ovarian radiation dose term was replaced with protocol-specified abdominal and pelvic radiation doses while retaining all other variables.
• Model performance was evaluated using the average predicted risk and the observed AOF status.

Statistical analysis

• The ovarian status was deemed not assessable for participants who were exposed to a pituitary radiation dose > 30 Gray (Gy), who had a tumour in the hypothalamus/pituitary region, or whose menstrual history information was incomplete, unclear, or provided by a proxy.
• Thus, these survivors were excluded from the analysis.
• The authors developed candidate prediction algorithms using three popular methods for binary outcome analysis (logistic regression, random forest, and support vector machines), which allows for examination of the sensitivity of the data to the modelling method.
• The best ovarian dose and prescribed dose models were selected from these candidate prediction algorithms and subsequently externally validated [27] [28] [29].
• The AP value can be interpreted as the AOF risk for a patient whose predicted risk is greater than the risk estimate of a randomly selected female survivor with AOF.

Of

• Table 2 and Table 3 present the results of categorising CCSS and SJLIFE patients into the predefined risk categories for the ovarian dose model and prescribed dose model, respectively.
• The ovarian dose model categorised survivors more precisely and accurately than the prescribed dose model, as seen by the larger counts in the high (≥ 50%) and low (< 5%) risk groups.
• On the other end of the predicted risk spectrum, the ovarian dose model classified 182 individuals as high risk (of whom 132 were AOF cases, 725%), while the prescribed dose model predicted only 97 individuals as high risk with 61 (629%) having developed AOF.
• A cross-table of the risk estimates from the prescribed dose and ovarian dose models based on CCSS cohort and a detailed comparison is included in the Supplementary Material (pages 2, 5).

Discussion

• With AUC values from 078 to 082 in the CCSS cohort and 094 to 096 in the external validation using the SJLIFE cohort, the authors have developed, to their knowledge, the first risk prediction models for AOF that provide a high level of confidence appropriate for use in a clinical setting.
• 8, 22, 23 Although the authors developed the models using the outcome data with a higher potential for misclassification, they observed an increase in the predictive ability in the SJLIFE cohort, which highlights the robustness of their models.
• A continuous AOF risk estimate and corresponding risk category is calculated from the ovarian dose and/or prescribed dose model depending on the radiation information provided.
• Further, it is important that cancer survivors deemed to be at high risk for AOF do not assume that they will develop ovarian failure and use appropriate contraception to prevent unplanned pregnancies and sexually transmitted infections.
• Eighty-four survivors in the CCSS cohort who were classified as non-AOF started OCP use within 5 years of their cancer diagnosis and were thus at risk for misclassification, but detailed review by the two study endocrinologists minimized this risk.

Figures

Table 2: Acute ovarian failure risk categories and cases as predicted by the best ovarian dose model

Figure 2: ROC and PR curves for the best models

Figure 1: CCSS study sample flowchart for inclusion in model development

Table 3: Acute ovarian failure risk categories and cases as predicted by the best prescribed dose model

Table 1: Diagnostic and treatment characteristics of the CCSS study cohort and the SJLIFE validation cohort

Figure 3: Calibration curves for the best models

Full text

1
Predicting Acute Ovarian Failure in Female Childhood Cancer Survivors:
A Cohort Study in the Childhood Cancer Survivor Study (CCSS) and
the St. Jude Lifetime Cohort (SJLIFE)
Authorship List Line
Rebecca A. Clark, Sogol Mostoufi-Moab, Yutaka Yasui, Ngoc Khanh Vu, Charles A. Sklar
†
,
Tarek Motan, Russell J. Brooke, Todd M. Gibson, Kevin C. Oeffinger
†
, Rebecca M. Howell, Susan
A. Smith, Zhe Lu, Leslie L. Robison, Wassim Chemaitilly, Melissa M. Hudson
†
, Gregory T.
Armstrong, Paul C. Nathan*
†
, Yan Yuan*

.
†
Full professor
* Co-senior authors

Corresponding author
List of Affiliations
School of Public Health - University of Alberta, Edmonton, Alberta, Canada. (RA Clark MSc, NK
Vu PhD, Z Lu MEng, Y Yuan PhD)
University of Pennsylvania Perelman School of Medicine, The Children’s Hospital of
Philadelphia, Philadelphia, Pennsylvania, United States of America. (S Mostoufi-Moab MD)
St. Jude Children’s Research Hospital, Memphis, Tennessee, United States of America. (Y Yasui
PhD, RJ Brooke PhD, TM Gibson PhD, LL Robison PhD, W Chemaitilly MD, MM Hudson MD,
GT Armstrong MD)
Memorial Sloan Kettering Cancer Center, New York, New York, United States of America. (CA
Sklar MD)
Manuscript

2
Faculty of Medicine and Dentistry - University of Alberta, Edmonton, Alberta, Canada. (T Motan
MB ChB)
Duke University School of Medicine, Durham, North Carolina, United States of America. (KC
Oeffinger MD)
The University of Texas MD Anderson Cancer Center, Houston, Texas, United States of America.
(RM Howell PhD, SA Smith MPH)
The Hospital for Sick Children, Toronto, Ontario, Canada. (PC Nathan MD)
Corresponding Author Information
Yan Yuan, PhD
Associate Professor
yyuan@ualberta.ca
Tel: +1 (780) 248-5853
School of Public Health, University of Alberta
3-299 Edmonton Clinic Health Academy
11405 87 Avenue
Edmonton, Alberta
T6G 1C9
Canada
Word count: 4,123

3
Summary
Background
Cancer therapy can cause gonadal impairment. Acute ovarian failure (AOF) is defined as the
permanent loss of ovarian function within five years of cancer diagnosis. We aimed to develop
and validate risk prediction tools to provide accurate clinical guidance to paediatric cancer patients.
Methods
AOF risk prediction models were developed using eligible female participants in the Childhood
Cancer Survivor Study (CCSS) cohort and validated in the St. Jude Lifetime Cohort (SJLIFE).
Eligibility criteria were at least age 18, had complete treatment exposure and adequate menstrual
history information available. Logistic regression, random forest, and support vector machines
were used as candidate methods. Prediction performance was evaluated internally and externally
using the areas under the ROC curve (AUC) and the precision-recall curve (AP). An online risk
calculator was developed for clinical use.
Findings
Three-hundred and fifty-three (6%) of 5,886 CCSS participants and 50 (5.7%) of 875 SJLIFE
participants experienced AOF. The median follow-up for the CCSS and SJLIFE analysis samples
was 23·9 (IQR=20·4-27·9) and 23·9 (19·0-30·0) years, respectively. A prescribed dose model
with abdominal and pelvic radiation doses and an ovarian dose model with ovarian radiation
dosimetry using logistic regression were selected. Common predictors in both models were history
of hematopoietic stem cell transplantation (HSCT), cumulative alkylating agent dose, and an
interaction between age at cancer diagnosis and HSCT. External validation produced an estimated

4
AUC of 0·94 (95% CI=0·90-0·98) and AP of 0·68 (95% CI=0·53-0·81) for the ovarian dose
model, and AUC of 0·96 (0·94-0·97) and AP of 0·46 (0·34-0·61) for the prescribed dose model.
Interpretations
Both AOF risk prediction models perform very well. The ovarian model is preferred if ovarian
radiation dosimetry is available. The models, along with the online risk calculator, can aid clinical
discussions regarding the need for fertility preservation interventions in young females newly
diagnosed with cancer.
Funding
Canadian Institutes for Health Research, Women and Children’s Research Institute, National
Cancer Institute, American Lebanese Syrian Associated Charities.

5
Research in Context
Evidence before this study
An increased risk of premature gonadal failure has been demonstrated in paediatric cancer
survivors treated with chemotherapy and radiation. Six percent of female childhood cancer
survivors lose ovarian function within five years of treatment (acute ovarian failure [AOF]), and
an additional nine percent experience premature, non-surgical menopause before age 40. The time
frame between primary cancer diagnosis and treatment resulting in AOF is limited to identify high-
risk patients that will benefit from interventions aimed at fertility preservation. We searched
PubMed with no date or language restrictions for all studies to evaluate the current knowledge of
AOF and the associated risk factors in childhood cancer survivors using the search terms “pediatric
cancer OR childhood cancer” AND “acute ovarian failure OR primary ovarian insufficiency”
AND “risk”. Five publications were considered for further review as they described AOF as an
independent condition without grouping patients in a broader premature menopause umbrella.
While high dose pelvic radiation, hematopoietic stem cell transplantation, and alkylating
chemotherapy have been identified as risk factors associated with AOF, clinicians lack a tool that
accurately estimates the risk of AOF for individual paediatric cancer patients at the time of cancer
diagnosis. We did not find any study that aimed to develop risk estimates of AOF for individual
paediatric cancer patients at the time of cancer diagnosis.
Added value of this study
To our knowledge, we have developed and validated the first models for predicting the risk of
AOF in female childhood cancer survivors. While physicians are aware of the gonadotoxic
treatment exposures with a high likelihood of causing AOF, there are no available prediction tools

Frequently asked questions

Q1. What are the contributions in "Predicting acute ovarian failure in female childhood cancer survivors: a cohort study in the childhood cancer survivor study (ccss) and the st. jude lifetime cohort (sjlife) authorship list line" ?

In this paper, the authors developed and validated risk prediction models for clinical use to predict primary ovarian insufficiency ( POI ) and premature menopause ( PM ) in female survivors of childhood cancer. 

Q2. What have the authors stated for future works in "Predicting acute ovarian failure in female childhood cancer survivors: a cohort study in the childhood cancer survivor study (ccss) and the st. jude lifetime cohort (sjlife) authorship list line" ?

Their goal was to develop and validate an easily accessible and user-friendly clinical tool to aid clinicians at the time of cancer diagnosis by providing personalised risk assessments of future ovarian function for patients. The outstanding performance in the external SJLIFE cohort further confirms that their prediction algorithms are generalisable. CCSS participant ovarian status was not verified clinically, and thus subject to potential misclassification. 8,22,23 Although the authors developed the models using the outcome data with a higher potential for misclassification, they observed an increase in the predictive ability in the SJLIFE cohort, which highlights the robustness of their models. 

Q3. What is the main focus of cancer survivorship research?

As the majority of childhood cancer patients will become long-term survivors, the focus of cancer survivorship research has shifted toward maximizing survivor quality of life. 

Q4. How many CCSS survivors had a HSCT?

Almost one-third (1,869 (31·8%) of 5,886) of the CCSS survivors had been diagnosed with leukaemia, and 3·7% (217 of 5,886) underwent a HSCT. 

Q5. What predictors were used in the CCSS and SJLIFE models?

Common predictors in both models were history of hematopoietic stem cell transplantation (HSCT), cumulative alkylating agent dose, and an interaction between age at cancer diagnosis and HSCT. 

Q6. How many survivors were diagnosed with AOF in the CCSS sample?

Three hundred and fifty-three of 5,886 survivors were classified with AOF in the CCSS sample, corresponding to a prevalence of 6·0%. 

Q7. What was the ovarian dose model AUC value?

The ovarian dose model AUC value was 0·94 (95% CI = 0·90-0·98), and the prescribed dose model AUC value was 0·96 (95% CI = 0·94-0·97). 

Q8. How much is the prevalence of premature menopause in CCSS female survivors?

In the general population, the prevalence of premature, non-surgical menopause is approximately 1%,11 whereas the cumulative incidence of PM (excluding AOF) reported in CCSS female survivors is 9% by age 4010. 

Q9. What was the AP value for the ovarian dose model?

For internal validation using the CCSS test sets, the AP value was 0·50 (95% CI = 0·45-0·56) for the ovarian dose model and 0·37 (95% CI = 0·32-0·43) for the prescribed dose model. 

Q10. What is the affiliation between MD Anderson and the St Jude Hospital Research Center?

MD Anderson Late Effects Group (RMH and SAS) has a subcontract with St Jude Hospital Research Center for CCSS dosimetry and also a contract from REB/NCI to perform dosimetry on various studies.