Development and validation of a surgical-pathologic staging and scoring system for cervical cancer

INTRODUCTION

Cervical cancer is the third leading cause of cancer-related mortality among women worldwide;[1] it results in approximately 275,000 deaths annually.[2] The International Federation of Gynecology and Obstetrics (FIGO) staging system is a widely accepted staging method for cervical cancers.[3, 4] The FIGO system is based solely on clinical examination, and once the clinical stage has been designated it cannot be changed, even if exact surgical or pathologic evidences were confirmed during or after surgery. Therefore, it is inherently inaccurate if patients accompanied by pelvic inflammatory disease, endometriosis, or obesity. Patients within each FIGO stage have markedly different outcomes due to significant discrepancies between clinically determined stage and surgical pathologic findings.[5-7]

According to the FIGO committee, surgical-pathological staging of cervical cancer is problematic in low income countries due to high incidence rates and lower surgical rates,[4] especially in Africa, where the majority of patients are diagnosed in advanced stages and surgical facilities and surgeons are extremely scarce. However, according to a World Health Organization report in 2012, 53.9% of cases worldwide occurred in Asia and 21.2% in Europe and America areas, whereas Africa accounts for only 18.8%.[8] The ten countries with the most cases worldwide were in Northern Asia, Eastern Europe, and America, where incidence rates ranged from 9.4/100,000 to 23.7/100,000, including India, China, Russian Federation, United States, and Japan.[8] Whereas the ten countries with the highest incidence rates (39.9/100,000-56.3/100,000) were mainly in Africa, they account for only a small percentage (3.2%) of cervical cancer cases worldwide (Supplementary Tables 1 and 2).[8] With rapid economic development in Asia, cervical cancer incidence trends and treatment patterns have fundamentally changed.[9] We recently published a 10-year investigation of 10,012 cervical cancer cases in China. [10-13] The vast majority of patients (83.9%) were treated with surgery. Similarly, the majority of cervical cancers are surgically treated in Asia, Europe, and America areas.[14, 15] Therefore, a surgical-pathologic staging system for cervical cancer is appropriate for worldwide implementation and should be proposed as soon as possible.

There is precedent for use of a more practical scoring system in staging of gynecological malignancies.[16-20] For example, patients with malignant trophoblastic tumors simultaneously have several prognostic risk factors and may result in the different outcomes. The introduction of a rational staging system for scoring of malignant trophoblastic tumor has resulted in improved prognostic accuracy to the patients.[21, 22]A surgical-pathologic scoring system (SPSs) can provide important clues to assist clinicians in developing more precise and individualized treatment schemes.

In this study, 4,220 eligible cases were extracted for screening the surgical-pathologic risk factors. The preliminary surgical-pathologic stages and SPSs for cervical cancer were assigned, and a prospective validation was performed to verify the reliability and practical applicability of the new system.

RESULTS

Univariate and multivariate analysis of risk factors

The univariate analysis of potential risk factors is shown in Table 1. Eight factors were found to be associated with OS and DFS rates, including lymph node metastasis (LNM) (OS,P=1.49E-32; DFS,P=2.29E-37), parametrial involvement (OS,P=4.23E-18; DFS,P=1.09E-22), histological type (OS,P=3.51E-16; DFS,P=4.13E-12), tumor size > 4cm (OS,P=5.31E-6; DFS,P=1.87E-6), grade2-3(OS,P=0.002; DFS,P=2.62E-4), stromal invasion (OS,P=1.10E-10; DFS,P=2.47E-13), lymph-vascular space invasion (LVSI) (OS,P=1.07E-6; DFS,P=4.64E-10), and vaginal involvement (OS,P=0.003; DFS,P=7.27E-5). Corpus uteri was not associated with the long-term outcomes in this study (OS,P=0.435; DFS,P=0.130). The other three general factors of age, parity, and tumor family history did not exhibit any significant effect on patient prognosis and were therefore discarded from further analysis.

Next, the multivariate analysis identified six independent factors associated with both 5-year OS and DFS rates and they were ranked based on the magnitude of their effect as follows: (1) LNM; (2) parametrial involvement; (3) histological type; (4) tumor size > 4cm; (5) grade; and (6) stromal invasion (Table 1). In addition to these factors, LVSI was found to be a risk factor associated only with 5-year DFS rate. In this study, vaginal involvement was not found to affect prognosis.

Effect of lymph node metastasis on prognosis

Among all of the prognostic parameters in our study, LNM was the strongest determinant of patients’ survival. Both 5-year OS and DFS rates were decreased significantly in patients with LNM in comparison to patients without LNM (Supplementary Figures 1A and 1B; OS, P=1.64E-32; DFS, P=2.82E-37) , indicating the positive nodes was defining prognostic factor for the long-term outcomes. Moreover, the 5-year OS and DFS rates decreased concomitant with an increasing number of positive nodes (Supplementary Figures 1C and 1D; OS, P=3.76E-11; DFS, P=1.96E-9; Supplementary Tables 4 and 5). Additionally, when para-aortic lymph nodes metastasis (pLNM) were detected, the 5-year outcomes in these patients were much poor than those of patients without pLNM (Supplementary Figures 1E and 1F; OS, P=4.85E-6; DFS, P=4.66E-5; Supplementary Table 5).

Establishment of the preliminary surgical-pathologic stages

Based on the analyses above, we created preliminary surgical-pathologic stages for cervical cancer. The goal was to put risk factors into surgical-pathologic stages in order according to their severity and prognostic impact. As shown in Table 2, the basic FIGO framework was retained. In stage I, when the lesions exceeded the criteria defining stage A, but measured tumor diameter was ≤4 cm in size, we divided FIGO IB1 into two surgical-pathologic stages, IB1 (no LVSI and invasion of stroma ≤1/2) and IB2 (positive LVSI and/or invasion of stroma >1/2). The FIGO IB2 criteria were used to define surgical-pathologic stage IB3. In stage II, surgical-pathologic stage IIC was added to include cases in which LNM was identified by pathologists, and the classification was further subdivided, based on the number of positive nodes, into IIC1(cases with 1 or2 LNM) and IIC2 (cases with ≥3 LNM). Similarly, patients with pLNM were classified as surgical-pathologic stage IIC3 (Table 2, Supplementary Tables 4 and 5). Corpus uteri and vaginal involvement were found not to be associated with 5-year OS and DFS rates in this study, and were consequently excluded from the list of criteria used to define surgical-pathologic stage.

Construction and validation of spss

In surgical-pathologic staging, two important parameters, histological type and grade, were not covered. To provide a more accurate and convenient system for evaluation of patients’ prognosis, the SPSs was proposed. Regression coefficients (β) yielded a statistical weight for the contribution of each factor (Table 3). The total score was generated by superposition when a single patient had more than one risk factor. We set up a rank of optimal cutoff points (Table 4): SPSs A (2 and 4, score 0-1; 2-3; ≥4), SPSs B (3 and 4, score 0-2; 3; ≥4), SPSs C (1 and 4, score 0; 1-3; ≥4), and SPSs D (1 and 5, score 0; 1-4; ≥5). The SPSs C system exhibited the best monotonicity of gradient, based on LRχ2 (highest homogeneity, 192.0) and liner trendχ2 (highest discriminatory score at 8 years, 28.2). The Akaike information criterion was the lowest if choosing SPSs C (Table 4).

Based on serial inference and optimization, the patients could be stratified by SPSs C array into three risk categories: zero risk, low risk, and high risk. To further validate the reliability of SPSs, all patients in Cohort 1 were classified into three score groups based on the sum of their risk factors. The 5-year OS and DFS outcomes in these three groups showed significant decreases associated with increased scores, and differences among three groups were found to be significant (Figure 1A and 1B, P=1.08E-45 and P=6.15E-55; Supplementary Table 6). Next, we performed external validation. To reach this purpose, a total of 1,104 cases in Cohort 2 were enrolled, from 2009 to 2013, in a prospective study. The results from this Cohort 2 study completely confirmed the results of the Cohort 1 study. (Figure 1C and 1D; P=9.04E-15 and P=3.23E-16; Supplementary Table 6).

Comparison of figo staging, surgical-pathologic staging, and spss

In this study, we compared FIGO staging, surgical-pathologic staging, and SPSs (SPSs C: zero, low, and high scores) by their monotonicity and discriminatory ability. The best monotonicity and discrimination was provided by SPSs, followed by surgical-pathologic staging and then FIGO staging (Table 4, LRχ2: 192.0, 152.4, and 54.2 respectively; linear trendχ2 at 8 year: 28.2, 15.7, and 7.3, respectively), SPSs was also showed the lowest Akaike information criterion among three systems (Table 4, AIC: 12101.4, 12153.3 and 12238.8, respectively), indicating that SPSs was the most informative system with respect to explaining or predicting the prognosis of patients with cervical cancer.

DISCUSSION

The Cohort 1 study included multicenter samples. Whereas the majority of parameters we concerned have also been considered in previous studies, [23-25] However, our study is the first to provide definitive, statistically significant findings based on a large cohort. External validation was performed with data from Cohort 2, a high-quality single center data set. The use of patient data from China in the Asia-Pacific region is particularly relevant and informative considering both the large population size and the high prevalence of cervical cancer, which makes this region the site of the most cases of cervical cancer worldwide (Supplementary Table 2).

Among the seven cervical cancer risk factors examined, LNM was found to be the strongest prognostic factor; this association has been extensively documented, [26-28] but lymph node status is still not considered within the FIGO staging system. Our present statistical analysis provides new insight into the relevance of LNM to long-term outcomes. Here we provide compelling evidence that LNM should be introduced into the surgical-pathologic staging system, and propose sub-classification of surgical-pathologic stages IIC1, IIC2, and IIC3 based on the number of positive nodes and presence or absence of pLNM (Table 2). A survey of pLNM is listed in Supplementary Table 7. Parametrial involvement, tumor size, and stromal invasion were found to be additional key risk factors for prognosis. All of the parameters were re-staged based on their impact values. LVSI was found to be a marginal independent risk factor and was therefore also included. In contrast, corpus uteri and vaginal involvement were not found to be associated with long-term outcomes. We presume that this is due to the ability to completely surgically excise both types of lesion, such that they probably lose any long-term effects on prognosis after surgical removal. However, there is no consensus on this view [28-31]. Therefore, we have not included these two parameters in the new surgical-pathologic staging system to date.

We noticed that two crucial surgical-pathologic parameters, histological types and grades, were not utilized in defining the surgical-pathologic stages. To provide a more reliable and convenient system to predict patient outcome, we propose a new scoring system for cervical cancer. There are several successful examples of scoring systems in gynecologic practice, such as those used for gestational trophoblastic neoplasms and endometriosis. [17, 18, 32, 33] In our system, the score value was compiled by statistical weight of each individual factor, and then optimized SPSs risk categories were proposed in order to introduce a logical standard for the classification of cervical cancer and for use in setting guidelines for post-operative treatment.  External-validation as well as homogeneity and discriminatory analysis indicate that SPSs has the most value of all of the three systems with respect to prognostic interpretation in cervical cancer.

In conclusion, the prognostic parameters used in our study are derived from both surgically-derived data and pathologic examination, which most objectively reflect the patient’s real condition. The surgical-pathologic staging and SPSs presented here may enable clinicians to more precisely predict the prognosis of patients and to standardize post-surgical therapeutic recommendations for defined sets of cervical cancer patients (e.g. non treatment for zero-score group; adjuvant external radiotherapy for low-score group; and concurrent chemoradiotherapy for high-score group), ultimately improving patient survival.

PATIENTS AND METHODS

Patients

The study was based on two cohorts. Cohort 1 comprised 4,220 eligible patients of 10,087 inpatient cases during 2002 to 2008, extracted from the Cervical Cancer Database v1.10 (http://clinicaltrials.gov, Supplementary Figure 2), which has been described previously. [11, 12] Cohort 2 comprised 1,104 inpatient cases enrolled in a prospective study at Tongji Hospital from 2009 to 2013. All cases in both cohorts were FIGO stage IA-III cervical cancer patients. All were treated surgically. Baseline of patient characteristics is summarized in Supplementary Table 3. Cohort 1 was used as the training set to screen surgical-pathologic risk factors for establishment of surgical-pathologic stages and development of the SPSs, and Cohort 2 was used for external validation of the SPSs. Pathological diagnoses were confirmed by two pathologists in both cohorts. Patients who met any of the following criteria were excluded: over 70 years of age, other serious complicating disease, or prior malignant disease. There was no overlapping of patients between the two cohorts. The protocol was approved by the Ethics Committee of Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, P. R. China. All patients in Cohort 2 provided written informed consent.

Clinical assessment

Previous reports describe the details of the Cohort 1data sources and the methods used for this analysis.[11, 12 ] Briefly, data were collected from patient records by trained gynecological oncology staff using standardized data collection and quality control procedures. The collected data included demographics; clinical and pathologic tumor characteristics; treatment regimens; physical, gynecologic, cytological, and pathological observations from patient examinations; and outcomes.

All participants in Cohort 2 were evaluated by trained gynecological oncology staff. A structured questionnaire was collected at recruitment, covering socio-demographic characteristics, cigarette smoking, alcohol consumption, family history of cancer, and history of menopause, pregnancy, delivery, and use of oral contraceptives. At the time of enrollment and during follow-up examinations the following tests were performed to determine patient status before and after treatment: gynecological examination, liquid-based thin-layer cytology and human papilloma virus tests, ultrasonographic evidence or computer tomography/magnetic resonance imaging, and serological tests including squamous cell carcinoma antigen.

Treatment and follow-up

Generally, patients at early FIGO stages without positive pathological findings received no further treatment except for regular follow-up examinations. Patients at later FIGO stages or with pathological findings received adjuvant radiotherapy or concurrent chemoradiotherapy (CCRT) according to Radiation Therapy Oncology Group (RTOG) guidelines. Follow-up examinations were advised for all patients every three months for the first two years, every six months for the next four years, and yearly thereafter. A few patients in the study did not complete follow-up, and were excluded from survival analysis.

Statistical analysis

Statistical analyses were performed using the SPSS 13.0 software package. Categorical variables are presented as frequencies and percentages, and continuous variables are presented as means ± standard deviation (SD). P value of less than 0.05was considered to indicate statistical significance. Overall survival (OS) and disease-free survival (DFS) rates were calculated using the Kaplan-Meier method, and the log-rank test was used to compare survival curves.

The developmental model comprised three steps. Firstly, univariate and multivariate Cox proportional-hazards models were used to determine the contribution of the variables (P<0.05). The Cox proportional hazard model with stepwise approach was used to identify independent predictors of OS and DFS rates in multivariate analyses. Second, a simple risk score was devised using significant variables obtained from the stepwise multivariate analysis (P<0.05).[34] The score was defined as the weighted sum of those variables (rounded to the nearest integer). The regression coefficients retained in the clinical score model provided a statistical weight for the contribution of each factor to the overall risk of long-term outcomes. The scoring system was then adjusted by the addition of a constant across all scores to ensure that none of the values were below zero.[23, 35] Third, cutoff values were applied to categorize the scores into three groups: zero, low-, and high-score groups. We used the ordinary prognostic score in the likelihood ratio (LR) test, rather than using dummy variables. [36, 37 ]The Cox regression results were expressed using the Akaike information criterion.[38] The linear trendχ2 test was used to evaluate the discriminatory ability and the monotonicity of survival gradients.[36, 37] The 5- and 8-year rates were determined using the training cohort data, and the cutoff points providing the best homogeneity, discriminatory ability, and monotonicity of gradients were chosen for use in the final scoring system. Furthermore, a comparison between FIGO staging, surgical-pathologic staging, and the SPSs was performed. Finally, the risk-scoring model was validated by comparing the OS and DFS rates among the three risk-based subgroups in Cohort 2.

Highlights

• The current International Federation of Gynecology and Obstetrics (FIGO) staging system do not consider surgical-pathologic data.

• We propose a more comprehensive and prognostically valuable surgical-pathologic staging and scoring system (SPSs).

• Surgical-pathologic staging and SPSs show greater homogeneity and discriminatory utility than FIGO staging.

ACKNOWLEDGMENTS

We would like to thank Dan Liu, Dan Wang, Quan Mei, Cui Feng, Ming Cao, Xiuyu Pan, Caoyang Sun, Ming Yuan, Zhen Zhen, EE gong, Bo Cao, Yingying Wang, Da Zhu, Shujuan Sun, Mei Gong, Yu Qin, and Yuting Yan for their useful help.

CONFLICTS OF INTEREST

The authors declare that they have no competing interests.

GRANT SUPPORT

This study was endorsed by the Key Basic Research and Development Program Foundation of China (973 Program; No. 2015CB553903, 2009CB521806), the National High Technology and Development Program of China (863 Program, 2012AA02A507) and was supported by grants from the National Natural Science Foundation of China (NO. 91529102; 81572571; 81402160; 81302267; 81230038; 81090414; 81101964; 81472444; 81272422; 81172464; 81572725,81302267).

References

1. Arbyn M, Castellsague X, de Sanjose S, Bruni L, Saraiya M, Bray F, Ferlay J. Worldwide burden of cervical cancer in 2008. Ann Oncol. 2011;22: 2675-86.

2. Wiebe E, Denny L, Thomas G. Cancer of the cervix uteri. Int J Gynaecol Obstet. 2012;119 Suppl 2: S100-9.

3. Chen WQ, Zheng RS, Zhang SW, Li N, Zhao P, Li GL, Wu LY, He J. Report of incidence and mortality in china cancer registries, 2008. Chin J Cancer Res. 2012;24: 171-80.

4. Oncology FCoG. FIGO staging for carcinoma of the vulva, cervix, and corpus uteri. Int J Gynaecol Obstet. 2014;125: 97-8.

5. Noh JM, Park W, Kim YS, Kim JY, Kim HJ, Kim J, Kim JH, Yoon MS, Choi JH, Yoon WS, Kim JY, Huh SJ. Comparison of clinical outcomes of adenocarcinoma and adenosquamous carcinoma in uterine cervical cancer patients receiving surgical resection followed by radiotherapy: a multicenter retrospective study (KROG 13-10). Gynecol Oncol. 2014;132: 618-23.

6. Rossi PJ, Horowitz IR, Johnstone PA, Jani AB. Lymphadenectomy for patients with cervical cancer: is it of value? J Surg Oncol. 2009;100: 404-6.

7. Kim HS, Kim JH, Chung HH, Kim HJ, Kim YB, Kim JW, Park NH, Song YS, Kang SB. Significance of numbers of metastatic and removed lymph nodes in FIGO stage IB1 to IIA cervical cancer: Primary surgical treatment versus neoadjuvant chemotherapy before surgery. Gynecol Oncol. 2011;121: 551-7.

8. Sweileh WM, Zyoud SH, Al-Jabi SW, Sawalha AF. Public, environmental, and occupational health research activity in Arab countries: bibliometric, citation, and collaboration analysis. Arch Public Health. 2015;73: 1.

9. Lertkhachonsuk AA, Yip CH, Khuhaprema T, Chen DS, Plummer M, Jee SH, Toi M, Wilailak S, Asian Oncology S. Cancer prevention in Asia: resource-stratified guidelines from the Asian Oncology Summit 2013. Lancet Oncol. 2013;14: e497-507.

10. Hu T, Li S, Chen Y, Shen J, Li X, Huang K, Yang R, Wu L, Chen Z, Jia Y, Wang S, Cheng X, Han X, et al. Matched-case comparison of neoadjuvant chemotherapy in patients with FIGO stage IB1-IIB cervical cancer to establish selection criteria. Eur J Cancer. 2012;48: 2353-60.

11. Li S, Hu T, Chen Y, Zhou H, Li X, Cheng X, Yang R, Wang S, Xie X, Ma D. Adjuvant chemotherapy, a valuable alternative option in selected patients with cervical cancer. PLoS One. 2013;8: e73837.

12. Hu T, Wu L, Xing H, Yang R, Li X, Huang K, Jia Y, Zhang Q, Chen Z, Wang S, Liu D, Han X, Lin Z, et al. Development of criteria for ovarian preservation in cervical cancer patients treated with radical surgery with or without neoadjuvant chemotherapy: a multicenter retrospective study and meta-analysis. Ann Surg Oncol. 2013;20: 881-90.

13. Li S, Hu T, Lv W, Zhou H, Li X, Yang R, Jia Y, Huang K, Chen Z, Wang S, Tang F, Zhang Q, Shen J, et al. Changes in prevalence and clinical characteristics of cervical cancer in the People’s Republic of China: a study of 10,012 cases from a nationwide working group. Oncologist. 2013;18: 1101-7.

14. Fleming S, Schluterman NH, Tracy JK, Temkin SM. Black and white women in Maryland receive different treatment for cervical cancer. PLoS One. 2014;9: e104344.

15. Roque DR, Wysham WZ, Soper JT. The surgical management of cervical cancer: an overview and literature review. Obstet Gynecol Surv. 2014;69: 426-41.

16. Ronco DA, Manahan KJ, Geisler JP. Ovarian cancer risk assessment: a tool for preoperative assessment. Eur J Obstet Gynecol Reprod Biol. 2011;158: 325-9.

17. Tocharoenvanich S, Chichareon S, Wootipoom V, Bu-ngachat R, Piyananjarassri K. Correlation of risk categorization in gestational trophoblastic tumor between old and new combined staging and scoring system. J Obstet Gynaecol Res. 2003;29: 20-7.

18. Kohorn EI. Dynamic staging and risk factor scoring for gestational trophoblastic disease. Int J Gynecol Cancer. 2007;17: 1124-30.

19. Bendifallah S, Canlorbe G, Huguet F, Coutant C, Hudry D, Graesslin O, Raimond E, Touboul C, Collinet P, Bleu G, Darai E, Ballester M. A risk scoring system to determine recurrence in early-stage type 1 endometrial cancer: a French multicentre study. Ann Surg Oncol. 2014;21: 4239-45.

20. Todo Y, Okamoto K, Hayashi M, Minobe S, Nomura E, Hareyama H, Takeda M, Ebina Y, Watari H, Sakuragi N. A validation study of a scoring system to estimate the risk of lymph node metastasis for patients with endometrial cancer for tailoring the indication of lymphadenectomy. Gynecol Oncol. 2007;104: 623-8.

21. Kohorn EI. The new FIGO 2000 staging and risk factor scoring system for gestational trophoblastic disease: description and critical assessment. Int J Gynecol Cancer. 2001;11: 73-7.

22. Kohorn EI. Negotiating a staging and risk factor scoring system for gestational trophoblastic neoplasia. A progress report. J Reprod Med. 2002;47: 445-50.

23. Biewenga P, van der Velden J, Mol BW, Stalpers LJ, Schilthuis MS, van der Steeg JW, Burger MP, Buist MR. Prognostic model for survival in patients with early stage cervical cancer. Cancer. 2011;117: 768-76.

24. Yang L, Jia X, Li N, Chen C, Liu Y, Wang H. Comprehensive clinic-pathological characteristics of cervical cancer in southwestern China and the clinical significance of histological type and lymph node metastases in young patients. PLoS One. 2013;8: e75849.

25. Mabuchi S, Okazawa M, Matsuo K, Kawano M, Suzuki O, Miyatake T, Enomoto T, Kamiura S, Ogawa K, Kimura T. Impact of histological subtype on survival of patients with surgically-treated stage IA2-IIB cervical cancer: adenocarcinoma versus squamous cell carcinoma. Gynecol Oncol. 2012;127: 114-20.

26. Rob L, Robova H, Halaska MJ, Hruda M, Skapa P. Current status of sentinel lymph node mapping in the management of cervical cancer. Expert Rev Anticancer Ther. 2013;13: 861-70.

27. Gien LT, Covens A. Lymph node assessment in cervical cancer: prognostic and therapeutic implications. J Surg Oncol. 2009;99: 242-7.

28. Kim MJ, Chung HH, Kim JW, Park NH, Song YS, Kang SB. Uterine corpus involvement as well as histologic type is an independent predictor of ovarian metastasis in uterine cervical cancer. J Gynecol Oncol. 2008;19: 181-4.

29. Narayan K, Fisher RJ, Bernshaw D, Shakher R, Hicks RJ. Patterns of failure and prognostic factor analyses in locally advanced cervical cancer patients staged by positron emission tomography and treated with curative intent. Int J Gynecol Cancer. 2009;19: 912-8.

30. Sethasathien S, Charoenkwan K, Settakorn J, Srisomboon J. Predicting factors for positive vaginal surgical margin following radical hysterectomy for stage IB1 carcinoma of the cervix. Asian Pac J Cancer Prev. 2014;15: 2211-5.

31. Kodama J, Seki N, Nakamura K, Hongo A, Hiramatsu Y. Prognostic factors in pathologic parametrium-positive patients with stage IB-IIB cervical cancer treated by radical surgery and adjuvant therapy. Gynecol Oncol. 2007;105: 757-61.

32. Wang W, Li R, Fang T, Huang L, Ouyang N, Wang L, Zhang Q, Yang D. Endometriosis fertility index score maybe more accurate for predicting the outcomes of in vitro fertilisation than r-AFS classification in women with endometriosis. Reprod Biol Endocrinol. 2013;11: 112-21.

33. Adamson GD. Endometriosis Fertility Index: is it better than the present staging systems? Curr Opin Obstet Gynecol. 2013;25: 186-92.

34. Dranitsaris G, Clemons M, Verma S, Lau C, Vincent M. Chemotherapy-induced anaemia during adjuvant treatment for breast cancer: development of a prediction model. Lancet Oncol. 2005;6: 856-63.

35. Regierer AC, Wolters R, Ufen MP, Weigel A, Novopashenny I, Kohne CH, Samonigg H, Eucker J, Possinger K, Wischnewsky MB. An internally and externally validated prognostic score for metastatic breast cancer: analysis of 2269 patients. Ann Oncol. 2014;25: 633-8.

36. Hosmer DW, Hosmer T, Le Cessie S, Lemeshow S. A comparison of goodness-of-fit tests for the logistic regression model. Stat Med. 1997;16: 965-80.

37. Ueno S, Tanabe G, Sako K, Hiwaki T, Hokotate H, Fukukura Y, Baba Y, Imamura Y, Aikou T. Discrimination value of the new western prognostic system (CLIP score) for hepatocellular carcinoma in 662 Japanese patients. Cancer of the Liver Italian Program. Hepatology. 2001;34: 529-34.

38. Forster MR. Key Concepts in Model Selection: Performance and Generalizability. J Math Psychol. 2000;44: 205-31.