Abstract
Jing Zhang1,*, Sunfu Zhang1,*, Yanlin Song2,*, Min He1, Qingqing Ren1, Chaoyue Chen1, Zhiyong Liu1, Yunhui Zeng1 and Jianguo Xu1
1Department of Neurosurgery, West China Hospital of Sichuan University, The First People’s Hospital of Yibin, Sichuan, PR China
2West China School of Medicine, West China Hospital of Sichuan University, Sichuan, PR China
*These authors have contributed equally to this work
Correspondence to:
Jianguo Xu, email: [email protected]
Keywords: neutrophil lymphocyte ratio, glioma, survival, prognosis
Received: April 27, 2017 Accepted: June 20, 2017 Published: July 22, 2017
ABSTRACT
The purpose of this study was to evaluate the prognostic role of neutrophil lymphocyte ratio (NLR) in patients with glioma. PubMed, EMBASE, Cochrane Library and China National Knowledge Infrastructure were searched for relevant literature. The study and patient characteristics were extracted. Hazard ratios (HRs) with 95% confidence intervals (CIs) were pooled to estimate the prognostic role of NLR in patients with glioma. Subgroup analysis and sensitivity analysis were also performed. Six studies with 1,021 patients were included. The pooled HR of elevated NLR for OS in patients with glioma was 1.48 (95% CI, 1.25-1.76). Among the included studies, five studies used 4 as the cut-off value of NLR. The pooled HR for OS of the five studies was 1.67 (95% CI, 1.37-2.03). No significant heterogeneity was observed (I2 = 42.4%, P=0. 122). Publication bias was not present. Elevated NLR was associated with poorer overall survival in patients with glioma.
INTRODUCTION
Gliomas are the most common type of primary intracranial tumors [1]. Glioblastoma, classified as grade IV glioma by the World Health Organization (WHO) [2], is a common type of glioma and account for 80% of all primary malignant central nervous system tumors [3, 4]. With the standard therapies for glioblastoma, including maximal safe resection, temozolomide chemotherapy and radiation, the median overall survival (OS) for patients with newly diagnosed glioblastoma is still only 12–18 months [3, 5]. Prognostic factors for glioma mainly include Karnofsky Performance Status scale at diagnosis, age, histology and molecular makers [2, 3, 6].
It has been demonstrated that chronic inflammation plays an important role in tumor development and progression [7-9]. And inflammation status is closely related to the pathogenesis of glioma [10]. In this context, many inflammatory markers were proposed to predict cancer outcome, such as peripheral blood neutrophil count, lymphocyte count, neutrophil lymphocyte ratio (NLR), platelet lymphocyte ratio (PLR), lymphocyte monocyte ratio (LMR) and so on [11-15]. Among them, NLR was the most studied parameter. And NLR was found to have a prognostic role in various tumors, including lung cancer, breast cancer, gastrointestinal cancers, urologic cancers, gynecologic cancers and metastatic disease, with 4 as the widely used cut-off value [16]. In recent years, several studies investigating the prognostic role of NLR in glioma were published [6, 17-19]. Most of them proved that higher NLR predicted worse outcome in glioma, however, Mason et al. suggested that NLR could not be an independent predictor of OS in glioma [17].
Therefore, due to the controversy, we aimed to perform a meta-analysis to systematically evaluate the prognostic role of NLR in patients with glioma.
RESULTS
Literature research
A total of 64 studies were retrieved from the initial search. After removing duplicates, 52 studies were screened. After reviewing the titles and abstracts, 42 articles were excluded according to the predefined criteria. The rest 10 studies were assessed in full text. One paper was a letter [20] and was excluded. Two studies [17, 21] were performed in the same institution by the same research team, and the study with larger sample and adjusted HR was included [17]. Two studies reported no enough data to estimate the prognostic role of NLR. Eventually, six articles [6, 17-19, 22, 23] were included. The study selection process was shown in Figure 1.
Figure 1: Selection process of studies.
Study characteristics
The main characteristics of the 6 included studies were shown in Table 1. All of them were published in recent years and were from 5 different countries. A total of 1,021 patients were included (range from 72 to 369). Four studies examined glioblastoma, and the rest two studies investigated glioma in different grades (1 with grade II-IV and 1 with grade I-IV). Four studies used NLR calculated from the preoperative blood sample, while the NLRs in two studies were from postoperative blood sample. Most studies used 4 as the cut-off value of NLR and one study used 7.5. Only one study did not perform multivariate cox regression analysis and reported unadjusted HR.
Table 1: Characteristics of the included studies
Author | Year | Country | N (F/M) | Age | Disease | Sampling time | NLR cut-off value | Multivariate HR |
|---|---|---|---|---|---|---|---|---|
Wiencke | 2017 | USA | 72 (20/52) | median 47 | glioma (grade II-IV) | postoperative | 4 | yes |
Wang | 2017 | China | 166 (70/96) | mean 52.1 | glioblastoma | preoperative | 4 | yes |
Mason | 2017 | Canada | 369 (131/238) | median 55 | glioblastoma | postoperative | 7.5 | yes |
Auezova | 2016 | Kazakhstan | 178 (85/93) | median 41, mean 41.58 | glioma (grade I-IV) | preoperative | 4 | no |
Han | 2015 | China | 152 (57/95) | mean 50.4 | glioblastoma | preoperative | 4 | yes |
Bambury | 2013 | Ireland | 84 (19/65) | median 58 | glioblastoma | preoperative | 4 | yes |
N (F/M): number of patients (female/male), NLR: neutrophil lymphocyte ratio, HR: hazard ratio.
Overall analysis
The pooled HR for OS of the six studies was 1.48 (95% CI, 1.25-1.76) (Figure 2), suggesting that patients with higher NLR had a worse overall outcome. The between-study heterogeneity was not significant (I2=42.4%, P=0.122), and fixed effect model was used. After performing sensitivity analysis, the study by Mason et al. [17] was found to greatly contribute to the heterogeneity. After excluding this study, the heterogeneity turned sharply to 0% (P=0.600) and the pooled HR remained statistically significant (1.67, 95% CI 1.37-2.03).
Figure 2: Pooled hazard ratio (HR) of elevated neutrophil lymphocyte ratio (NLR) for overall survival (OS) in patients with glioma.
Subgroup analysis
We grouped the studies into Asian group and non-Asian group according to the study region. The pooled HR for OS in Asian studies was 1.59 (95% CI, 1.27-2.00), while the pooled HR was 1.47 (95% CI, 0.92-2.36) in non-Asian group. The pooled HR for OS in glioblastoma was 1.55 (95% CI, 1.08-2.22), and the pooled HR was 1.50 (95% CI, 1.14-1.97) for the two studies examining various grades of glioma. As to the four studies using blood samples before surgery, the pooled HR for OS was 1.63 (95% CI, 1.32-2.01). However, the pooled HR was 1.36 (95% CI, 0.69-2.70) for the two studies using blood samples after surgery. In the five studies using the cut-off value of 4 for NLR, the pooled HR for OS was 1.67 (95% CI, 1.37-2.03). In the five studies that reported HRs from multivariate analyses, the pooled HR for OS was 1.61 (95% CI, 1.18-2.20).
All the pooled results above were shown in Table 2.
Table 2: Summary of meta-analysis results
Groups | N | Model | Pooled HR (95% CI) | p value | Heterogeneity (I2, P) | Conclusion |
|---|---|---|---|---|---|---|
Total | 6 | fixed | 1.48 (1.25-1.76) | <0.001 | 42.4%, 0.122 | positive |
Asian | 3 | fixed | 1.59 (1.27-2.00) | <0.001 | 5.4%, 0.348 | positive |
Non-Asian | 3 | random | 1.47 (0.92-2.36) | 0.106 | 64.6%, 0.059 | negative |
Glioblastoma | 4 | random | 1.55 (1.08-2.22) | 0.017 | 59.8%, 0.058 | positive |
Glioma (various grades) | 2 | fixed | 1.50 (1.14-1.97) | 0.004 | 17.0%, 0.272 | positive |
Preoperative blood | 4 | fixed | 1.63 (1.32-2.01) | <0.001 | 0.0%, 0.510 | positive |
Postoperative blood | 2 | random | 1.36 (0.69-2.70) | 0.376 | 74.2%, 0.049 | negative |
Cut-off value = 4 | 5 | fixed | 1.67 (1.37-2.03) | <0.001 | 0.0%, 0.600 | positive |
Cut-off value = 7.5 | 1 | — | 1.00 (0.70-1.44) | — | — | — |
Multivariate analysis | 5 | random | 1.61 (1.18-2.20) | 0.003 | 52.4%, 0.078 | positive |
Univariate analysis | 1 | — | 1.39 (1.02-1.88) | — | — | — |
N: number of included studies, HR: hazard ratio, CI: confidence interval.
Publication bias
No significant publication bias was found in this meta-analysis (p=0.45 for Begg’s test, p=0.12 for Egger’s test). The Begg’s plot of publication bias of the 6 included studies was shown in Figure 3.
Figure 3: The Begg’s publication bias plot of the 6 included studies.
DISCUSSION
Implications
This study aimed to evaluate the prognostic role of NLR in patients with glioma. We performed a meta-analysis to summarize the existing evidence, and to our best knowledge, this is the first meta-analysis on this topic. Our results showed that higher NLR in patients with glioma indicated poorer overall survival. And Mason et al. reported that decline in NLR during treatment could lead to better survival (HR= 0.70, 95% CI 0.58-0.88) [17]. We also did subgroup analyses to examine the prognostic role of NLR in glioma. In Asian patients, higher NLR was found to be associated with worse outcome, but the association was not significant among non-Asian participants. In the non-Asian subgroup, the heterogeneity was significant and random effect model was used to pool the results of the three studies. Thus, caution should be applied as to this finding and more studies are needed to address the difference due to the limited study number and significant heterogeneity. The pooled HRs in the glioblastoma group, in the glioma group (various grades) and of the studies using multivariate analysis were all significant, and further validated the prognostic role of NLR in glioma.
Many researchers recommended 4 as the cut-off value of NLR, and demonstrated that NLR>4 was suggestive of unfavorable outcome in gliomas [18, 19], as well as other tumors [16]. In our study, for the five studies using the cut-off value of 4, the pooled result was consistent with previous findings. Mason et al. used the cut-off value of 7.5 and their result was not significant [17]. As the authors stated, the reason was likely due to the postoperative setting in patients largely exposed to corticosteroids that affected the NLR value. Another explanation may be from the stress of surgery, which had an impact on systemic inflammation [18]. Han et al. [18] examined the prognostic roles of both preoperative NLR and postoperative NLR in glioblastoma, and concluded that postoperative NLR could not predict the clinical outcome. Our pooled results in the preoperative and postoperative subgroups accords with the findings by Han et al. Besides, some diseases, like infection, rheumatic diseases, cardiovascular diseases, or drug treatments may influence the neutrophil and lymphocyte counts [18]. Therefore, based on the present evidence, more reasonable conclusion should be that preoperative NLR>4 reflect worse overall survival in glioma patients without other confounding diseases. Future validating research should adopt preoperative blood samples and exclude patients with diseases that may affect the systemic inflammation.
The underlying mechanism of such an association remains unclear. Many researchers suggested that the tumor microenvironment was altered in the state of chronic inflammation [17, 18, 23]. Han et al. identified that elevated preoperative NLR was associated with high neutrophil infiltration and low CD3+ T-cell infiltration into glioblastoma [18]. Tumor associated neutrophilia was reported to be associated with worse outcomes in metastatic cancer [24, 25]. As to glioma, Fossati et al. found a marked and significant correlation between tumour grade and the extent of the neutrophil infiltration, and proposed glioma-derived factors might affect the number of circulating neutrophils and influence their infiltration into tumors [26]. Reduced cell mediated immunity due to lowered lymphocyte may also contribute to this phenomenon [19]. It was proven that elevated tumor infiltrating lymphocytes correlated with a better survival glioblastoma [27]. Future work correlating peripheral blood parameters and immune cell infiltration in the tumor microenvironment may elucidate the prognostic role of NLR in glioma.
The between-study heterogeneity was acceptable (I2=42.4%) in our meta-analysis. Sensitivity analysis was performed and identified the study by Mason et al. [17] to be the major contributor to the heterogeneity. After excluding this study, the heterogeneity became 0% and the pooled HR remained statistically significant. The reason why this study greatly contributed to heterogeneity might be that it was the only study that did not use the cut-off value of 4, and the authors adopted postoperative blood samples to calculate the NLR. Other potential sources of heterogeneity may be from different patient sources and glioma grades.
Limitations
There are several limitations in our study. Firstly, this meta-analysis was based on a limited number of studies and the number of studies in each subgroup was even smaller. Therefore, the subgroup results need to be interpreted with caution. Secondly, some basic characteristics were quite different between the studies. For example, the glioma grade, blood sampling time, cut-off value of NLR and adjustment of HR were not coherent. Although no significant heterogeneity was present, more well-designed studies were needed to validate the results. Besides, although no significant publication bias was found in this meta-analysis, publication bias was a major concern for all meta-analyses and could not be completely excluded.
MATERIALS AND METHODS
Search strategy
We followed the developed guidelines for meta-analyses in performing our study [28]. PubMed, EMBASE, Cochrane Library and China National Knowledge Infrastructure (CNKI) were searched for potentially relevant literature (last search ran on Apr 22ed, 2017). The following keywords were used: ‘neutrophil lymphocyte ratio’ AND ‘glioma’. Reference lists of selected articles were also screened for additional studies. No language restriction was used.
Study selection
The study selection process was performed by two reviewers (J.Z. and Y.S.) independently, with any disagreements being discussed. Studies were included according to the following inclusion criteria: (1) the patients were diagnosed with glioma in any grade and received standard treatments; (2) either preoperative or postoperative NLR was assessed in patients; (3) patients were followed up for enough time; (4) enough data were reported to estimate the prognostic role of NLR in patients with glioma. Reviews, case reports, conference abstracts, letters, unrelated articles, and studies without enough data were excluded.
Data extraction
Two independent researchers (J.Z. and Y.S.) extracted relevant data from the included studies and disagreements were resolved by consensus. The primary data was hazard ratio (HR) with 95% confidence interval (CI), or Kaplan-Meier survival curves with p values. HR using multivariate analysis was extracted if univariate and multivariate survival analyses were both provided. The study and patients characteristics included first author, publication year, country, number of patients, median or mean age of patients, grade of glioma, sampling time of the blood and the cut-off value of NLR.
Statistical analysis
The logHR and variance were calculated according to the methods developed by Parmar, Williamson and Tierney [29-31], and were then used for aggregation of the prognostic role of NLR in glioma. Forest plots were used to estimate the pooled HR. The pooled HR was considered significant if the 95% CI did not overlap 1 and the p value was less than 0.05. Subgroup analyses were performed based on patient source, glioma grade, blood sampling time, cut-off value of NLR and adjustment of HR. The between-study heterogeneity was assessed, with P<0.10 or I2>50% indicating significant heterogeneity [32]. If heterogeneity existed, random effects models were used. Sensitivity analysis was performed to examine the contribution of each study to heterogeneity by excluding individual studies one at a time. Publication bias was assessed by Begg’s and Egger’s tests and p>0.05 was considered that there was no potential publication bias. All the statistical analyses were performed by STATA 11.0 (STATA Corporation, College Station, TX).
CONCLUSIONS
In conclusion, this study investigated the prognostic role of NLR in patients with glioma, and found that elevated NLR was associated with poorer overall survival in glioma patients. This readily accessible and cheap prognostic marker may serve as a useful tool in clinical work. The findings also suggest further investigation on cancer therapies based upon modulating host immune response. However, more studies are needed to corroborate these findings and address the underlying mechanisms.
ACKNOWLEDGMENTS
We would like to thank the reviewers for their constructive comments.
CONFLICTS OF INTEREST
The authors declare no conflicts of interest.
FUNDING
We received no funding.
REFERENCES
1. Dolecek TA, Propp JM, Stroup NE, Kruchko C. CBTRUS statistical report: primary brain and central nervous system tumors diagnosed in the United States in 2005-2009. Neuro Oncol. 2012; 14: v1-49. doi: 10.1093/neuonc/nos218 nos218.
2. Louis DN, Perry A, Reifenberger G, von Deimling A, Figarella-Branger D, Cavenee WK, Ohgaki H, Wiestler OD, Kleihues P, Ellison DW. The 2016 World Health Organization Classification of Tumors of the Central Nervous System: a summary. Acta Neuropathol. 2016; 131: 803-20. doi: 10.1007/s00401-016-1545-1.
3. Bush NA, Chang SM, Berger MS. Current and future strategies for treatment of glioma. Neurosurg Rev. 2017; 40: 1-14. doi: 10.1007/s10143-016-0709-8.
4. Ostrom QT, Gittleman H, Stetson L, Virk SM, Barnholtz-Sloan JS. Epidemiology of gliomas. Cancer Treat Res. 2015; 163: 1-14. doi: 10.1007/978-3-319-12048-5_1.
5. Stupp R, Mason WP, van den Bent MJ, Weller M, Fisher B, Taphoorn MJ, Belanger K, Brandes AA, Marosi C, Bogdahn U, Curschmann J, Janzer RC, Ludwin SK, et al. Radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma. N Engl J Med. 2005; 352: 987-96. doi: 10.1056/NEJMoa043330.
6. Wiencke JK, Koestler DC, Salas LA, Wiemels JL, Roy RP, Hansen HM, Rice T, McCoy LS, Bracci PM, Molinaro AM, Kelsey KT, Wrensch MR, Christensen BC. Immunomethylomic approach to explore the blood neutrophil lymphocyte ratio (NLR) in glioma survival. Clin Epigenetics. 2017; 9: 10. doi: 10.1186/s13148-017-0316-8.
7. Grivennikov SI, Greten FR, Karin M. Immunity, inflammation, and cancer. Cell. 2010; 140: 883-99. doi: 10.1016/j.cell.2010.01.025.
8. Trinchieri G. Cancer and inflammation: an old intuition with rapidly evolving new concepts. Annu Rev Immunol. 2012; 30: 677-706. doi: 10.1146/annurev-immunol-020711-075008.
9. Hasselbalch HC. Chronic inflammation as a promotor of mutagenesis in essential thrombocythemia, polycythemia vera and myelofibrosis. A human inflammation model for cancer development? Leuk Res. 2013; 37: 214-20. doi: 10.1016/j.leukres.2012.10.020.
10. Yeung YT, McDonald KL, Grewal T, Munoz L. Interleukins in glioblastoma pathophysiology: implications for therapy. Br J Pharmacol. 2013; 168: 591-606. doi: 10.1111/bph.12008.
11. Shaverdian N, Wang J, Levin-Epstein R, Schaue D, Kupelian P, Lee P, Yang I, Kaprealian T. Pro-inflammatory state portends poor outcomes with stereotactic radiosurgery for brain metastases. Anticancer Res. 2016; 36: 5333-7. doi: 10.21873/anticanres.11106.
12. Ethier JL, Desautels D, Templeton A, Shah PS, Amir E. Prognostic role of neutrophil-to-lymphocyte ratio in breast cancer: a systematic review and meta-analysis. Breast Cancer Res. 2017; 19: 2. doi: 10.1186/s13058-016-0794-1.
13. Zhou X, Du Y, Huang Z, Xu J, Qiu T, Wang J, Wang T, Zhu W, Liu P. Prognostic value of PLR in various cancers: a meta-analysis. PLoS One. 2014; 9: e101119. doi: 10.1371/journal.pone.0101119.
14. Song W, Wang K, Zhang RJ, Zou SB. Prognostic value of the lymphocyte monocyte ratio in patients with colorectal cancer: a meta-analysis. Medicine (Baltimore). 2016; 95: e5540. doi: 10.1097/md.0000000000005540.
15. Su L, Zhang M, Zhang W, Cai C, Hong J. Pretreatment hematologic markers as prognostic factors in patients with nasopharyngeal carcinoma: a systematic review and meta-analysis. Medicine (Baltimore). 2017; 96: e6364. doi: 10.1097/md.0000000000006364.
16. Templeton AJ, McNamara MG, Seruga B, Vera-Badillo FE, Aneja P, Ocana A, Leibowitz-Amit R, Sonpavde G, Knox JJ, Tran B, Tannock IF, Amir E. Prognostic role of neutrophil-to-lymphocyte ratio in solid tumors: a systematic review and meta-analysis. J Natl Cancer Inst. 2014; 106: dju124. doi: 10.1093/jnci/dju124 dju124.
17. Mason M, Maurice C, McNamara MG, Tieu MT, Lwin Z, Millar BA, Menard C, Laperriere N, Milosevic M, Atenafu EG, Mason W, Chung C. Neutrophil-lymphocyte ratio dynamics during concurrent chemo-radiotherapy for glioblastoma is an independent predictor for overall survival. J Neurooncol. 2017; 132: 463-71. doi: 10.1007/s11060-017-2395-y.
18. Han S, Liu Y, Li Q, Li Z, Hou H, Wu A. Pre-treatment neutrophil-to-lymphocyte ratio is associated with neutrophil and T-cell infiltration and predicts clinical outcome in patients with glioblastoma. BMC Cancer. 2015; 15: 617. doi: 10.1186/s12885-015-1629-7.
19. Bambury RM, Teo MY, Power DG, Yusuf A, Murray S, Battley JE, Drake C, O’Dea P, Bermingham N, Keohane C, Grossman SA, Moylan EJ, O'Reilly S. The association of pre-treatment neutrophil to lymphocyte ratio with overall survival in patients with glioblastoma multiforme. J Neurooncol. 2013; 114: 149-54. doi: 10.1007/s11060-013-1164-9.
20. Alexiou GA, Vartholomatos E, Voulgaris S. Prognostic value of neutrophil-to-lymphocyte ratio in patients with glioblastoma. J Neurooncol. 2013; 115: 521-2. doi: 10.1007/s11060-013-1240-1.
21. McNamara MG, Lwin Z, Jiang H, Templeton AJ, Zadeh G, Bernstein M, Chung C, Millar BA, Laperriere N, Mason WP. Factors impacting survival following second surgery in patients with glioblastoma in the temozolomide treatment era, incorporating neutrophil/lymphocyte ratio and time to first progression. J Neurooncol. 2014; 117: 147-52. doi: 10.1007/s11060-014-1366-9.
22. Wang PF, Song HW, Cai HQ, Kong LW, Yao K, Jiang T, Li SW, Yan CX. Preoperative inflammation markers and IDH mutation status predict glioblastoma patient survival. Oncotarget. 2017; 8:50117-50123. doi: 10.18632/oncotarget.15235.
23. Auezova R, Ryskeldiev N, Doskaliyev A, Kuanyshev Y, Zhetpisbaev B, Aldiyarova N, Ivanova N, Akshulakov S, Auezova L. Association of preoperative levels of selected blood inflammatory markers with prognosis in gliomas. Onco Targets Ther. 2016; 9: 6111-7. doi: 10.2147/ott.s113606.
24. Schmidt H, Bastholt L, Geertsen P, Christensen IJ, Larsen S, Gehl J, von der Maase H. Elevated neutrophil and monocyte counts in peripheral blood are associated with poor survival in patients with metastatic melanoma: a prognostic model. Br J Cancer. 2005; 93: 273-8. doi: 10.1038/sj.bjc.6602702.
25. Atzpodien J, Reitz M. Peripheral blood neutrophils as independent immunologic predictor of response and long-term survival upon immunotherapy in metastatic renal-cell carcinoma. Cancer Biother Radiopharm. 2008; 23: 129-34. doi: 10.1089/cbr.2007.0429.
26. Fossati G, Ricevuti G, Edwards SW, Walker C, Dalton A, Rossi ML. Neutrophil infiltration into human gliomas. Acta Neuropathol. 1999; 98: 349-54. doi: 10.1007/s004010051093.
27. Lohr J, Ratliff T, Huppertz A, Ge Y, Dictus C, Ahmadi R, Grau S, Hiraoka N, Eckstein V, Ecker RC, Korff T, von Deimling A, Unterberg A, et al. Effector T-cell infiltration positively impacts survival of glioblastoma patients and is impaired by tumor-derived TGF-β. Clin Cancer Res. 2011; 17: 4296-308. doi: 10.1158/1078-0432.CCR-10-2557.
28. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015; 4: 1. doi: 10.1186/2046-4053-4-1.
29. Parmar MK, Torri V, Stewart L. Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Stat Med. 1998; 17: 2815-34.
30. Williamson PR, Smith CT, Hutton JL, Marson AG. Aggregate data meta-analysis with time-to-event outcomes. Stat Med. 2002; 21: 3337-51. doi: 10.1002/sim.1303.
31. Tierney JF, Stewart LA, Ghersi D, Burdett S, Sydes MR. Practical methods for incorporating summary time-to-event data into meta-analysis. Trials. 2007; 8: 16. doi: 10.1186/1745-6215-8-16.
32. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003; 327: 557-60. doi: 10.1136/bmj.327.7414.557.
