Abstract
Ping Zeng1,*, Peng Zhang2,*, Li-Na Zhou1,*, Min Tang1, Yi-Xin Shen2, Jun Jin1, Ya-Qun Zhu3 and Min-Bin Chen1
1 Department of Radiotherapy and Oncology, Kunshan First People’s Hospital Affiliated to Jiangsu University, Kunshan, Jiangsu Province, China
2 Department of Orthopedics, the Second Affiliated Hospital of Soochow University, Suzhou, China
3 Department of Radiotherapy and Oncology, The Second Affiliated Hospital of Soochow University, Institute of Radiotherapy & Oncology, Soochow University, Suzhou, China
* These authors have contributed equally to this work and should be considered co-first authors
Correspondence to:
Min-Bin Chen, email:
Keywords: TMPRSS4, tumor, prognosis, overall survival, time to tumor progression
Received: April 10, 2016 Accepted: June 02, 2016 Published: June 17, 2016
Abstract
Recent studies have investigated the potential prognostic value of the transmembrane protease serine 4 (TMPRSS4) in various solid tumors. Yet, the results are inconclusive. Here, we performed this meta-analysis to clarify this issue. Relevant articles were identified by searching PubMed, Web of Science and Embase databases. The primary outcome endpoints were patients’ overall survival (OS) and time to tumor progression (TTP). Twelve studies involving 1,955 participants were included. We showed that high TMPRSS4 expression in tumor tissues was significantly associated with patients’ poor OS (pooled HR = 2.981, 95% CI = 2.296-3.869, P < 0.001) and short TTP (pooled HR = 2.456, 95% CI = 1.744-3.458, P < 0.001). A subgroup analysis revealed that the association between TMPRSS4 and the outcome endpoints (OS or TTP) was also significant within China region. We conclude that TMPRSS4 overexpression in solid tumors is associated with patients’ poor prognosis. TMPRSS4 could be a valuable prognosis biomarker or a promising therapeutic target of solid tumor.
Introduction
The transmembrane protease serine 4 (TMPRSS4) is a single-pass type II membrane protein and a novel serine protease [1]. It contains a serine protease domain, a scavenger receptor cysteine-rich domain and a low-density lipoprotein receptor class A domain [1]. Several TMPRSS4 specific substrates have been identified thus far, including hemagglutinin of the influenza virus (a protein that is vital for virus infection) [2] and the urokinase-type plasminogen activator (uPA), the latter is important for cancer cell invasion [1, 3].
Existing evidences have reported an important role of TMPRSS4 in cell motility, invasion, proliferation and tumor metastasis [1, 4-13]. It has been implied that TMPRSS4 could be an emerging potential therapeutic target in cancer [1]. Inhibition of TMPRSS4 reduced migration and invasion of lung and colon cancer cells [9, 14]. Knockdown of TMPRSS4 by targeted shRNA inhibited proliferation of lung cancer cells [15]. Reversely, forced overexpression of this protease enhanced lung cancer cell migration and invasion [9]. At the molecular level, TMPRSS4 overexpression was shown to activate several important pro-cancerous signalling cascades, including focal adhesion kinase (FAK), ERK1/2, Akt, Src and Rac1 pathways [7]. Among them, FAK and Rac1 activation was required for TMPRSS4-mediated cancer cell invasion and epithelial to mesenchymal transition (EMT) [7]. Meanwhile, PI3K or Src inhibitors were shown to inhibit invasiveness of TMPRSS4-overexpressing cells [7].
Growing evidences have demonstrated that TMPRSS4 is overexpressed in multiple solid tumors, including non-small cell lung cancer [6, 9, 15-17], malignant thyroid neoplasm [18-20], breast cancer [21, 22], and colon cancer [5]. Further, multiple studies have implied that elevated TMPRSS4 expression in tumor tissues was correlated with poor survival of cancer patients [1]. The results of those individual studies were however not conclusive. In the present study, we performed this comprehensive meta-analysis to clarify the prognostic value of TMPRSS4 in solid tumors .
Results
Demographic characteristics
A total of 80 articles were retrieved by a literature search from PubMed, Embase, and Web of Science databases, using different combinations of key terms. As indicated in the search flow diagram (Figure. 1). Twelve studies reported at least one outcome endpoint and were included in this meta-analysis [16, 21-31]. All studies were assessed by Newcastle-Ottawa Scale (NOS). The quality scores ranged from 6 to 8, indicating that the methodological quality was high. The main features of these eligible studies were summarized in Table 1. In total, the 12 studies provided a sample of 1,955 patients to assess the relationship between TMPRSS4 expression in solid tumor tissues and prognosis of affected patients. The median sample-size was 122, with a range from 69 to 436. Among all cohorts, China (n = 10) became the major source region of literatures, followed by Japan (n = 1) and Spain (n = 1). As for the cancer type, 2 studies evaluated breast cancer, 2 studies evaluated lung cancer, 2 studies study evaluated gastric cancer, 1 study evaluated colorectal cancer, 1 study evaluated prostate cancer, 1 study evaluated salivary adenoid cystic carcinoma, 1 study evaluated cervical squamous carcinoma, 1 study evaluated gallbladder cancer, 1 study evaluated hepatocellular carcinoma. Of all the studies, 11 studies focused on OS, and 8 studies focused on TTP.
Table 1: Characteristics of studies included in the meta-analysis.
First author | Year | Country | case | Cancer type | Disease Stage | Detection | Provided information on cutoff value | outcome endpoints | NOS score |
Chikaishi Y [16] | 2016 | Japan | 161 | lung adenocarcinoma | IA-IIIB | ICH | positively stained tumor cells>50% | OS | 8 |
Villalba M [25] | 2016 | Spain | 79 | lung squamous carcinoma | I-II | ICH | The H-score was >the median | OS, RFS | 7 |
Wang CH [31] | 2015 | China | 398 | hepatocellular carcinoma | I-III | ICH | Score≥4(rang of 0-12) | OS, RFS | 8 |
Wu XY [32] | 2014 | China | 97 | gallbladder cancer | I-IV | ICH | Score≥4(rang of 0-9) | OS | 7 |
Sheng H [33] | 2014 | China | 200 | gastric cancer | I-IV | ICH | Score≥2(rang of 0-3) | OS | 7 |
Shi G [37] | 2014 | China | 69 | prostate cancer | ICH | Score≥9(rang of 0-12) | DFS | 6 | |
Cheng D [21] | 2013 | China | 72 | Triple-negative breast cancer | I-III | ICH | Score≥3(rang of 0-6) | OS, DFS | 7 |
Cheng D [34] | 2013 | China | 87 | cervical squamous carcinoma | I-IV | ICH | Score≥3(rang of 0-7) | OS, DFS | 7 |
Dai W [35] | 2013 | China | 125 | salivary adenoid cystic carcinoma | I-IV | ICH | Score≥3(rang of 0-6) | OS, DFS | 7 |
Huang A [36] | 2013 | China | 122 | colorectal cancer | I-IV | ICH | Score≥4(rang of 0-9) | OS, DFS | 7 |
Liang B [22] | 2013 | China | 109 | breast cancer | I-III | ICH | Score≥3(rang of 0-6) | OS, DFS | 8 |
Luo ZY [38] | 2013 | China | 436 | gastric cancer | I-IV | ICH | Score≥2(rang of 0-3) | OS | 7 |
ICH : Immunohistochemistry; NOS: Newcastle-Ottawa Scale; DFS: disease-free survival; RFS: recurrence-free survival.
Figure 1: The flow chart of the selection process in the meta-analysis.
Evidence synthesis
The meta-analysis of TMPRSS4 expression was therefore based on two outcome endpoints: OS and TTP. Eleven studies were included in the meta-analysis of OS. A random effects model was applied to calculate the pooled hazard ratio (HR) and 95% confidence interval (CI). The heterogeneity test reported a P value of 0.025 and an I2 values of 51.1%. The results showed that TMPRSS4 overexpression was associated with poor OS of solid tumors (pooled HR = 2.981, 95% CI = 2.296-3.869, P<0.001) (Figure 2). Eight studies included in the meta-analysis reported TTP. A random effects model was again utilized to calculate the pooled HR and 95% CI. The heterogeneity test reported a P value of 0.040 and an I2 values of 52.4%. The results again demonstrated a significant association between TMPRSS4 overexpression and short TTP (pooled HR = 2.456, 95% CI = 1.744-3.458, P<0.001) (Figure 3). Among all the studies, China is the major source region. We therefore calculated the associations between TMPRSS4 overexpression and outcome endpoints (OS and TTP) within China region. Significant correlations were observed in either OS (pooled HR = 2.985, 95% CI = 2.245-3.969, P<0.001) or TTP (pooled HR = 2.466, 95% CI = 1.663-3.458, P<0.001) in the random-effects model with moderate heterogeneity (I2 = 57.2%, P=0.016; I2 = 59.2%, P=0.023, respectively).
Figure 2: The correlation between TMPRSS4 expression and overall survival (OS) in solid tumor. TMPRSS4 overexpression in solid tumors is significantly associated with patients’ poor OS (pooled HR = 2.981, 95% CI = 2.296-3.869, P < 0.001).
Figure 3: The correlation between TMPRSS4 expression and time to tumor progression (TTP) in solid tumor. TMPRSS4 overexpression in solid tumors is significantly associated with patients’ short TTP (pooled HR = 2.456, 95% CI = 1.744-3.458, P < 0.001).
Publication bias and sensitivity analysis
Begg’s funnel plot and Egger’s test were used to estimate the publication bias of the included literatures. The shapes of the funnel plots for the OS and TTP showed no evidence of obvious asymmetry (Figure 4), and Egger’s tests revealed non-significant values (P =0.374 and 0.798, respectively). Moreover, sensitivity analysis was carried out to assess the influence of individual study on the overall results of OS and TTP. No individual study dominated this meta-analysis, and the removal of any single study had no significant effect on the overall conclusion (Figure 5).
Figure 4: Begg’s funnel plots for the studies involved in the meta-analysis of TMPRSS4 expression and the prognosis of patients with solid tumors. A. Overall survival. B. Time to tumor progression. Abbreviations: loghr, logarithm of hazard ratios; s.e., standard error.
Figure 5: Sensitivity analysis of the meta-analysis. A. Overall survival. B. Time to tumor progression.
Discussion
High TMPRSS4 expression could promote cancer progression and is implied as a poor prognoses marker in solid tumor patients [1, 10, 13, 21, 22, 27]. However, up to now, no meta-analyses have been performed to evaluate the prognostic value of TMPRSS4 expression in the tumor patients. To the best of our knowledge, this meta-analysis is the first comprehensive assessment of the literatures studying TMPRSS4 expression and tumor prognosis. We systematically evaluated survival data for 1,955 solid tumor patients of 12 independent studies. Our results indicated that the expression of TMPRSS4 is a poor prognostic factor of solid tumor, which is negatively associated with patients’ OS (pooled HR = 2.226, 95% CI = 1.798-2.655, P<0.001) and TTP (pooled HR = 2.248, 95% CI = 1.745-2.751, P<0.001). Meanwhile, the subgroup analysis revealed that the associations between TMPRSS4 overexpression and poor OS or short TTP were again significant within the China region.
There are several important implications in this meta-analysis. First, high TMPRSS4 expression could be a general poor prognostic marker in solid tumors. We included eight different cancer types, including lung cancer [16, 23], breast cancer [21, 22], gastric cancer [26, 31], hepatocellular cancer [24], prostate cancer [30], salivary adenoid cystic cancer [28], cervical squamous cancer [27], gallbladder cancer [25]. The pooled results from these different types of cancer demonstrated that high TMPRSS4 expression was significantly associated with patients’ poor OS and short TTP. This finding can be extended to all solid tumors [16, 21-31]. Second, we demonstrated that high TMPRSS4 expression is correlated with poor OS and short TTP in the Chinese region. Unfortunately, we could not calculate the same associations within other regions due to the limited number of studies. Therefore, additional other region studies analyzing TMPRSS4 expression and cancer prognosis are needed to verify our results. Finally, it emphasizes the potential of TMPRSS4 as a valuable therapeutic target and prognostic biomarker for solid tumor.
There are also some limitations in this meta-analysis that should be considered in interpreting the outcomes. One of the main limitations is the moderate heterogeneity with the studies. However, we used a random-effects model with the pooled data. The heterogeneity among these studies could be explained by the different patients’ characteristics or differences in the specific study designs in different tumor types. Furthermore, most of the included studies were designed as retrospective studies, and such studies are more likely to be published if they have positive results than if they have negative results. Therefore, our estimate of the association between increased TMPRSS4 expression and outcome endpoints could have been overestimated. Additionally, the methodology for assessing TMPRSS4 expression and definition of TMPRSS4 positivity are inconsistent.
Taken together, the results clearly demonstrate that high TMPRSS4 expression in solid tumor tissues is associated with poor survival of affected patients. TMPRSS4 could be a useful prognostic biomarker or a promising therapeutic target for solid tumors [1]. However, further studies analyzing specific tumor types and perspectives are needed to further verify the clinical significance of TMPRSS4 expression in solid tumors.
Materials and Methods
Publication search
PubMed, Embase, and Web of Science databases were searched (up to April 10, 2016) using the search terms: ‘TMPRSS4’, ‘transmembrane protease serine 4’ and “cancer” / “tumor” / “neoplasm” / “carcinoma”. All potentially eligible studies were retrieved and their bibliographies were carefully scanned to identify other eligible studies. Additional studies were identified by searching the references that have cited the original studies. When multiple studies of the same patient population were identified, we included the published study with the largest sample size.
Inclusion and exclusion criteria
Studies included in this meta-analysis had to meet all of the following criteria: (a) Evaluation of TMPRSS4 expression for predicting prognosis in human cancer; (b) Studies reporting survival data; (c) Studies with adequate data of pooled hazard ratios (HRs) and 95% confidence intervals (CIs), and (d) Studies published in English. Inadequate survival data for further quantification or the follow-up duration shorter than 3 years, letters to the editor and abstracts, as well as case reports, review articles, experimental studies and commentary articles were excluded.
Data extraction and methodological quality assessment
Information was carefully and independently extracted from all eligible publications by two independent authors. Two outcome endpoints, disease-free survival (DFS) and recurrence-free survival (RFS), were combined, and an unified prognostic parameter, time to tumor progression (TTP), was utilized. The meta-analysis of TMPRSS4 expression was therefore based on two outcome endpoints: overall survival (OS) and TTP.
The following data were extracted for each study: the first author’s surname, publication year, country of origin, number of patients analyzed, types of cancer, disease stage, type of detection, score for TMPRSS4 assessment and cut-off values to determine TMPRSS4 positivity. Data for OS and TTP were extracted from tables with respect to TMPRSS4 expression. The Newcastle-Ottawa Scale (NOS) was applied to evaluate the methodological quality, which scored studies by the selection of the study groups, the comparability of the groups, and the ascertainment of the outcome of interest [32]. The studies with 6 scores or more were considered as high quality studies. For studies that presented only Kaplan-Meier curves, Engauge Digitizer version 4.3 was used to extract the survival data. The estimated HRs and 95% CIs were calculated by Tierney’s method [33]. Any potential disagreements between the authors were resolved by discussions until a consensus was reached.
Statistical analysis
Using the data collected from each eligible study, we performed the meta-analysis to evaluate the relationship between solid tumor’s TMPRSS4 expression and patients’ prognosis. Pooled HRs and 95% CIs for two outcome endpoints (OS, TTP) were calculated via a fixed effects model or random effects model. Heterogeneity assumption was checked using the Q test, and a P values of >0.10 indicated a lack of heterogeneity among studies. We also quantified the effect of heterogeneity using I2 =100%×(Q - df)/Q. I2 values of <25% may be considered “low”, values around 50% may be considered “moderate”, and values of >75% may be considered “high” [34]. In the absence of statistical heterogeneity, a fixed effects model was employed to calculate the pooled HRs, otherwise the random effects model was applied [35]. Funnel plots and the Egger’s test were employed to estimate the possible publication bias [36]. We performed sensitivity analysis by omitting each study or specific studies to find potential outliers. Statistical analyses were conducted using Stata 14.0 (StataCorp, College Station, TX). P values for all comparisons were two-tailed and statistical significance was defined as P<0.05, unless otherwise mentioned.
Acknowledgments
This work is supported by the National Natural Science Foundation (81472786), The Six Talents Peak Project of Jiangsu Province (2012-WSN-012). Kunshan Science and Technology Program (KS1418, KS1528).
Conflicts of interest
The authors have declared that no competing interests exist.
References
1. de Aberasturi AL and Calvo A. TMPRSS4: an emerging potential therapeutic target in cancer. Br J Cancer. 2015; 112:4-8.
2. Bertram S, Glowacka I, Steffen I, Kuhl A and Pohlmann S. Novel insights into proteolytic cleavage of influenza virus hemagglutinin. Rev Med Virol. 2010; 20:298-310.
3. Passero CJ, Mueller GM, Myerburg MM, Carattino MD, Hughey RP and Kleyman TR. TMPRSS4-dependent activation of the epithelial sodium channel requires cleavage of the gamma-subunit distal to the furin cleavage site. Am J Physiol Renal Physiol. 2012; 302:F1-8.
4. Hamamoto J, Soejima K, Naoki K, Yasuda H, Hayashi Y, Yoda S, Nakayama S, Satomi R, Terai H, Ikemura S, Sato T, Arai D, Ishioka K, Ohgino K and Betsuyaku T. Methylation-induced downregulation of TFPI-2 causes TMPRSS4 overexpression and contributes to oncogenesis in a subset of non-small-cell lung carcinoma. Cancer science. 2015; 106:34-42.
5. Huang A, Zhou H, Zhao H, Quan Y, Feng B and Zheng M. TMPRSS4 correlates with colorectal cancer pathological stage and regulates cell proliferation and self-renewal ability. Cancer biology & therapy. 2014; 15:297-304.
6. Jung H, Lee KP, Park SJ, Park JH, Jang YS, Choi SY, Jung JG, Jo K, Park DY, Yoon JH, Park JH, Lim DS, Hong GR, Choi C, Park YK, Lee JW, et al. TMPRSS4 promotes invasion, migration and metastasis of human tumor cells by facilitating an epithelial-mesenchymal transition. Oncogene. 2008; 27:2635-2647.
7. Kim S, Kang HY, Nam EH, Choi MS, Zhao XF, Hong CS, Lee JW, Lee JH and Park YK. TMPRSS4 induces invasion and epithelial-mesenchymal transition through upregulation of integrin alpha5 and its signaling pathways. Carcinogenesis. 2010; 31:597-606.
8. Kim S and Lee JW. Membrane Proteins Involved in Epithelial-Mesenchymal Transition and Tumor Invasion: Studies on TMPRSS4 and TM4SF5. Genomics & informatics. 2014; 12:12-20.
9. Larzabal L, de Aberasturi AL, Redrado M, Rueda P, Rodriguez MJ, Bodegas ME, Montuenga LM and Calvo A. TMPRSS4 regulates levels of integrin alpha5 in NSCLC through miR-205 activity to promote metastasis. Br J Cancer. 2014; 110:764-774.
10. Min HJ, Lee MK, Lee JW and Kim S. TMPRSS4 induces cancer cell invasion through pro-uPA processing. Biochem Biophys Res Commun. 2014; 446:1-7.
11. Min HJ, Lee Y, Zhao XF, Park YK, Lee MK, Lee JW and Kim S. TMPRSS4 upregulates uPA gene expression through JNK signaling activation to induce cancer cell invasion. Cellular signalling. 2014; 26:398-408.
12. Ohler A and Becker-Pauly C. Morpholino knockdown of the ubiquitously expressed transmembrane serine protease TMPRSS4a in zebrafish embryos exhibits severe defects in organogenesis and cell adhesion. Biological chemistry. 2011; 392:653-664.
13. Choi SY, Shin HC, Kim SY and Park YW. Role of TMPRSS4 during cancer progression. Drug news & perspectives. 2008; 21:417-423.
14. Jung H, Lee KP, Park SJ, Park JH, Jang YS, Choi SY, Jung JG, Jo K, Park DY, Yoon JH, Lim DS, Hong GR, Choi C, Park YK, Lee JW, Hong HJ, et al. TMPRSS4 promotes invasion, migration and metastasis of human tumor cells by facilitating an epithelial-mesenchymal transition. Oncogene. 2008; 27:2635-2647.
15. Larzabal L, Nguewa PA, Pio R, Blanco D, Sanchez B, Rodriguez MJ, Pajares MJ, Catena R, Montuenga LM and Calvo A. Overexpression of TMPRSS4 in non-small cell lung cancer is associated with poor prognosis in patients with squamous histology. Br J Cancer. 2011; 105:1608-1614.
16. Chikaishi Y, Uramoto H, Koyanagi Y, Yamada S, Yano S and Tanaka F. TMPRSS4 Expression as a Marker of Recurrence in Patients with Lung Cancer. Anticancer research. 2016; 36:121-127.
17. de Aberasturi AL, Redrado M, Villalba M, Larzabal L, Pajares MJ, Garcia J, Evans SR, Garcia-Ros D, Bodegas ME, Lopez L, Montuenga L and Calvo A. TMPRSS4 induces cancer stem cell-like properties in lung cancer cells and correlates with ALDH expression in NSCLC patients. Cancer letters. 2016; 370:165-176.
18. Guan H, Liang W, Liu J, Wei G, Li H, Xiu L, Xiao H and Li Y. Transmembrane protease serine 4 promotes thyroid cancer proliferation via CREB phosphorylation. Thyroid : official journal of the American Thyroid Association. 2015; 25:85-94.
19. Izkhakov E, Somjen D, Sharon O, Knoll E, Aizic A, Fliss DM, Limor R and Stern N. Vitamin D receptor expression is linked to potential markers of human thyroid papillary carcinoma. The Journal of steroid biochemistry and molecular biology. 2016; 159:26-30.
20. Kebebew E, Peng M, Reiff E, Duh QY, Clark OH and McMillan A. ECM1 and TMPRSS4 are diagnostic markers of malignant thyroid neoplasms and improve the accuracy of fine needle aspiration biopsy. Annals of surgery. 2005; 242:353-361; discussion 361-353.
21. Cheng D, Kong H and Li Y. TMPRSS4 as a poor prognostic factor for triple-negative breast cancer. International journal of molecular sciences. 2013; 14:14659-14668.
22. Liang B, Wu M, Bu Y, Zhao A and Xie F. Prognostic value of TMPRSS4 expression in patients with breast cancer. Medical oncology. 2013; 30:497.
23. Villalba M, Diaz-Lagares A, Redrado M, de Aberasturi AL, Segura V, Bodegas ME, Pajares MJ, Pio R, Freire J, Gomez-Roman J, Montuenga LM, Esteller M, Sandoval J and Calvo A. Epigenetic alterations leading to TMPRSS4 promoter hypomethylation and protein overexpression predict poor prognosis in squamous lung cancer patients. Oncotarget. 2016; 7:22752-69. doi: 10.18632/oncotarget.8045.
24. Wang CH, Guo ZY, Chen ZT, Zhi XT, Li DK, Dong ZR, Chen ZQ, Hu SY and Li T. TMPRSS4 facilitates epithelial-mesenchymal transition of hepatocellular carcinoma and is a predictive marker for poor prognosis of patients after curative resection. Scientific reports. 2015; 5:12366.
25. Wu XY, Zhang L, Zhang KM, Zhang MH, Ruan TY, Liu CY and Xu JY. Clinical implication of TMPRSS4 expression in human gallbladder cancer. Tumour biology : the journal of the International Society for Oncodevelopmental Biology and Medicine. 2014; 35:5481-5486.
26. Sheng H, Shen W, Zeng J, Xi L and Deng L. Prognostic significance of TMPRSS4 in gastric cancer. Neoplasma. 2014; 61:213-217.
27. Cheng D, Liang B and Li Y. High TMPRSS4 expression is a predictor of poor prognosis in cervical squamous cell carcinoma. Cancer epidemiology. 2013; 37:993-997.
28. Dai W, Zhou Q, Xu Z and Zhang E. Expression of TMPRSS4 in patients with salivary adenoid cystic carcinoma: correlation with clinicopathological features and prognosis. Medical oncology. 2013; 30:749.
29. Huang A, Zhou H, Zhao H, Quan Y, Feng B and Zheng M. High expression level of TMPRSS4 predicts adverse outcomes of colorectal cancer patients. Medical oncology. 2013; 30:712.
30. Shi G, Yang X, Dai B, Zhang H, Shen Y, Zhu Y, Zhu Y, Xiao W, Ma C, Wen L, Qin X, Cao D and Ye D. Clinical significance of TMPRSS4 in prostate cancer. International journal of clinical and experimental pathology. 2014; 7:8053-8058.
31. Luo ZY, Wang YY, Zhao ZS, Li B and Chen JF. The expression of TMPRSS4 and Erk1 correlates with metastasis and poor prognosis in Chinese patients with gastric cancer. PloS one. 2013; 8:e70311.
32. Stang A. Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. European journal of epidemiology. 2010; 25:603-605.
33. Tierney JF, Stewart LA, Ghersi D, Burdett S and Sydes MR. Practical methods for incorporating summary time-to-event data into meta-analysis. Trials. 2007; 8:16.
34. Higgins JP, Thompson SG, Deeks JJ and Altman DG. Measuring inconsistency in meta-analyses. Bmj. 2003; 327:557-560.
35. DerSimonian R and Laird N. Meta-analysis in clinical trials. Controlled clinical trials. 1986; 7:177-188.
36. Egger M, Davey Smith G, Schneider M and Minder C. Bias in meta-analysis detected by a simple, graphical test. Bmj. 1997; 315:629-634.