Assessment of methodological quality of systematic reviews of acupuncture for chronic prostatitis/chronic pelvic pain symptom: an overview of systematic review

Introduction

Chronic prostatitis/chronic pelvic pain syndrome (CP/CPPS) is a common disorder in urology, which affects 2.2% to 13.8% worldwide (1-3). The main symptom of CP/CPPS is urinary pain, lower urinary tract symptoms and/or sexual dysfunction, lasting for at least three months of the last half year (4). Patients with CP/CPPS also experienced a range of other symptoms such as erectile dysfunction (ED), which were found in 45.4% CP/CPPS patients (5). The combination of aforementioned symptoms significantly threats the quality of life of patients with CP/CPPS. However, limited by the unclear etiology and poorly understood pathophysiology, there is no “gold standard” diagnostic method (6,7). Currently, the diagnosis of CP/CPPS mainly based on excluded from other urological conditions such as chronic bacterial prostatitis (CBP), benign prostate hyperplasia (BPH), and the like. To date, the most common used approaches to CP/CPPS are alpha-blockers and antibiotics, however, whether alpha-blockers or antibiotics failed to show an notable results from the well-design, high-quality randomized controlled trials (8,9). Furthermore, antibiotics were often given as an experienced approach to patients with CP/CPPS. Alexander et al reported that there is no statistical difference between intervention and placebo group, and whether ciprofloxacin (antibiotic) or tamsulosin (alpha-blocker) could reduce symptoms in men with long-standing CP/CPPS who had at least moderate symptoms (9). As a part of complementary and alternative medicine, acupuncture has been used for 2,000 years in the East Asia as a useful approach to urological disorders. Recently, a number of randomized controlled trials or systematic reviews (SRs) have been conducted, and the results showed that acupuncture could serve as an effective approach to CP/CPPS (10-13). Considering the potential of acupuncture for treating CP/CPPS, we conducted this overview of reviews using Methodological Quality of Systematic Reviews (AMSTAR) to assess the quality of up-to-date SRs of acupuncture for treating CP/CPPS (14). Besides, we also summarize the current evidence of SRs of acupuncture for treating CP/CPPS.

Methods

Included criteria

Two forms of included criteria were set. The first criteria (criteria A) is apply to all SRs that compared acupuncture to other treatment; the second criteria (criteria B) is apply to SRs that has been assessed as “formulating conclusions appropriately” and “combing the findings of studies appropriately”. SRs that met criteria A will be assessed the methodological quality using AMSTAR; SRs that met criteria B will be extract data to summarize the findings of this overview.

Criteria A

In terms of participants, the SRs had to include clinical trials that involved patients with a diagnosis of CP/CPPS or non-bacterial prostatitis. For treatment group, any forms of acupuncture were considered in this overview, including needle acupuncture, electro-acupuncture and any other inserted acupuncture. We excluded unpenetrated needle, such as laser acupuncture, transcutaneous electrical nerve stimulation, and moxibustion. For control groups, we include placebo/sham acupuncture, conventional medication (alpha-blockers, antibiotics, etc.), usual care, Chinese herb medicine, physical therapy, and waiting list. In terms of outcome, we selected National Institutes of Health Chronic Prostatitis Symptom Index (NIH-CPSI), International Prostate Symptom Score (IPSS), response rate, and laboratory indicators as the outcomes measurements.

Criteria B

SRs should be assessed as “formulating conclusions appropriately” and “combing the findings of studies appropriately” according to AMSTAR. For the SRs met Criteria B, we summarized the findings of each SR as the effectiveness of acupuncture.

Literature research

We searched three international databases include MEDLINE, EMBASE, the Cochrane Database of Systematic Review, and three Chinese databases include Chinese Biomedical Databases, Wan Fang Digital Journals and China national knowledge internet (CNKI) from their inception through March 2017 to identify potential SRs, using the terms of “acupuncture”, “prostatitis”, “chronic prostatitis”, “chronic pelvic pain syndrome”. The terms of “systematic review” and “meta-analysis” were used as the filters.

Literature selection and data extraction

Two reviewers (ZQ and XL) scanned the abstracts and full-text, if needed. The data extraction was based on an electro-sheet established previously. The reviewers extracted the characteristics of included SRs including the basic characteristics such as authors’ name, design, publication years, number of included trials, outcomes, and the results of data synthesis.

Quality assessment of included reviews

Two reviews (ZQ and JW) assess the quality of included reviews using AMSTAR, which could overall evaluate the methodological quality of SRs with good reliability, validity, and responsibility. The eleven items of AMSTAR are listed as following:

Item 1: was a prior design provided? Item 2: was there duplicate study selection and data extraction? Item 3: was a comprehensive literature search performed? Item 4: was the status of publication used as an inclusion criterion? Item 5: was a list of studies provided? Item 6: were the characteristic of the included studies provided? Item 7: was the scientific quality of the included studies assessed and documented? Item 8: was the scientific quality of the included studies used appropriately in formulating conclusions? Item 9: were the methods used to combine the findings of studies appropriate? Item 10: was the likelihood of publication bias assessed? Item 11: was the conflict of interest included? Levels of agreement for each item were assessment using the kappa statistics, any disagreements were resolved through discussion or rated by a third investigator (ZL).

Data synthesis

The data synthesis were based on the included SRs. Regarding continuous outcome, we referred to the mean difference (MD), if the studies combined the continuous outcomes using varied measurements, the standard mean difference (SMD) were extracted. In terms of dichotomous outcomes, the odds ratio (OR) or relative risk (RR) were used.

Results

SR search and screening results

The database search strategies yielded 103 records, and 61 duplicates were identified and excluded. We excluded 27 studies after screening the titles and abstracts; the full texts of the remaining 15 studies were retrieved for further assessment. Six studies were excluded for the following reasons: 4 SRs included patients with BPH, 1 was not SR, and 1 SR assessed moxibustion instead of acupuncture. Finally, a total of 9 SRs met the inclusion criteria A (12,13,15,16), and only 4 SRs met the inclusion criteria B (12,13,15-21). Figure 1 describes the flow chart of searching and screening results.

Figure 1 Study flow diagram.

Characteristics of SRs

Eight SRs were published after 2010 (12,13,15-18,20,21), four of them were published in 2016 (12,16,20,21), and the latest searching period was June 2016 (16). Four SRs were published in English (12,13,15,16), and five SRs were published in Chinese (17-21). Two SRs included moxibustion and cupping in the treatment group (19,21), Six SRs included Chinese herb medicine in the control group (15,17-21). In terms of outcome, six SRs used NIH-CPSI as one of the outcome (12,13,15,16,18,21). Table 1 indicates the characteristics of included SRs.

Methodological quality of SRs

Regarding item 1, only 1 SR provided an a priori protocol and registered on the PROSPERO platform of York University (12). In terms of item 2, six of the SRs provided the information and details of data selection and extraction progress (12,13,15-17,21). Regarding item 3, seven SRs conducted a comprehensive literature search (12,13,15-17,20,21). In terms of item 4, two SRs declared that gray literature will be included (12,15). In terms of item 5, only one SR provided the list of excluded studies (15). In terms of item 6 and 7, all of the SRs provided the characteristics of the included studies and assessed the scientific quality of the included studies. In terms of item 8, four SRs formulated conclusion appropriately (12,13,15,16). In terms of item 9 and 10, four of the studies used appropriate methods to combine the findings (12,13,16,17), and six SRs assessed the likelihood of publication (13,16,17-21), respectively. None of the SRs included the conflict of interest. The agreement on study qualification between two reviewers for each item assessment domain ranged from 76% to 100% and the overall agreement was high at 91%. Table 2 indicates the assessment results of AMSTAR.

NIH-CPSI total score

Four SRs set the NIH-CPSI total score as of the outcome measurements (12,13,15,16), and summarized evidence on the effectiveness of acupuncture for the treatment of CP/CPPS. Three SRs conducted meta-analysis of NIH-CPSI total score (12,13,16), in which, Posadzki et al. described that among the included trials (15), only one randomized controlled trial conducted by Lee et al. (10), who set it as an objective scale outcome instead of subjective assessment such as self-assessment response rate. Posadzki et al. did not extract the original data from the trial of Lee, instead, they calculated the proportion of responders, which is defined as the number of patients who has an improvement of NIH-CPSI more than 6-point after 10-week acupuncture treatment period. Three SRs updated the same topic in 2016 (12,13,16). According to the results of meta-analysis of four trials, Qin et al. indicated that compared with sham acupuncture (MD: −6.09, 95% CI: −8.12 to −5.68), acupuncture might be more effectiveness at improving the total score of NIH-CPSI (12). In addition, the results of meta-analysis synthesizing three trials also suggested that compared with conventional medicine (MD: −4.57, 95% CI: −7.58 to −1.56), acupuncture was more effectiveness at decreasing the NIH-CPSI total score. Chang et al. supported aforementioned result (13), indicating that compared with sham acupuncture, the real needle could improve the total score of NIH-CPSI with a significant outcome (MD: −6.09, 95% CI: −7.85 to −4.33). Regarding the direct comparison between acupuncture and medication, however, Chang et al failed to combine the data owing to the insufficient included trials. Moreover, Liu et al. conducted meta-analysis of NIH-CPSI either (16), despite they combined the data from control group without subgroups, and the result suggested that compared with control group, acupuncture could improve the total score of NIH-CPSI (MD: −3.98, 95% CI: −5.78 to −2.19).

NIH-CPSI subscores

Two SRs set the scores of NIH-CPSI sub-scale as one of the secondary outcomes (12,16). The NIH-CPSI score which was recommended by National Institute of Health (NIH) consists three subscales, including pain, voiding, and the quality of life, respectively. Qin et al. initially synthesized the subscale outcome using meta-analysis (12). Accordingly, compared with sham acupuncture, the result combining four trials showed a significant improvement of pain (MD: −2.95, 95% CI: −5.05 to −0.85), voiding (MD: −1.31, 95% CI: −1.68 to −0.95) and quality of life (MD: −0.88, 95% CI: −1.20 to −0.56) were observed, respectively. Besides, compared with conventional medication, the meta-analysis, which involved three trials also indicated that acupuncture might be more effective for relieving pain symptoms (MD: −0.30, 95% CI: −4.4 to −1.98). However, the results did not support the evidence that acupuncture might be more effective for improving the symptoms of voiding (MD: 0.26, 95% CI: −2.03 to 2.56) and quality of life (MD: −0.79, 95% CI: −1.58 to 0.00). Furthermore, Liu et al. also reported this outcome and meta-analysis were conducted (16). Based on the result of Liu et al., the meta-analysis of three trials indicated that compared with sham acupuncture, the real needle could decrease the pain symptoms (MD: −3.76, 95% CI: −6.81 to −0.70), voiding (MD: −2.30, 95% CI: −4.47 to −0.12), and the quality of life (MD: −2.68, 95% CI: −4.69 to −0.77), respectively. Additionally, compared with the medication, the results of a meta-analysis included five trials also showed a statistical difference in pain (MD: −2.12, 95% CI: −3.54 to −0.69), and the quality of life (MD: −1.60, 95% CI: −3.02 to −0.18), respectively (16). However, in terms of voiding (MD: −0.54, 95% CI: −1.32 to 0.42), there is no statistical difference between the two groups.

IPSS score

Two SRs reported IPSS score as an outcome measurement (12,13). Qin et al. conducted a meta-analysis which included two trials suggested that there is no statistical difference between acupuncture and sham acupuncture (MD: −1.78, 95% CI: −4.30 to 0.75). However, the meta-analysis conducted by Chang et al. showed a controversial conclusion, which indicated that compared with sham acupuncture and conventional medicine (MD: −2.44, 95% CI: −4.86 to −0.03), acupuncture might be more effective in terms of improving the score of IPSS (13).

Response rate

Four SRs reported response rate, three of them conducted meta-analysis (12,13,15,16). Posadzki et al. failed to synthesis the data from trials owing to the insufficient direct comparisons and the distribution of the inconsistent control groups. According to the result of Posadzki et al., acupuncture was always effective to CP/CPPS compared with the conventional drugs for BPH, antibiotics, and sham acupuncture. Whereas there is no statistical difference between acupuncture and traditional herbal medicines. In addition, Qin et al. included three trials and conduct a meta-analysis, which indicated that regarding response rate, acupuncture might be more effective compared with sham acupuncture (RR: 1.60, 95% CI: 1.26 to 2.04) and medication (RR: 1.43, 95% CI: 1.08 to 1.90), respectively. The results of Liu et al. also supported aforementioned results, compared with sham acupuncture (RR: 1.93, 95% CI: 1.31 to 2.88) and medication (RR: 2.03, 95% CI: 1.04 to 3.97), the meta-analysis indicated acupuncture might be more effective. Tang et al. and Chang et al. used OR as the statistical outcome, compared with sham acupuncture (OR: 5.15, 95% CI: 2.72 to 9.75) and standard medicine (OR: 3.57, 95% CI: 1.78 to 7.15), the results suggested that acupuncture was more effective in terms of response rate.

Laboratory indicators

One SR including six trials reported laboratory indicators (16). Laboratory indicators include prostaglandin, E2, beta-endorphin, lecithin body, cortisol, leucine enkephalin, natural killer cell, TNF-alpha, IL-1 beta, and plasma substance P. Regarding TNF-alpha, the results of meta-analysis including two trials showed that after the acupuncture treatment, no statistical difference between two groups existed (MD: −18.47, 95% CI: −37.76 to 0.81). In terms of IL-1 beta, the results of meta-analysis including the same two trials showed that there is a statistical difference between two groups (MD: −27.18, 95% CI: −36.30 to −18.06). However, the heterogeneity was both significant observed among aforementioned results, with 88% I-square value and 56% I-square value, respectively.

Discussion

To date, there is 9 SRs and meta-analysis has been conducted for comparing acupuncture with sham acupuncture, conventional medication, Chinese herb medicine, physical therapy, and usual care according to the included criteria. Based on the results of this overview, we found that most trials included by the aforementioned SRs were overlapped. Posadzki et al. conducted the first SR of acupuncture for treating CP/CPPS published in English in 2012, and used NIH-CPSI as the outcome measurement. It concluded that acupuncture might be an effective approach to control the symptoms. However, the quality of evidence was low owing to the poorly report quality of included trials, in addition, the insufficient direct comparisons also limited data combination. After Posadzki’s study, five SRs published at a relative short period, and all SRs conducted meta-analysis successfully because the amounts were sufficient after 2012. Moreover, the results of these studies could support the conclusion that Posadzki has reported. This overview of review finds that acupuncture could be a safe and effective treatment in managing CP/CPPS symptoms, especially related pain symptoms. Given that acupuncture has less adverse effect. Acupuncture could be considered as an optional treatment for reliving the symptoms for patients with CP/CPPS.

The methodological quality of SRs of acupuncture for treating CP/CPPS is satisfactory. However, there are several respects should be improved in the further related study. First, although the researchers could conduct “high quality” SRs and meta-analysis under the guidance of AMSTAR or other methodological tools, the quality of involved original studies should be valued. Owing to the characteristics of acupuncture, it is difficult to blind acupuncturist. In addition, it is impossible to blind participant when the trial comparing acupuncture with oral medication or other kind of treatment instead sham acupuncture or placebo needle. All the aforementioned factors will potentially affect the results of acupuncture clinical controlled trials, and these factors will as well have an impact to the findings that conducted by SRs and meta-analysis. Thus, for researcher in acupuncture field should attach importance to both findings of SRs and original research. Second, in terms of methodological quality, improvements should be made in the following respects: (I) researchers should take advantage of the register platform online. It could not only focus researchers to formulate the protocol in advance to improve the quality of SRs, but also notice other researchers that this study has been working, so that duplicate work on the same topic would be prevented; (II) regarding the literature selection section of SRs. Instead of the list of included studies, the list of excluded studies should also be provided whether listed in the reference section or listed as an attachment affiliated to the SRs; (III) in terms of the synthesis section and finding summary section, authors of SRs should consult statistical and methodological experts about the statistics and methodology. Third, although this review did not assess the reporting quality of included SRs, as there are overlaps between assessment of reporting quality and methodological quality. Researchers should conduct and describe the SRs and meta-analysis according to the AMSTAR and PRISMA.

Acknowledgements

Funding: None.

Footnote

Conflicts of Interest: The authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/amj.2017.08.21). Dr. Li serves as an unpaid Section Editor of AME Medical Journal. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Nickel JC, Downey J, Hunter D, et al. Prevalence of prostatitis-like symptoms in a population based study using the National Institutes of Health chronic prostatitis symptom index. J Urol 2001;165:842-5. [Crossref] [PubMed]
2. Roberts RO, Jacobson DJ, Girman CJ, et al. Prevalence of prostatitis-like symptoms in a community based cohort of older men. J Urol 2002;168:2467-71. [Crossref] [PubMed]
3. Collins MM, Meigs JB, Barry MJ, et al. Prevalence and correlates of prostatitis in the health professionals follow-up study cohort. J Urol 2002;167:1363-6. [Crossref] [PubMed]
4. Krieger JN, Nyberg L Jr, Nickel JC. NIH consensus definition and classification of prostatitis. JAMA 1999;282:236-7. [Crossref] [PubMed]
5. Hao ZY, Li HJ, Wang ZP, et al. The prevalence of erectile dysfunction and its relation to chronic prostatitis in Chinese men. J Androl 2011;32:496-501. [Crossref] [PubMed]
6. Rees J, Abrahams M, Doble A, et al. Diagnosis and treatment of chronic bacterial prostatitis and chronic prostatitis/chronic pelvic pain syndrome: a consensus guideline. BJU Int 2015;116:509-25. [Crossref] [PubMed]
7. Engeler DS, Baranowski AP, Dinis-Oliveira P, et al. The 2013 EAU guidelines on chronic pelvic pain: is management of chronic pelvic pain a habit, a philosophy, or a science? 10 years of development. Eur Urol 2013;64:431-9. [Crossref] [PubMed]
8. Nickel JC, Krieger JN, McNaughton-Collins M, et al. Alfuzosin and symptoms of chronic prostatitis-chronic pelvic pain syndrome. N Engl J Med 2008;359:2663-73. [Crossref] [PubMed]
9. Alexander RB, Propert KJ, Schaeffer AJ, et al. Ciprofloxacin or tamsulosin in men with chronic prostatitis/chronic pelvic pain syndrome: a randomized, double-blind trial. Ann Intern Med 2004;141:581-9. [Crossref] [PubMed]
10. Lee SW, Liong ML, Yuen KH, et al. Acupuncture versus sham acupuncture for chronic prostatitis/chronic pelvic pain. Am J Med 2008;121:79.e1-7. [Crossref] [PubMed]
11. Lee SH, Lee BC. Electroacupuncture relieves pain in men with chronic prostatitis/chronic pelvic pain syndrome: three-arm randomized trial. Urology 2009;73:1036-41. [Crossref] [PubMed]
12. Qin Z, Wu J, Zhou J, et al. Systematic Review of Acupuncture for Chronic Prostatitis/Chronic Pelvic Pain Syndrome. Medicine (Baltimore) 2016;95:e3095. [Crossref] [PubMed]
13. Chang SC, Hsu CH, Hsu CK, et al. The efficacy of acupuncture in managing patients with chronic prostatitis/chronic pelvic pain syndrome: a systematic review and meta-analysis. Neurourol Urodyn 2017;36:474-81. [Crossref] [PubMed]
14. Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007;7:10. [Crossref] [PubMed]
15. Posadzki P, Zhang J, Lee MS, et al. Acupuncture for chronic nonbacterial prostatitis/chronic pelvic pain syndrome: a systematic review. J Androl 2012;33:15-21. [Crossref] [PubMed]
16. Liu BP, Wang YT, Chen SD. Effect of acupuncture on clinical symptoms and laboratory indicators for chronic prostatitis/chronic pelvic pain syndrome: a systematic review and meta-analysis. Int Urol Nephrol 2016;48:1977-91. [Crossref] [PubMed]
17. Li Y, Xiong J, Du Y, et al. Acupuncture versus medicine for chronic prostaitits: a systematic review. Journal of Liaoning University of TCM 2010;37:1567-71.
18. He Y, Xia C, Liu B. Meta-analysis on acupuncture in treating chronic prostatitis. Chin J Tradit Chin Med Pharm 2015;30:226-9.
19. Yu X, Na M, Chen Y. A Cochrane systematic review of acupuncture therapy for chronic non-bacterial prostatitis. Journal of Shanghai University of TCM. 2007;23:47-9.
20. Pang Y, Liu C. Meta-analysis of Acupuncture Treatment of Chronic Prostatitis. Journal of Liaoning University of TCM. 2016;18:14-7.
21. Tang Y. Acupuncture for chronic prostatitis: a systematic review. Chongqing Medical University 2016. Available online: http://d.g.wanfangdata.com.cn/Thesis_D01120247.aspx