Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study

Ref ID 7
First Author M. J. Page
Journal PLOS MEDICINE
Year Of Publishing 2016
URL https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002028
Keywords Reproducibility
Protocols
Multiplicity
Grey literature
Risk of bias
Pre-specification
Language
General medical
Low reporting quality
Searching
Problem(s) No registered or published protocol
Grey literature excluded
Poor consideration of publication bias
Low reporting (PRISMA) quality
No quality assessment undertaken or reported
Multiplicity of outcomes and lack of pre-specification for outcome reporting
Meta-analyses and forest plots presented without considering risk of bias / quality
Inadequate analysis of heterogeneity
Single reviewer / lack of double checking
Methods not described to enable replication
Insufficient literature searches
Search strategy not provided
Lack of prespecification in eligibility criteria
Reasons for excluding potentially eligible studies not provided
Interpreted without considering certainty or overall quality of the evidence base
Risk of bias not incorporated into conclusions of review
Conflicts of interest or funding of included studies not assessed
Funding or sponsor of systematic review not reported
Language restriction
Number of systematic reviews included 300
Summary of Findings Few of the included systematic reviews (12/300 [4%]) had been prospectively registered (e.g., in PROSPERO). Authors mentioned working from a review protocol in 77/300 (26%), but a publicly accessible protocol was cited in only 49/300 (16%). In 103/300 (34%) systematic reviews, publication status criteria were not reported. Systematic reviews considering all languages (129/300 [43%]) than considering English only (92/300 [31%]). 9% of systematic reviews searched only one database, Years of coverage of the search were completely reported in 196/300 (65%) systematic reviews, and a full Boolean search logic for at least one database was reported in 134/300 (45%). Searching of trial registries (e.g., ClinicalTrials. gov) was reported in 58/300 (19%). There was no information in approximately a third of systematic reviews on how authors performed screening (87/300 [29%]), data extraction (97/296 [33%]), or risk of bias assessment (76/206 [37%]). Risk of bias/quality assessment was formally conducted in 206/296 (70%) systematic reviews. Risk of bias information was incorporated into the analysis (e.g., via subgroup or sensitivity analyses) in only 31/189 (16%) of systematic reviews with meta-analysis. Reasons for excluding studies were described in 211/300 (70%). At least one type of grey literature (e.g., conference abstract, thesis) was stated to have been included in 26/300 (9%). No outcomes were specified in the methods section of 66/300 (22%). In a third of systematic reviews (72/189 [38%]), it was stated that a heterogeneity statistic guided the choice of the meta-analysis model (e.g., random-effects model used if I2 > 50%). Methods commonly used to infer publication bias (e.g., funnel plot, test for small-study effects) were used in approximately a third of systematic reviews (93/300 [31%]). Few systematic reviews included a GRADE assessment of the quality of the body of evidence (32/300 [11%]); Study limitations (including risk of bias/quality) were incorporated into the abstract conclusions of only 99/164 (60%) therapeutic systematic reviews. Only 21/296 (7%) reported the conflicts of interest or funding sources of the studies included in their SR. The funding source was not declared in a third of systematic reviews (109/300 [36%]).
Did the article find that the problem(s) led to qualitative changes in interpretation of the results? Not Applicable
Are the methods of the article described in enough detail to replicate the study? Yes