Paper – Systematic Reviewlution

Ref ID	7
First Author	M. J. Page
Journal	PLOS MEDICINE
Year Of Publishing	2016
URL	https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002028
Keywords	• General medical • Reproducibility • Risk of bias • Searching • Low reporting quality • Language • Pre-specification • Grey literature • Multiplicity • Protocols
Problem(s)	• No quality assessment undertaken or reported • Multiplicity of outcomes and lack of pre-specification for outcome reporting • Meta-analyses and forest plots presented without considering risk of bias / quality • Inadequate analysis of heterogeneity • Single reviewer / lack of double checking • Methods not described to enable replication • Insufficient literature searches • Search strategy not provided • Lack of prespecification in eligibility criteria • Reasons for excluding potentially eligible studies not provided • Interpreted without considering certainty or overall quality of the evidence base • Risk of bias not incorporated into conclusions of review • Conflicts of interest or funding of included studies not assessed • Funding or sponsor of systematic review not reported • Language restriction • No registered or published protocol • Grey literature excluded • Poor consideration of publication bias • Low reporting (PRISMA) quality
Number of systematic reviews included	300
Summary of Findings	Few of the included systematic reviews (12/300 [4%]) had been prospectively registered (e.g., in PROSPERO). Authors mentioned working from a review protocol in 77/300 (26%), but a publicly accessible protocol was cited in only 49/300 (16%). In 103/300 (34%) systematic reviews, publication status criteria were not reported. Systematic reviews considering all languages (129/300 [43%]) than considering English only (92/300 [31%]). 9% of systematic reviews searched only one database, Years of coverage of the search were completely reported in 196/300 (65%) systematic reviews, and a full Boolean search logic for at least one database was reported in 134/300 (45%). Searching of trial registries (e.g., ClinicalTrials. gov) was reported in 58/300 (19%). There was no information in approximately a third of systematic reviews on how authors performed screening (87/300 [29%]), data extraction (97/296 [33%]), or risk of bias assessment (76/206 [37%]). Risk of bias/quality assessment was formally conducted in 206/296 (70%) systematic reviews. Risk of bias information was incorporated into the analysis (e.g., via subgroup or sensitivity analyses) in only 31/189 (16%) of systematic reviews with meta-analysis. Reasons for excluding studies were described in 211/300 (70%). At least one type of grey literature (e.g., conference abstract, thesis) was stated to have been included in 26/300 (9%). No outcomes were specified in the methods section of 66/300 (22%). In a third of systematic reviews (72/189 [38%]), it was stated that a heterogeneity statistic guided the choice of the meta-analysis model (e.g., random-effects model used if I2 > 50%). Methods commonly used to infer publication bias (e.g., funnel plot, test for small-study effects) were used in approximately a third of systematic reviews (93/300 [31%]). Few systematic reviews included a GRADE assessment of the quality of the body of evidence (32/300 [11%]); Study limitations (including risk of bias/quality) were incorporated into the abstract conclusions of only 99/164 (60%) therapeutic systematic reviews. Only 21/296 (7%) reported the conflicts of interest or funding sources of the studies included in their SR. The funding source was not declared in a third of systematic reviews (109/300 [36%]).
Did the article find that the problem(s) led to qualitative changes in interpretation of the results?	N/A
Are the methods of the article described in enough detail to replicate the study?	Yes