Paper – Systematic Reviewlution

Ref ID	7
First Author	M. J. Page
Journal	PLOS MEDICINE
Year Of Publishing	2016
URL	https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002028
Keywords	Reproducibility Protocols Multiplicity Grey literature Risk of bias Pre-specification Language General medical Low reporting quality Searching
Problem(s)	No registered or published protocol Grey literature excluded Poor consideration of publication bias Low reporting (PRISMA) quality No quality assessment undertaken or reported Multiplicity of outcomes and lack of pre-specification for outcome reporting Meta-analyses and forest plots presented without considering risk of bias / quality Inadequate analysis of heterogeneity Single reviewer / lack of double checking Methods not described to enable replication Insufficient literature searches Search strategy not provided Lack of prespecification in eligibility criteria Reasons for excluding potentially eligible studies not provided Interpreted without considering certainty or overall quality of the evidence base Risk of bias not incorporated into conclusions of review Conflicts of interest or funding of included studies not assessed Funding or sponsor of systematic review not reported Language restriction
Number of systematic reviews included	300
Summary of Findings	Few of the included systematic reviews (12/300 [4%]) had been prospectively registered (e.g., in PROSPERO). Authors mentioned working from a review protocol in 77/300 (26%), but a publicly accessible protocol was cited in only 49/300 (16%). In 103/300 (34%) systematic reviews, publication status criteria were not reported. Systematic reviews considering all languages (129/300 [43%]) than considering English only (92/300 [31%]). 9% of systematic reviews searched only one database, Years of coverage of the search were completely reported in 196/300 (65%) systematic reviews, and a full Boolean search logic for at least one database was reported in 134/300 (45%). Searching of trial registries (e.g., ClinicalTrials. gov) was reported in 58/300 (19%). There was no information in approximately a third of systematic reviews on how authors performed screening (87/300 [29%]), data extraction (97/296 [33%]), or risk of bias assessment (76/206 [37%]). Risk of bias/quality assessment was formally conducted in 206/296 (70%) systematic reviews. Risk of bias information was incorporated into the analysis (e.g., via subgroup or sensitivity analyses) in only 31/189 (16%) of systematic reviews with meta-analysis. Reasons for excluding studies were described in 211/300 (70%). At least one type of grey literature (e.g., conference abstract, thesis) was stated to have been included in 26/300 (9%). No outcomes were specified in the methods section of 66/300 (22%). In a third of systematic reviews (72/189 [38%]), it was stated that a heterogeneity statistic guided the choice of the meta-analysis model (e.g., random-effects model used if I2 > 50%). Methods commonly used to infer publication bias (e.g., funnel plot, test for small-study effects) were used in approximately a third of systematic reviews (93/300 [31%]). Few systematic reviews included a GRADE assessment of the quality of the body of evidence (32/300 [11%]); Study limitations (including risk of bias/quality) were incorporated into the abstract conclusions of only 99/164 (60%) therapeutic systematic reviews. Only 21/296 (7%) reported the conflicts of interest or funding sources of the studies included in their SR. The funding source was not declared in a third of systematic reviews (109/300 [36%]).
Did the article find that the problem(s) led to qualitative changes in interpretation of the results?	Not Applicable
Are the methods of the article described in enough detail to replicate the study?	Yes