Overly stringent inclusion criteria affecting external validity

11

Insufficient literature searches

86

Grey literature excluded

59

Language restriction

22

Lack of supplementary searches beyond databases

17

Errors or omissions in search strategy

13

Outdated searches

7

Perpetuates citation of poor quality primary study data

24

Ignores setting or context of included studies which limits review applicability

15

Errors in study inclusion or omission of relevant studies

34

Reliance on randomised controlled trials for harms / safety data

12

Poor consideration of publication bias

92

Redundant / overlapping / duplicated review question; leads to research waste

21

Untimely (taking too long) or resource intensive

8

Not updated regularly

7

Intervention not described / defined

21

Data extraction errors and double counting

20

Inclusion of observational / non-randomised studies

12

Conflicts of interest or funding of included studies not assessed

85

Flawed risk of bias undertaken

62

No quality assessment undertaken or reported

66

Limited quality assessment or no risk of bias

38

Meta-analyses and forest plots presented without considering risk of bias / quality

25

Incorrect interpretation or statistical inference error from meta-analysis

12

Inadequate analysis of heterogeneity

65

Unplanned or unjustified subgroup or sensitivity analyses

16

Small number of trials in meta-analyses

6

Errors in effect estimate calculations or data synthesis

33

Risk of bias not incorporated into conclusions of review

67

Poor execution of narrative synthesis

13

Low methodological (AMSTAR) quality

159

Weaknesses identified in some Cochrane reviews

70

Cochrane reviews more rigorous/higher quality than non-Cochrane reviews

35

Low reporting or methodological quality (OTHER GUIDANCE)

20

Lack of guidance or consistency in systematic overview / umbrella / review of systematic reviews

7

Following guidelines is no guarantee of a rigorous systematic review

18

No registered or published protocol

128

Methods not described to enable replication

40

Lack of prespecification in eligibility criteria

62

Multiplicity of outcomes and lack of pre-specification for outcome reporting

37

Undocumented or unjustified deviations to the review protocol

20

Funding or sponsor of systematic review not reported

54

Conflict of interest statement or disclosures for review authors missing

46

Search strategy not provided

22

Reasons for excluding potentially eligible studies not provided

104

Individual study characteristics not reported sufficiently

28

Selective reporting of harms / safety / adverse events / side effects

20

Failure to address missing outcome data in analyses

16

Errors in systematic review abstracts or plain language summaries

16

Unwieldy/ difficult to read

6

Low reporting (PRISMA) quality

50

Lack of diversity in review authorship teams

4

Lack of clinical expert/ stakeholder/ user perspective

10

Review question not justified / important

5

Inflexible methods to complex questions

8

Failure to consider equity, different socioeconomic groups or disadvantaged populations

17

Failure to define clinically meaningful outcomes

17

Literature searches not validated by information specialist

15

Single reviewer / lack of double checking

62

Lack of statistical expertise in handling of quantitative data

32

Spin or subjective interpretation of findings

23

Unpublished or "zombie" reviews (the file-drawer effect)

13

Interpreted without considering certainty or overall quality of the evidence base

19

Inconclusive or lack of recommendations

10

Guest / gift / ghost authorship

4

Non-financial conflicts of interest of review authors

14

Financial conflicts of interest of review authors

21

High risk of bias (ROBIS)

15