Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice

Ref ID 293
First Author J. Higgins
Journal JOURNAL OF HEALTH SERVICES RESEARCH & POLICY
Year Of Publishing 2002
URL https://pubmed.ncbi.nlm.nih.gov/11822262/
Keywords • Heterogeneity
• Pre-specification
• General medical
• Subgroup
Problem(s) • Unplanned or unjustified subgroup or sensitivity analyses
• Inadequate analysis of heterogeneity
Article Type Empirical
Article Subtype Cross-sectional survey/Methodological systematic review
First Author Country United Kingdom
Aim To compare published protocols with all corresponding completed systematic reviews available in the Cochrane Database of Systematic Reviews. The authors also examined key systematic review guidance to assess adequacy of advice on methods to authors.
Level of Investigation Descriptive
Summary of Findings Plans for addressing heterogeneity between studies in systematic review protocols were seldom followed in the 39 Cochrane reviews examined. Fifteen of the 28 protocols pre-specified potential effect modifiers for use in subgroup analyses or meta-regression. Just two reviews performed only subgroup analyses that had been pre-specified in the protocol. Ten reviews did not undertake any of the planned subgroup analyses or meta-regressions, and in four further reviews at least one planned subgroup analysis was not undertaken. Ten reviews undertook subgroup analyses that were not mentioned in the protocol. Eight of the 11 reviews without protocols undertook subgroup analyses; it was not possible to determine whether these had been prespecified.
Number of systematic reviews included 39
Number of eligible systematic reviews assessed 39
Treatment impacted No
Treatment impacted description
Interpretation impacted Not Applicable
Interpretation impacted description