P value-driven methods were underpowered to detect publication bias: analysis of Cochrane review meta-analyses

Ref ID 332
First Author L. Furuya-Kanamori
Journal JOURNAL OF CLINICAL EPIDEMIOLOGY
Year Of Publishing 2020
URL https://www.jclinepi.com/article/S0895-4356(19)30381-6/fulltext
Keywords • General medical
• Publication bias
Problem(s) • Poor consideration of publication bias
• Weaknesses identified in some Cochrane reviews
Article Type Empirical
Article Subtype Cross-sectional survey/Methodological systematic review
First Author Country Australia
Aim To assess the effect of number of studies in a meta-analysis on the detection of publication bias using P value-driven methods in Cochrane reviews published 2003-2018.
Level of Investigation Descriptive
Summary of Findings The proportion of meta-analyses detected as asymmetrical via Egger’s, Harbord’s, Peters’, and Begg’s tests decreased by 42.6%, 41.1%, 29.3%, and 28.3%, respectively, when the median number of studies in the meta-analysis decreased from 87 to 14. P values decreased as the number of studies increased in the meta-analysis, despite the level of symmetry remaining constant. The simulation study confirmed that when publication bias is present, P value tests underestimate the presence of publication bias, particularly when study numbers are small.
Number of systematic reviews included 5014
Number of eligible systematic reviews assessed 18229
Treatment impacted No
Treatment impacted description
Interpretation impacted Not Applicable
Interpretation impacted description