The normality assumption on between-study random effects was questionable in a considerable number of Cochrane meta-analyses

Ref ID 885
First Author Z. Liu
Journal BMC MEDICINE
Year Of Publishing 2023
URL https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-023-02823-9
Keywords Cochrane
Statistical
Heterogeneity
General medical
Problem(s) Inadequate analysis of heterogeneity
Weaknesses identified in some Cochrane reviews
Number of systematic reviews included 5652
Summary of Findings From 7667 Cochrane meta-analyses from 5652 overarching Cochrane systematic reviews. The between-study normality assumption was commonly violated in Cochrane meta-analyses. Based on 4234 eligible meta-analyses with binary outcomes and 3433 with non-binary outcomes, the proportion of meta-analyses that had statistically significant non-normality varied from 15.1 to 26.2%. Risk differences and non-binary outcomes led to more frequent non-normality issues than odds ratios and risk ratios. For binary outcomes, the between-study non-normality was more frequently found in meta-analyses with larger sample sizes and event rates away from 0 and 100%.
Did the article find that the problem(s) led to qualitative changes in interpretation of the results? Not Applicable
Are the methods of the article described in enough detail to replicate the study? No