Systematic Reviews of Anesthesiologic Interventions Reported as Statistically Significant: Problems with Power, Precision, and Type 1 Error Protection

Ref ID 39
First Author G. Imberger
Year Of Publishing 2015
Keywords Pain
Problem(s) Perpetuates citation of poor quality primary study data
Small number of trials in meta-analyses
Errors in effect estimate calculations or data synthesis
Number of systematic reviews included 50
Summary of Findings In the 50 sampled systematic reviews, most nominally statistically significant meta-analyses of anesthesiologic interventions were underpowered, and many did not maintain their risk of type 1 error <5% if trial sequential analysis monitoring boundaries are applied. The median number of trials included was 8 (interquartile range [IQR], 5–14), the median number of participants was 964 (IQR, 523–1736), and the median number of participants with the outcome was 202 (IQR, 96–443). By using two trial sequential analysis approaches, only 12% (95% CI, 5%–25%) of the meta-analyses had power ≥80%, and only 32% (95% CI, 20%–47%) of the meta-analyses preserved the risk of type 1 error <5%.
Did the article find that the problem(s) led to qualitative changes in interpretation of the results? Not Applicable
Are the methods of the article described in enough detail to replicate the study? No