Systematic Reviews of Anesthesiologic Interventions Reported as Statistically Significant: Problems with Power, Precision, and Type 1 Error Protection

Ref ID 39
First Author G. Imberger
Journal ANESTHESIA & ANALGESIA
Year Of Publishing 2015
URL https://pubmed.ncbi.nlm.nih.gov/26579662/
Keywords • Power
• Error
• Statistical
• Pain
Problem(s) • Perpetuates citation of poor quality primary study data
• Small number of trials in meta-analyses
• Errors in effect estimate calculations or data synthesis
Article Type Empirical
Article Subtype Meta-epidemiological analysis
First Author Country Denmark
Aim To assess power and imprecision of systematic reviews with meta-analyses that were reported as statistically significant in anaesthesia in the context of sparse data and repeated updates.
Level of Investigation Descriptive
Summary of Findings In the 50 sampled systematic reviews, most nominally statistically significant meta-analyses of anesthesiologic interventions were underpowered, and many did not maintain their risk of type 1 error <5% if trial sequential analysis monitoring boundaries are applied. The median number of trials included was 8 (interquartile range [IQR], 5–14), the median number of participants was 964 (IQR, 523–1736), and the median number of participants with the outcome was 202 (IQR, 96–443). By using two trial sequential analysis approaches, only 12% (95% CI, 5%–25%) of the meta-analyses had power ≥80%, and only 32% (95% CI, 20%–47%) of the meta-analyses preserved the risk of type 1 error <5%.
Number of systematic reviews included 50
Number of eligible systematic reviews assessed 11870
Treatment impacted No
Treatment impacted description
Interpretation impacted Not Applicable
Interpretation impacted description