| Ref ID | 491 |
| First Author | L. A. Kahale |
| Journal | BMJ |
| Year Of Publishing | 2020 |
| URL | https://www.bmj.com/content/bmj/370/bmj.m2898.full.pdf |
| Keywords |
• General medical • Statistical • Missing data • Cochrane |
| Problem(s) |
• Failure to address missing outcome data in analyses • Failure to define clinically meaningful outcomes |
| Article Type | Empirical |
| Article Subtype | Meta-epidemiological analysis |
| First Author Country | Lebanon |
| Aim | To assess the risk of bias associated with missing outcome data in a sample of both Cochrane and non-Cochrane systematic reviews by data imputation and estimating the median % change in the relative effect estimate. Combinations of plausible and implausible assumptions based on best and worst case scenarios were considered. Other outcomes were: percentage of meta-analyses that crossed the threshold of the null effect for each method; and percentage of meta-analyses that qualitatively changed direction of effect for each method. |
| Level of Investigation | Analytical |
| Summary of Findings | The median change in the relative effect estimate varied from 0% to 30.4% (implausible assumptions) and from 1.4% to 7.0%. (plausible assumptions). Meta-analyses crossing the threshold of the null effect varied from 1% (best case scenario) to 60% (worst case scenario) and 26% changed direction with the worst case scenario. Using an imputation approach based on information missingness odds ratio, meta-analyses crossing the threshold of the null effect varied from 6% to 22%, and 2% changed direction with the most stringent. Judgments of whether changes in effect estimate are clinically significant requires using minimal clinically important difference, which varies by clinical question. |
| Number of systematic reviews included | 100 |
| Number of eligible systematic reviews assessed | 100 |
| Treatment impacted | Yes |
| Treatment impacted description | Sensitivity analyses based on the eight different methods of handling missing data were conducted to assess relative change in the treatment effect including direction and crossing the threshold of null effect. |
| Interpretation impacted | Yes |
| Interpretation impacted description | The average change in pooled relative effect estimate is substantive, and almost a quarter (22%) of meta-analyses crossed the threshold of the null effect. |