| Ref ID | 837 |
| First Author | C. Xu |
| Journal | BMJ |
| Year Of Publishing | 2022 |
| URL | https://www.bmj.com/content/377/bmj-2021-069155.full |
| Keywords |
• Harms • Reproducibility • Error • Non-Cochrane reviews |
| Problem(s) |
• Data extraction errors and double counting |
| Article Type | Empirical |
| Article Subtype | Meta-epidemiological analysis |
| First Author Country | China |
| Aim | To assess the the validity of data extraction in systematic reviews of adverse events published between 1 January 2015 and 1 January 2020 and to assess the effect of data extraction errors on the results, and to develop a classification framework for data extraction errors. |
| Level of Investigation | Descriptive |
| Summary of Findings | From 201 included systematic reviews of RCTs indexed on Pubmed between 1 January 2015 and 1 January 2020. The 201 comprising reviews included 829 pairwise meta-analyses, and data extraction could not be reproduced in 1762 (17.0%) of 10 386 trials. In 554 (66.8%) of 829 meta-analyses, at least one randomised controlled trial had data extraction errors; 171 (85.1%) of 201 systematic reviews had at least one meta-analysis with data extraction errors. Meta-analyses that had two or more different types of errors were more susceptible to these changes than those with only one type of error (for moderate changes, 11 (28.2%) of 39 v 26 (10.4%) 249, P=0.002; for large changes, 5 (12.8%) of 39 v 8 (3.2%) of 249, P=0.01). |
| Number of systematic reviews included | 201 |
| Number of eligible systematic reviews assessed | 18636 |
| Treatment impacted | Yes |
| Treatment impacted description | |
| Interpretation impacted | Yes |
| Interpretation impacted description | Data extraction errors led to 10 (3.5%) of 288 meta-analyses changing the direction of the effect and 19 (6.6%) of 288 meta-analyses changing the significance of the P value. |