Validity of data extraction in evidence synthesis practice of adverse events: reproducibility study

Ref ID 837
First Author C. Xu
Journal BMJ
Year Of Publishing 2022
URL https://www.bmj.com/content/377/bmj-2021-069155.full
Keywords Harms
Reproducibility
Error
Non-Cochrane reviews
Problem(s) Data extraction errors and double counting
Number of systematic reviews included 201
Summary of Findings From 201 included systematic reviews of RCTs indexed on Pubmed between 1 January 2015 and 1 January 2020. The 201 comprising reviews included 829 pairwise meta-analyses, and data extraction could not be reproduced in 1762 (17.0%) of 10 386 trials. In 554 (66.8%) of 829 meta-analyses, at least one randomised controlled trial had data extraction errors; 171 (85.1%) of 201 systematic reviews had at least one meta-analysis with data extraction errors. Meta-analyses that had two or more different types of errors were more susceptible to these changes than those with only one type of error (for moderate changes, 11 (28.2%) of 39 v 26 (10.4%) 249, P=0.002; for large changes, 5 (12.8%) of 39 v 8 (3.2%) of 249, P=0.01).
Did the article find that the problem(s) led to qualitative changes in interpretation of the results? Yes
Are the methods of the article described in enough detail to replicate the study? Yes