Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study

Ref ID 467
First Author B. Tendal
Journal BMJ
Year Of Publishing 2009
URL https://www.bmj.com/content/bmj/339/bmj.b3128.full.pdf
Keywords Cochrane
General medical
Problem(s) Errors in effect estimate calculations or data synthesis
Lack of statistical expertise in handling of quantitative data
Data extraction errors and double counting
Number of systematic reviews included 10
Summary of Findings Agreement was 53% at trial level and 31% at meta-analysis level. Reasons for disagreement reasons for disagreement were broadly related to: different choices, exclusion of a trial, and data extraction errors. In 14 out of the 100 standardised mean differences calculated at the meta-analysis level, individual observers reached different conclusions than the originally published review.
Did the article find that the problem(s) led to qualitative changes in interpretation of the results? Yes
Are the methods of the article described in enough detail to replicate the study? Yes