Inter-review agreement of risk-of-bias judgments varied in Cochrane reviews

Ref ID	343
First Author	N. Konsgen
Journal	JOURNAL OF CLINICAL EPIDEMIOLOGY
Year Of Publishing	2020
URL	https://www.jclinepi.com/article/S0895-4356(19)30647-X/fulltext
Keywords	• General medical • Risk of bias • Cochrane
Problem(s)	• Flawed risk of bias undertaken • Weaknesses identified in some Cochrane reviews
Number of systematic reviews included	53
Summary of Findings	The level of agreement between Cochrane reviews which included overlapping randomised controlled trials regarding risk of bias judgments ranged from slight to substantial depending on the risk of bias domain: ‘‘allocation concealment’’: kappa = 0.51 (0.41 to 0.61), kappa = 0.75; ‘‘blinding’’: kappa = 0.19 (0.02 to 0.37), kappa = 0.52; ‘‘blinding of outcome assessment’’: kappa = 0.43 (0.14 to 0.72) kappa = 0.67; and ‘‘incomplete outcome data’’: kappa = 0.15 (-0.03 to 0.32), kappa = 0.64. For ‘‘blinding of participants and personnel’’ and ‘‘selective reporting’’, kappa could not be calculated. The raw agreement was 0.40 and 0.42, respectively.
Did the article find that the problem(s) led to qualitative changes in interpretation of the results?	N/A
Are the methods of the article described in enough detail to replicate the study?	Yes