Poor reliability between Cochrane reviewers and blinded external reviewers when applying the Cochrane risk of bias tool in physical therapy trials

Ref ID 563
First Author S. Armijo-Olivo
Journal PLOS ONE
Year Of Publishing 2014
URL https://pubmed.ncbi.nlm.nih.gov/24824199/
Keywords • Cochrane
• Physiotherapy
• Inference
• Team
• Single reviewer
Problem(s) • Weaknesses identified in some Cochrane reviews
• Single reviewer / lack of double checking
Article Type Empirical
Article Subtype Inter-rater agreement study
First Author Country Canada
Aim To compare the inter-rater reliability of the Cochrane risk of bias tool applied to physical therapy trials by comparing ratings from published Cochrane reviews with those of blinded external reviewers
Level of Investigation Descriptive
Summary of Findings Inter-rater reliability on the overall risk of bias and the individual domain assessment between Cochrane review authors and blinded external reviewers was poor. The inter-rater reliability between blinded external reviewers on the overall risk of bias rating was fair. The study proposes that risk of bias from published Cochrane review could in future be replicated consistently by an external panel of reviewers using consensus risk of bias assessments.
Number of systematic reviews included 17
Number of eligible systematic reviews assessed 3901
Treatment impacted No
Treatment impacted description
Interpretation impacted Not Applicable
Interpretation impacted description