Evaluation of the Cochrane tool for assessing risk of bias in randomized clinical trials: overview of published comments and analysis of user practice in Cochrane and non-Cochrane reviews

Ref ID 501
First Author L. Jørgensen
Journal SYSTEMATIC REVIEWS
Year Of Publishing 2016
URL https://pubmed.ncbi.nlm.nih.gov/27160280/
Keywords Cochrane
Risk of bias
General medical
Problem(s) Flawed risk of bias undertaken
Number of systematic reviews included 68
Summary of Findings The tool was used in all Cochrane reviews (100/100) and was the preferred tool in non-Cochrane reviews (31/100). The tool is often used in ways not recommended such as implementing non-standard domains either as fully new domains or incorporating them into the “other bias” function, which is primarily intended for special situations, e.g. crossover trials. Both strengths and challenges of the Cochrane risk of bias tool were identified from comments. Challenges included: concern whether the chosen domains comprehensively address all threats to validity (for example, including funding as an independent bias domain); difficulties in the subjective interpretation of the tool; modest inter-observer agreement, difficulty in assessing selective reporting of outcomes, terminological ambiguity (i.e. of the terms subjective/objective) and the low proportion of reviews using risk of bias assessments as a basis for sensitivity analyses; challenges in assigning an overall risk of bias to a trial based on risk of bias of single domains to the trial as such.
Did the article find that the problem(s) led to qualitative changes in interpretation of the results? Not Applicable
Are the methods of the article described in enough detail to replicate the study? Yes