There were large discrepancies in risk of bias tool judgments when a randomized controlled trial appeared in more than one systematic review

Ref ID 379
First Author V. M. Jordan
Journal JOURNAL OF CLINICAL EPIDEMIOLOGY
Year Of Publishing 2017
URL https://www.jclinepi.com/article/S0895-4356(16)30414-0/fulltext
Keywords • Risk of bias
• Cochrane
• Gynaecology
Problem(s) • Flawed risk of bias undertaken
Article Type Empirical
Article Subtype Inter-rater agreement study
First Author Country New Zealand
Aim To assess consistency of risk of bias judgements for trials included in multiple reviews from the Cochrane Gynaecology and Fertility group in the field of subfertility.
Level of Investigation Descriptive
Summary of Findings From 34 Cochrane reviews including 46 duplicated trials, there were 156 bias judgments that were completed by at least two separate groups of authors and appeared in systematic reviews in the Cochrane Library. In 45% of cases, these risk of bias judgments differed. For random sequence generation there was 29% disagreement; For allocation concealment there was 41% disagreement; for blinding there was 65% disagreement; for incomplete outcome reporting there was 30% disagreement; and for selective reporting there was 57% disagreement.
Number of systematic reviews included 34
Number of eligible systematic reviews assessed 84