| Ref ID | 455 |
| First Author | O. Olsen |
| Journal | BMJ |
| Year Of Publishing | 2001 |
| URL | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC57800/pdf/829.pdf |
| Keywords |
• General medical • Cochrane • Spin |
| Problem(s) |
• Weaknesses identified in some Cochrane reviews • Spin or subjective interpretation of findings • Following guidelines is no guarantee of a rigorous systematic review |
| Article Type | Empirical |
| Article Subtype | Survey/questionnaire of authors opinions/practices |
| First Author Country | Denmark |
| Aim | Ten methodologists affiliated with the Cochrane Collaboration independently examined, in a semi-structured way, the quality of reviews first published in 1998. Each review was assessed by two people; if one of them noted any major problems, they agreed on a common assessment. Predominant types of problem were categorised. |
| Level of Investigation | Descriptive |
| Summary of Findings | 29% of Cochrane reviews first published in 1998 were judged to have major problems. There were three areas of concern. Firstly, the evidence did not support the conclusions in 17% reviews. Secondly, the conduct or reporting of the reviews was unsatisfactory in 23% reviews. Thirdly, there were stylistic concerns with 23% reviews. The problematic conclusions all described the effect of the experimental treatment in terms that we judged to have been too optimistic (table 3). None of these nine reviews indicated a bias towards the control treatment. |
| Number of systematic reviews included | 52 |
| Number of eligible systematic reviews assessed | 53 |
| Treatment impacted | Yes |
| Treatment impacted description | |
| Interpretation impacted | Yes |
| Interpretation impacted description | The authors conclude that too often, reviewers' conclusions overrated the benefits of new interventions |