- Framework of problems / Rigourous
- Inadequate analysis of heterogeneity
- Dealing with substantial heterogeneity in Cochrane reviews. Cross-sectional study
| Ref ID | 414 |
| First Author | J. B. Schroll |
| Journal | BMC MEDICAL RESEARCH METHODOLOGY |
| Year Of Publishing | 2011 |
| URL | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3056846/pdf/1471-2288-11-22.pdf |
| Keywords |
Cochrane Error Heterogeneity General medical |
| Problem(s) |
Inadequate analysis of heterogeneity Weaknesses identified in some Cochrane reviews Errors in effect estimate calculations or data synthesis |
| Number of systematic reviews included | 60 |
| Summary of Findings | One third of reviews were regarded as problematic with regards to handling of heterogeneity and choice of statistical model for data synthesis. For 10% of included reviews, a significant result using a fixed-effects model changed to a non-significant when a random effects model was used. Appropriate caution was not expressed around the reliability of the pooled treatment effect in the majority of these reviews. One review calculated mean differences instead of standardized mean differences, although the outcomes were measured on very different scales. Whilst two thirds of reviews overall were devoid of major problems in relation to their handling of heterogeneity, only 27 reviews (45%) gave a rationale for choice of statistical model. |
| Did the article find that the problem(s) led to qualitative changes in interpretation of the results? | Yes |
| Are the methods of the article described in enough detail to replicate the study? | No |