| Ref ID | 84 |
| First Author | I. Roberts |
| Journal | BMJ: BRITISH MEDICAL JOURNAL |
| Year Of Publishing | 2015 |
| URL | https://pubmed.ncbi.nlm.nih.gov/26041754/ |
| Keywords |
• General medical • Retraction • Cochrane |
| Problem(s) |
• Perpetuates citation of poor quality primary study data • Small number of trials in meta-analyses |
| Article Type | Editorial |
| Article Subtype | Discussion piece |
| First Author Country | United Kingdom |
| Aim | A letter highlighting several limitations of systematic reviews, both with regards to published examples up to 2015 and also theoretically. |
| Level of Investigation | Descriptive |
| Summary of Findings | Discusses examples of limitations of published systematic reviews including: a Cochrane review showing that high dose mannitol reduced the risk of death after head injury that was retracted after the review group editors were unable to confirm that any of the included trials took place. The authors discuss the paradox that clinical trial protocols invariably include a estimate of the sample size needed for the results to be reliable but systematic reviewers rarely estimate how many participants would need to be included in a meta-analysis for reliable results. The median number of trials in Cochrane reviews is between six and 16, and the median number of patients per trial is about 80. Consequently, most meta-analyses include small numbers of trial participants. Meta-analyses with sparse data can miss modest but clinically important treatment effects, and the potential for random error to result in false positive conclusions is considerable. The authors assert that the number of trial participants needed for a reliable meta-analysis should be at least as large as for an appropriately powered trial. |
| Number of systematic reviews included | 1 |
| Number of eligible systematic reviews assessed | 1 |
| Treatment impacted | No |
| Treatment impacted description | |
| Interpretation impacted | Not Applicable |
| Interpretation impacted description |