- Framework of problems / Rigourous
- Following guidelines is no guarantee of a rigorous systematic review
- Changing patterns in reporting and sharing of review data in systematic reviews with meta-analysis of the effects of interventions: cross sectional meta-research study
| Ref ID | 924 |
| First Author | P.Y. Nguyen |
| Journal | BMJ |
| Year Of Publishing | 2022 |
| URL | https://dx.doi.org/10.1136/bmj-2022-072428 |
| Keywords |
Reproducibility Open data Transparency General medical Low reporting quality |
| Problem(s) |
No registered or published protocol Limited quality assessment or no risk of bias Search strategy not provided Methods not described to enable replication Funding or sponsor of systematic review not reported Following guidelines is no guarantee of a rigorous systematic review |
| Number of systematic reviews included | 300 |
| Summary of Findings | From 300 included systematic reviews of health, social, behavioural, or educational interventions indexed across PubMed, Science Citation Index, Social Sciences Citation Index, Scopus, and Education Collection up to November 2020. Items reported suboptimally from the systematic reviews were: registration record for the review (n=113; 38%), a full search strategy for at least one database (n=214; 71%), methods used to assess risk of bias (n=185; 62%), methods used to prepare data for meta-analysis (n=101; 34%), and source of funding for the review (n=215; 72%). Only a few items not already reported at a high frequency in 2014 were reported more frequently in 2020. Reviews published in 2020 in journals that mandated either data sharing or inclusion of data availability statements were more likely to share their review materials (eg, data, code files) than reviews in journals without such mandates (16/87 (18%) v 4/213 (2%)). Most systematic reviews in 2020 cited a reporting guideline, yet such guideline use was not clearly associated with improved reporting for any of the assessed items. This uncertain association between citation of a reporting guideline and completeness of reporting challenges the assumption that referencing a reporting guideline guarantees adherence to the guideline. |
| Did the article find that the problem(s) led to qualitative changes in interpretation of the results? | Not Applicable |
| Are the methods of the article described in enough detail to replicate the study? | Yes |