Inadequate analysis of heterogeneity

This problem is addressed in PRISMA 2009, PRISMA 2020, AMSTAR 1, AMSTAR 2, ROBIS and MECIR. If trials differ due to clinical, methodological or statistical heterogeneity this should be elaborated appropriately and meta-analyses of such trials may be misleading. Common errors include meta-analysing trials in spite of substantial heterogeneity, ignoring heterogeneity, or over-reliance on the I squared statistic as a descriptor of heterogeneity.

Articles that support this problem:

Reporting and methodologic quality of Cochrane Neonatal review group systematic reviews

2009 : Bmc pediatrics

Journal impact factor is associated with PRISMA endorsement, but not with the methodological quality of low back pain systematic reviews: a methodological review

2020 : European spine journal

Methodological Quality of Systematic Reviews Relating to Performance of All-Ceramic Implant Abutments, Frameworks, and Restorations

2021 : Journal of prosthodontics

An AMSTAR assessment of the methodological quality of systematic reviews of oral healthcare interventions published in the Journal of Applied Oral Science (JAOS)

2011 : Journal of applied oral science

Quality of urological systematic reviews registered in PROSPERO

2019 : Bju international

The methodological quality of systematic reviews on the treatment of adult major depression needs improvement according to AMSTAR 2: A cross-sectional study

2020 : Heliyon

Heterogeneity of systematic reviews in oncology

2017 : Baylor university medical center proceedings

Heterogeneity can impair the results of Cochrane meta‐analyses despite accordance with statistical guidelines

2008 : Allergy

The impact of including different study designs in meta-analyses of diagnostic accuracy studies

2013 : European journal of epidemiology

Methodologically weak systematic review of Chinese herbal medicine for diabetic peripheral neuropathy calls for more rigorous trials

2013 : Focus on alternative and complementary therapies

Potential Problems With Systematic Reviews and Meta-Analyses

2017 : Journal of pain

Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice

2002 : Journal of health services research & policy

Clinical heterogeneity was a common problem in Cochrane reviews of physiotherapy and occupational therapy

2006 : Journal of clinical epidemiology

Statistical methods can be improved within Cochrane pregnancy and childbirth reviews

2011 : Journal of clinical epidemiology

Systematic overview finds variation in approaches to investigating and reporting on sources of heterogeneity in systematic reviews of diagnostic studies

2014 : Journal of clinical epidemiology

Are systematic reviews taking heterogeneity into account? An analysis from the Infectious Diseases Module of the Cochrane Library

2000 : Unknown

Strong heterogeneity of outcome reporting in systematic reviews

2016 : Journal of clinical epidemiology

Dealing with substantial heterogeneity in Cochrane reviews. Cross-sectional study

2011 : Bmc medical research methodology

The association between periodontitis and coronary heart disease: a quality assessment of systematic reviews

2013 : Journal of the american dental association

A re-analysis of the Cochrane Library data: the dangers of unobserved heterogeneity in meta-analyses

2013 : Plos one [electronic resource]

Apparently conclusive meta-analyses may be inconclusive--Trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta-analyses

2009 : International journal of epidemiology

Evolution of heterogeneity (I2) estimates and their 95% confidence intervals in large meta-analyses

2012 : Plos one

Incorrect inclusion of individual studies and methodological flaws in systematic review and meta-analysis

2014 : British journal of general practice

Quality of reporting in systematic reviews of implantable medical devices

2012 : Unknown

Cochrane systematic reviews of Chinese herbal medicines: an overview

2011 : Plos one

Dissemination bias in systematic reviews of animal research: a systematic review

2014 : Plos one

Not all systematic reviews are created equal

2018 : Canadian journal of occupational therapy

In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias

2005 : Journal of clinical epidemiology

A systematic survey showed important limitations in the methods for assessing drug safety among systematic reviews

2020 : Journal of clinical epidemiology

Misleading funnel plot for detection of bias in meta-analysis

2000 : Journal of clinical epidemiology

A review of methodological quality of systematic reviews on multiple pregnancies

2006 : Journal of obstetrics & gynaecology

Quality of the Evidence Supporting the Role of Oral Nutritional Supplements in the Management of Malnutrition: An Overview of Systematic Reviews and Meta-Analyses

2021 : Adv nutr

Trust, but verify. The errors and misinterpretations in the Cochrane analysis by O. J. Storebo and colleagues on the efficacy and safety of methylphenidate for the treatment of children and adolescents with ADHD

2016 : Zeitschrift fur kinder-und jugendpsychiatrie und psychotherapie

A case study of a retracted systematic review on interactive health communication applications: impact on media, scientists, and patients

2005 : Journal of medical internet research

A Cochrane review on brain [18F]FDG PET in dementia: limitations and future perspectives

2015 : European journal of nuclear medicine & molecular imaging

Epidemiology, quality, and reporting characteristics of systematic reviews and meta-analyses of nursing interventions published in Chinese journals

2015 : Nursing outlook

Survey of the reporting characteristics of systematic reviews in rehabilitation

2013 : Physical therapy

Quality of Cochrane reviews: Quality of Cochrane reviews is better than that of non-Cochrane reviews

2002 : Bmj: british medical journal

The fate of urological systematic reviews registered in PROSPERO

2019 : World journal of urology

Low Methodological Quality of Systematic Reviews Published in the Urological Literature (2016-2018)

2020 : Urology

Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study

2016 : Plos medicine

A methodological review of how heterogeneity has been examined in systematic reviews of diagnostic test accuracy.

2005 : Health technology assessment

Cochrane vertebroplasty review misrepresented evidence for vertebroplasty with early intervention in severely affected patients

2020 : Bmj evidence-based medicine

Methodological issues and recommendations for systematic reviews of prognostic studies: an example from cardiovascular disease

2014 : Systematic reviews

Letter to the editor regarding "the prevalence and influencing factors of methicillin-resistant Staphylococcus aureus carriage in people in contact with livestock: A systematic review"

2015 : American journal of infection control

Reduce variation and improve quality in meta-analyses

2014 : Annals of surgery

Letter to the Editor on "Prognostic Role of Serum Albumin, Total Lymphocyte Count, and Mini Nutritional Assessment on Outcomes After Geriatric Hip Fracture Surgery: A Meta-Analysis and Systematic Review"

2019 : Journal of arthroplasty

Characteristics and quality of systematic reviews and meta-analyses of observational nutritional epidemiology: a cross-sectional study

2021 : American journal of clinical nutrition

Results from a meta-analysis comparing bovine carotid artery grafts with polytetrafluoroethylene grafts must be interpreted with caution due to methodological flaws

2021 : Journal of vascular access

Methodological quality was critically low in 9/10 systematic reviews in advanced cancer patients-A methodological study

2021 : Journal of clinical epidemiology

Errors and Biases in Meta-analysis of the Prevalence of Olfactory Dysfunction in Patients With COVID-19

2021 : Otolaryngology - head and neck surgery (united states)

Comments on "the role of percutaneous neurolysis in lumbar disc herniation: Systematic review and meta-analysis"

2022 : Korean journal of pain

Commentary on "The association between serum vitamin D, fertility and semen quality: A systematic review and meta-analysis" [Int. J. Surg. 71 (2019) 101-109]

2021 : International journal of surgery

Methodological issues in designing and reporting of systematic reviews in assessing association between vitamin D supplementation and COVID-19 severity

2023 : Qjm

Methodological considerations for systematic review and meta-analysis of Xpert bladder cancer monitor

2022 : Urologic oncology: seminars and original investigations

Striking Errors in the Methodology, Execution, and Conclusions of the Cochrane Library Review of Spinal Cord Stimulation for Low Back Pain by Traeger et al

2023 : Pain medicine

How is the quality of the available evidence on molar-incisor hypomineralization treatment? An overview of systematic reviews

2022 : Clinical oral investigations

Comment on "Ticks infected with Crimean-Congo hemorrhagic fever virus (CCHFV): A decision approach systematic review and meta-analysis regarding their role as vectors"

2023 : Travel medicine and infectious disease

A commentary on "Robot-assisted kidney transplantation as a minimally invasive approach for kidney transplant recipients: A systematic review and meta-analyses" [Int. J. Surg. 99 (2022) 106264]

2022 : International journal of surgery

Letter to the Editor concerning "the influence of cognitive behavioral therapy on lumbar spine surgery outcomes: a systematic review and meta-analysis" by Parish JM, et al. (Eur Spine J [2021]; 30(5):1365-1379)

2022 : European spine journal

The normality assumption on between-study random effects was questionable in a considerable number of Cochrane meta-analyses

2023 : Bmc medicine

Impact of industry sponsorship on the quality of systematic reviews of vaccines: a cross-sectional analysis of studies published from 2016 to 2019

2022 : Systematic reviews