Skip to main content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Gerstein Science Information Centre

Knowledge syntheses: Systematic & Scoping Reviews, and other review types

You've finally narrowed the thousands of citations down to a smaller, more relevant group of articles. Now what do you do to make sense of this information? Take a look below at the various tools and guides to help you determine the quality of and risk of bias for each article.

"Critical appraisal skills enable you to assess the trustworthiness, relevance and results of published papers so that you can decide if they are believable and useful."

Source: https://libraryguides.mayo.edu/systematic-reviews/appraisingcitations

*Our librarians do not provide consultations on critical appraisal, however we're happy to provide you with the information and resources below.

Critical appraisal by study design

Systematic Reviews

AMSTAR

AMSTAR can be used to assess the methodological quality of a systematic review and as a guide to performing a systematic review. Two agreements are required during quality assessment ensuring lower risk of bias. AMSTAR has guidelines explaining each outlined item.

https://amstar.ca/index.php

ROBIS

ROBIS is a new tool for assessing the risk of bias in systematic reviews (rather than in primary studies). Here you can find the tool itself, information to help you complete a ROBIS assessment, and resources to help you present the results of your ROBIS assessment.

http://www.bristol.ac.uk/population-health-sciences/projects/robis/

CASP: Systematic Review Checklist
Summary: Critical Appraisal Skills Program (CASP): This is a methodological checklist which provides key criteria relevant to systematic reviews.
Authors: Public Health Resource Unit, NHS, England

https://casp-uk.net/wp-content/uploads/2018/01/CASP-Systematic-Review-Checklist_2018.pdf

CEBM:  Systematic Reviews Critical Appraisal Worksheet 
Authors: Centre for Evidence Based Medicine (CEBM), University of Oxford

https://casp-uk.net/wp-content/uploads/2018/01/CASP-Systematic-Review-Checklist_2018.pdf


Randomized Controlled Trials

RoB 2.0 Risk Of Bias Tool

New as of July 2019

RoB 2 Tool : A revised Cochrane risk of bias tool for randomized trials.

https://sites.google.com/site/riskofbiastool/welcome/rob-2-0-tool?authuser=0

Archived: There are 3 variations of the tool based on trial design.  Choose the appropriate version.

1.  Individually randomized, parallel group trials (traditional RCT)

2.  Cluster randomized, parallel group trials (like traditional but randomized by groups)

3.  Individual randomized, cross-over trials (randomization occurs to determine the order the patient receives both treatments)

http://www.bristol.ac.uk/population-health-sciences/centres/cresyda/barr/riskofbias/rob2-0/​


CASP: Randomised Controlled Trial Checklist
Summary: Critical Appraisal Skills Program (CASP): RCT CAT is a methodological checklist which provides key criteria relevant to randomised controlled trials.
Authors: Public Health Resource Unit, NHS, England

https://casp-uk.net/wp-content/uploads/2018/01/CASP-Randomised-Controlled-Trial-Checklist-2018.pdf


The CONSORT Statement
Summary: The Consolidated Standards of Reporting Trials (CONSORT) Statement is a detailed document which outlines an explanation and elaboration of the CONSORT statement for reporting randomized controlled trials. It also includes (in table two) the critical appraisal tool.
Authors: The CONSORT Group, Canada 

http://www.consort-statement.org/consort-statement/


Non-randomized Studies or Observational Studies

ROBINS-I

Risk of Bias for non-randomized (observational) studies or cohorts of Interventions

http://www.bristol.ac.uk/population-health-sciences/centres/cresyda/barr/riskofbias/robins-i/​

ROBINS-E

Risk of Bias for non-randomized (observational) studies or cohorts of Exposures other than interventions, including environmental and occupational exposures

http://www.bristol.ac.uk/population-health-sciences/centres/cresyda/barr/riskofbias/robins-e/​

Newcastle-Ottawa scale (NOS) -most widely used for case control or cohort studies

>http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp

AXIS-To assess the quality of cross-sectional (prevalence) studies

https://bmjopen.bmj.com/content/6/12/e011458

NIH – NHLBI – Study Quality Assessment Tools

Tools for Controlled Intervention Studies, Systematic Reviews & Meta-Analyses, Observational Cohort and Cross-Sectional Studies, Case-Control Studies, Before-After (Pre-Post) Studies With No Control Group, Case Series Studies.

https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools

IHE Case Series Studies Critical Appraisal Checklist

https://www.ihe.ca/research-programs/rmd/cssqac/cssqac-about

Methodological Index for Non-Randomized Studies (MINOR) is a tool for assessing non-randomized interventional studies  

http://cobe.paginas.ufsc.br/files/2014/10/MINORS.pdf

Joanna Briggs Institute

Critical Appraisal Tools

http://joannabriggs.org/research/critical-appraisal-tools.html

McMaster Critical Review Form - Quantitative Studies

http://srs-mcmaster.ca/research/evidence-based-practice-research-group/


Animal Research

Systematic Review Centre for Laboratory Animal Experimentation (SYRCLE) risk of bias assessment tool for assessing animal studies

http://www.ncbi.nlm.nih.gov/pubmed/24667063

(invivo studies) Review the ARRIVE guidelines  

https://www.nc3rs.org.uk/arrive-guidelines

Critical Appraisal of Studies using Animal Models

http://ilarjournal.oxfordjournals.org/content/55/3/405.full.pdf+html


Diagnostic Accuracy

QUADAS-2 

QUADAS-2 is the current version of QUADAS and the tool that we recommend for use in systematic reviews to evaluate the risk of bias and applicability of primary diagnostic accuracy studies.

http://www.bristol.ac.uk/media-library/sites/quadas/migrated/documents/quadas2.pdf 

STARD Checklist
Summary: STAndards for the Reporting of Diagnostic accuracy checklist is helps determine the accuracy and completeness of reporting of studies of diagnostic accuracy. The STARD Statement consists of a 25 item checklist and flow diagram.

http://www.stard-statement.org/​


CASP: Diagnostic Test Checklist

https://casp-uk.net/wp-content/uploads/2018/01/CASP-Diagnostic-Checklist-2018.pdf

CEBM: Diagnosis Critical Appraisal Worksheet
Authors: Centre for Evidence Based Medicine (CEBM), University of Oxford

https://www.cebm.net/wp-content/uploads/2018/11/Diagnostic-Accuracy-Studies.pdf

 

Source:https://libraryguides.mayo.edu/systematic-reviews/appraisingcitations