Skip to main content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Gerstein Science Information Centre

Knowledge syntheses: Systematic & Scoping Reviews, and other review types

You've finally narrowed the thousands of citations down to a smaller, more relevant group of articles. Now what do you do to make sense of this information? Take a look below at the various tools and guides to help you determine the quality of and risk of bias for each article.

"Critical appraisal skills enable you to assess the trustworthiness, relevance and results of published papers so that you can decide if they are believable and useful." (Mayo Clinic)

*Our librarians do not provide consultations on critical appraisal, however we're happy to provide you with the information and resources below.

Critical appraisal by study design

Systematic Reviews

Tool Author(s)/Organization Description
AMSTAR 2 A MeaSurement Tool to Assess systematic Reviews (AMSTAR)

A critical appraisal tool for systematic reviews that include randomised or nonrandomised studies of healthcare interventions, or both.

Used to assess the methodological quality of a systematic review and as a guide to performing a systematic review. Two agreements are required during quality assessment ensuring lower risk of bias. AMSTAR has guidelines explaining each outlined item.

CASP Checklist Critical Appraisal Skills Programme (CASP)Public Health Resource Unit, NHS, England A methodological checklist that provides key criteria relevant to systematic reviews. 
ROBIS Bristol Medical School: Population Health Sciences, University of Bristol Tool for assessing the risk of bias in systematic reviews (rather than in primary studies), offering information to help you complete a ROBIS assessment, and resources to help you present the results of your ROBIS assessment.
Systematic Review Critical Appraisal Worksheet The Centre for Evidence-Based Medicine (CEBM), University of Oxford A checklist with questions for assessing the reliability, importance, and applicability of the studies included based on the alignment of the research question and eligibility criteria.

For other tools and appraisal worksheets on different types of medical evidence.


Randomized Controlled Trials

Tool Author(s)/Organization Description
CASP: Randomised Controlled Trial Checklist Critical Appraisal Skills Programme (CASP), Public Health Resource Unit, NHS, England RCT CAT is a methodological checklist which provides key criteria relevant to randomised controlled trials.
The Consolidated Standards of Reporting Trials (CONSORT) Statement The CONSORT Group A detailed document that outlines an explanation and elaboration of the CONSORT statement for reporting randomized controlled trials. It also includes a critical appraisal flow diagram.
RoB 2 Tool (New as of July 2019) Risk of Bias, University of Bristol

A revised Cochrane risk of bias tool for randomized trials.

Archived: There are 3 variations of the tool based on trial design. Choose the appropriate version.

1.  Individually randomized, parallel group trials (traditional RCT)

2.  Cluster randomized, parallel group trials (like traditional but randomized by groups)

3.  Individual randomized, cross-over trials (randomization occurs to determine the order the patient receives both treatments)


Non-randomized Studies or Observational Studies

Tool Author(s)/Organization Description
AXIS Downes, M.J., Brennan. M.L., Williams, H.C., & Dean, R.S. A critical appraisal tool used to assess the quality of cross-sectional (prevalence) studies.
The Case Series Studies Critical Appraisal Checklist Institute of Health Economics (IHE) A quality appraisal checklist for case studies. 
Critical Appraisal Tools Joanna Briggs Institute (JBI) Includes checklists for various studies such as analytical cross-sectional studies, case-control studies, case reports, case series, cohort studies, diagnostic test accuracy studies, prevalence studies, quasi-experimental studies, randomized controlled trials, and systematic reviews.
The Critical Review Form - Quantitative Studies McMaster Occupational Evidence-based Practice Group The form is available for download in multiple languages, see bottom of the page.
MINORS

Slim, K., Nini, E., Forestier, D. et al.

Methodological Index for Non-Randomized Studies (MINORS).

A tool for assessing non-randomized interventional studies.  

NOS The Ottawa Hospital Research Institute. Wells, G.A.,Shea, B., O'Connell, D., et al.

The Newcastle-Ottawa Scale (NOS)

Most widely used for case control or cohort studies. modified scale for cross-sectional studies was made based on this article.

ROBINS-I Centre for Research Synthesis and Decision Analysis, University of Bristol

Risk of Bias for non-randomized (observational) studies or cohorts of Intervention.

Useful for conducting systematic reviews that include non-randomized studies.

ROBINS-E Centre for Research Synthesis and Decision Analysis, University of Bristol

Risk of Bias for non-randomized (observational) studies or cohorts of Exposures

Useful for conducting systematic reviews that include non-randomized studies.

Study Quality Assessment Tools National Heart, Lung, and Blood Institute (NHLBI) Study Quality Assessment Tools for Controlled Intervention Studies, Systematic Reviews & Meta-Analyses, Observational Cohort and Cross-Sectional Studies, Case-Control Studies, Before-After (Pre-Post) Studies With No Control Group, Case Series Studies.


Animal Research

Tool Author(s)/Organization Description
ARRIVE guidelines National Centre for the Replacement Refinement & Reduction of Animals in Research (NC3RS)

The Animal Research: Reporting of In Vivo Experiments (ARRIVE) guidelines.

A checklist for improving the reliability and quality of reporting for animal research. 

Critical Appraisal of Studies Using Laboratory Animal Models O’Connor., A.M., & Sargeant, J.M. Includes two tables covering the components of critical appraisal.
SYRCLE's RoB tool

Hoojimans, C.R., Rovers, M.M, de Vries, R. et al.

Systematic Review Centre for Laboratory Animal Experimentation (SYRCLE)

risk of bias assessment tool for assessing animal studies.


Diagnostic Accuracy

Tool Author(s)/Organization Description
CASP: Diagnostic Test Checklist Critical Appraisal Skills Programme (CASP), Public Health Resource Unit, NHS, England methodological checklist that provides key criteria relevant to diagnostic test studies.
Diagnostic Accuracy Studies Worksheet The Centre for Evidence-Based Medicine (CEBM), University of Oxford Helps assess the validity and applicability of included diagnostic tests.
QUADAS-2

Whitling, P.F., Rutjes, A.W., Westwood, M.E. et al.

QUADAS-2 Group.

A revised tool for the quality assessment of diagnostic accuracy studies.

The current version of QUADAS and the tool that we recommend for use in systematic reviews to evaluate the risk of bias and applicability of primary diagnostic accuracy studies.

 

STARD 2015 Checklist Equator Network

Standards for the Reporting of Diagnostic accuracy studies (2015) 

Helps determine the accuracy and completeness of reporting of studies of diagnostic accuracy. The STARD Statement consists of a 30 item checklist and flow diagram.

(Source: Mayo Clinic Libraries, 2021)