There are a variety of fee-based and open-source (i.e., free) tools available for conducting the various steps of your scoping or systematic review.
The NIH Library currently provides free access for NIH customers to Covidence. At least one user must be from NIH in order to request access and use Covidence. Please contact the NIH Library's Systematic Review Service to request access.
You can use Covidence to import citations from any citation management tool and then screen your citations at title and abstract and then full text levels. Covidence keeps track of who voted and manages the flow of the citations to ensure the correct number of screeners reviews each citation. It can also support single or dual screeners. In the full text screening step, you can upload PDFs into Covidence and it will keep track of your excluded citations and reasons for exclusion. Later, export this information to help you complete the PRISMA flow diagram. If you chose, you can also complete your data extraction and risk of bias assessments in Covidence by creating templates based on your needs and type of risk of bias tool. Finally, export all of your results for data management purposes or export your data into another data analysis tool for further work.
Other tools available for conducting scoping or systematic reviews are:
- DistillerSR (fee-based)
- EPPI-Reviewer (fee-based)
- JBI SUMARI (from the Joanna Briggs Institute for their reviews) (fee-based)
- LitStream (from ICF International)
- Systematic Review Data Repository (SRDR+) (from AHRQ) (free)
- Abstrackr (open source)
- Colandr (open source)
- Google Sheets and Forms
- HAWC (Health Assessment Workplace Collaborative) (free)
- Rayyan (open source)
And check out the Systematic Review Toolbox for additional software suggestions for conducting your review.
Quality Assessment Tools (i.e., risk of bias, critical appraisal)
- 2022 Repository of Quality Assessment and Risk of Bias Tools - A comprehensive resource for finding and selecting a risk of bias or quality assessment tool for evidence synthesis projects. Continually updated.
- AMSTAR 2 - AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews). Use for critically appraising ONLY systematic reviews of healthcare interventions including randomised controlled clinical trials.
- JADAD Scale for Reporting Randomized Controlled Trials - The Jadad scale, sometimes known as Jadad scoring or the Oxford quality scoring system, is a procedure to independently assess the methodological quality of a clinical trial. Jadad et al. published a three-point questionnaire that formed the basis for a Jadad score.
- Joanna Briggs Institute Critical Appraisal Tools - includes 13 checklists to appraise a variety of different studies and publication types including qualitative studies.
- RoB 2.0: Cochrane Risk of Bias Tool for Randomized Trials Version 2 of the Cochrane RoB 2 can be used to assess the risk of bias in randomized trials.
- CASP (Critical Appraisal Skills Program) - a number of checklists are available to appraise systematic reviews, randomised controlled trials, cohort studies, case control studies, economic evaluations, diagnostic studies, qualitative studies and clinical prediction rule.
- Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomised Studies in Meta-Analyses - Nonrandomised studies, including case-control and cohort studies, can be challenging to implement and conduct. Assessment of the quality of such studies is essential for a proper understanding of nonrandomised studies. The Newcastle-Ottawa Scale (NOS) is an ongoing collaboration between the Universities of Newcastle, Australia and Ottawa, Canada. It was developed to assess the quality of nonrandomised studies with its design, content and ease of use directed to the task of incorporating the quality assessments in the interpretation of meta-analytic results.
Background information on this important step of systematic reviews can be found at the following resources:
- Cochrane Handbook for Sysetmatic Reviews of Interventions (version 6.2) 2021 - see Chapter 7: Considering bias and conflicts of interest among the included studies, Chapter 8: Assessing risk of bias in a randomized trial, Chapter 13: Assessing risk of bias due to missing results in a synthesis and Chapter 25: Assessing risk of bias in a non-randomized study
- Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. JBI, 2020. https://doi.org/10.46658/JBIMES-20-01 - see appropriate chapter for type of review and the section on risk of bias.
- Chapter: Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Healthcare Interventions from AHRQ. 2017 December. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville, MD, AHRQ.
- Chapter 3: Standards for Finding and Assessing Individual Studies in Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press.
- GRADE Working Group
The working group has developed a common, sensible and transparent approach to grading quality of evidence and strength of recommendations.
- Oxford Centre for Evidence Based Medicine - Levels of Evidence and Grades of Recommendations
Reporting Standards for Systematic Reviews
The Appraisal of Guidelines for Research and Evaluation (AGREE) Instrument evaluates the process of practice guideline development and the quality of reporting.
Collects guidance documents on reporting systematic reviews and other types of health research
- PRISMA 2020
Preferred Reporting Items for Systematic Reviews and Meta-Analyses. PRISMA 2020 was published in 2021 with an revised checklist, flow diagram, and a new elaboration and explanation paper.
The Methodological Expectations of Cochrane Intervention Reviews (MECIR) are methodological standards to which all Cochrane Protocols, Reviews, and Updates are expected to adhere
- RAMESES publication standards: meta-narrative reviews
Online Videos on Systematic Reviews
- The Campbell Collaboration
A collection of introductory and advanced videos on systematic reviews
- Cochrane Introduction to Systematic Reviews
This module provides an overview to Cochrane systematic reviews, and will take approximately 45 minutes to complete.
- Systematic Review and Evidence-Based Medicine
Dr. Aaron Carroll (The Incidental Economist) take on evidenced-based practice and systematic reviews
- How to Critically Appraise Evidence
A collection of videos on evidence-based practice, common statistical methods in medicine, and systematic reviews
- Introduction to Meta-Analysis
Dr. Michael Borenstein short introduction to meta-analysis