Systematic Reviews

What is a Systematic Review?

“A systematic review uses systematic and explicit methods to identify, select, critically appraise, and extract and analyze data from relevant research" - Cochrane Handbook


Systematic Review Process Diagram

Welcome to the interactive process map.

This tool was developed by the St. Michael’s hospital KT program. We have a defined and rigorous process for all our systematic reviews.

This process map takes you through the steps to conduct a systematic review. Click on the individual steps for more information about each step.

Plan Literature Search

• Ask a librarian to develop the search to ensure relevant keywords and search strategies are used for a comprehensive search; peer-review by another librarian


Additional Resources

Cochrane Handbook Section 2.6.4: Designing search strategies

PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. McGowan J et al. 2016. J Clin Epidemiol.

Level 1 Screening: Titles and Abstracts

• Pilot test to assess accuracy and usability of eligibility criteria developed for the Level 1 screening process with a small sample of studies

• Screening of all titles and abstracts by two independent reviewers. Conflicts usually resolved by discussion


Additional Resources

Cochrane Handbook Section 2.7.2: Selecting studies 

Centre for Reviews and Dissemination: 1.3.2 Study Selection

Identifying relevant studies for systematic reviews. Dickersin et al. 1994. BMJ.

The science of reviewing research. Oxman et al. 1993. Ann N Y Acad Sci.

The art and science of study identification: a comparative analysis of two systematic reviews. Rosen et al. 2016. BMC Med Res Methodol.

Develop Data Abstraction Form

Critical Appraisal and Assessment

• Critical appraisal is the assessment of evidence by systematically reviewing its relevance, validity, and results.


Additional Resources

Cochrane Handbook Section 2.12.2: Assessing the quality of a body of evidence

Cochrane Handbook Section 2.8: Assessing risk of bias in included studies

Centre for Reviews and Dissemination: 1.3.4 Quality Assessment

Critical Appraisals Skills Programme (CASP)

Cochrane Effective Practice and Organisation of Care Group

Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods. Dixon WM et al.  2007. J Health Serv Res Policy.

Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Sanderson S et al. 2007. Int J Epidemiol.

The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. Higgins JP et al. 2011. BMJ.

The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Wells G et al. 2000.

AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Shea BJ et al. 2009. J Clin Epidemiol.

Few systematic reviews exist documenting the extent of bias: a systematic review. Tricco AC et al. 2008. J Clin Epidemiol.

AGREE Next Steps Consortium. AGREE II: advancing guideline development, reporting and evaluation in health care. Brouwers MC et al. 2010. J Clin Epidemiol. 

ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Sterne et al. 2016. BMJ. 

GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. Guyatt et al. 2011. J Clin Epidemiol.

Summarize Study and Patient Characteristics

• Use tables and text to describe study and patient characteristics


Additional Resources

Cochrane Handbook Section Characteristics of included studies tables


Download: Systematic Review Process Map (.pdf, 198kb)


Examples of methodology projects conducted by our team

  1. Tricco, A. C., Cogo, E., Page, M. J., Polisena, J., Booth, A., Dwan, K., ... & Moher, D. (2016). A third of systematic reviews changed or did not specify the primary outcome: A PROSPERO register study. In Press, Journal of Clinical Epidemiology. doi: 10.1016/j.jclinepi.2016.03.025.
  2. Tricco, A. C., Straus, S. E., & Moher, D. (2011). How can we improve the interpretation of systematic reviews? BMC Medicine, 9(31). doi: 10.1186/1741-7015-9-31.


Examples of Systematic Reviews conducted by our team

  1. Tricco, A. C., Soobiah, C., Berliner, S., Ho, J. M., Ng, C. H., Ashoor, H. M., ... & Straus, S. E. (2013). Efficacy and safety of cognitive enhancers for patients with mild cognitive impairment: A systematic review and meta-analysis. Canadian Medical Association Journal, 185(16), 1393-1401. doi: 10.1503/cmaj.130451.
  2. Tricco, A. C., Lillie, E., Soobiah, C., Perrier, L., & Straus, S. E. (2012). Impact of H1N1 on socially disadvantaged populations: systematic review. PloS one, 7(6), e39437. doi: 10.1371/journal.pone.0039437.