Systematic Reviews

What is a Systematic Review?

“A systematic review uses systematic and explicit methods to identify, select, critically appraise, and extract and analyze data from relevant research” – Cochrane Handbook


Systematic Review Process Diagram

Welcome to the interactive process map.

This tool was developed by the St. Michael’s hospital KT program. We have a defined and rigorous process for all our systematic reviews.

This process map takes you through the steps to conduct a systematic review. Click on the individual steps for more information about each step.

Systematic review and network meta-analysis process diagram
Develop Research Question using PICOST Determine Eligibility Criteria Plan Literature Search Perform Literature Search Starting a Systematic Review Level 1 Screening: Titles and Abstracts Level 2 Screening: Full Text Articles Develop Data Abstraction Form Abstract Data Critical Appraisal and Assessment Summarize Study and Patient Characteristics Synthesize the Results Presentation of Results Prepare Manuscript End-of-Review Knowledge Translation Systematic Review Protocol Development

Plan Literature Search

• Ask a librarian to develop the search to ensure relevant keywords and search strategies are used for a comprehensive search; peer-review by another librarian


Additional Resources

Cochrane Handbook Section 2.6.4: Designing search strategies

PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. McGowan J et al. 2016. J Clin Epidemiol.

Level 1 Screening: Titles and Abstracts

• Pilot test to assess accuracy and usability of eligibility criteria developed for the Level 1 screening process with a small sample of studies

• Screening of all titles and abstracts by two independent reviewers. Conflicts usually resolved by discussion


Additional Resources

Cochrane Handbook Section 2.7.2: Selecting studies 

Centre for Reviews and Dissemination: 1.3.2 Study Selection

Identifying relevant studies for systematic reviews. Dickersin et al. 1994. BMJ.

The science of reviewing research. Oxman et al. 1993. Ann N Y Acad Sci.

The art and science of study identification: a comparative analysis of two systematic reviews. Rosen et al. 2016. BMC Med Res Methodol.

Develop Data Abstraction Form

Critical Appraisal and Assessment

• Critical appraisal is the assessment of evidence by systematically reviewing its relevance, validity, and results.


Additional Resources

Cochrane Handbook Section 2.12.2: Assessing the quality of a body of evidence

Cochrane Handbook Section 2.8: Assessing risk of bias in included studies

Centre for Reviews and Dissemination: 1.3.4 Quality Assessment

Critical Appraisals Skills Programme (CASP)

Cochrane Effective Practice and Organisation of Care Group

Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods. Dixon WM et al.  2007. J Health Serv Res Policy.

Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Sanderson S et al. 2007. Int J Epidemiol.

The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. Higgins JP et al. 2011. BMJ.

The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Wells G et al. 2000.

AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Shea BJ et al. 2009. J Clin Epidemiol.

Few systematic reviews exist documenting the extent of bias: a systematic review. Tricco AC et al. 2008. J Clin Epidemiol.

AGREE Next Steps Consortium. AGREE II: advancing guideline development, reporting and evaluation in health care. Brouwers MC et al. 2010. J Clin Epidemiol. 

ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Sterne et al. 2016. BMJ. 

GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. Guyatt et al. 2011. J Clin Epidemiol.

Summarize Study and Patient Characteristics

• Use tables and text to describe study and patient characteristics


Additional Resources

Cochrane Handbook Section Characteristics of included studies tables


Download: Systematic Review Process Map (.pdf, 198kb)


Examples of methodology projects conducted by our team

  1. Robson, R. C., Pham, B., Hwee, J., Thomas, S. M., Rios, P., Page, M. J., Tricco, A. C. (2019). Few studies exist examining methods for selecting studies, abstracting data, and appraising quality in a systematic review. Journal of Clinical Epidemiology, 106, 121-135. doi: 10.1016/j.jclinepi.2018.10.003.
  2. Tricco, A. C., Cogo, E., Page, M. J., Polisena, J., Booth, A., Dwan, K., … & Moher, D. (2016). A third of systematic reviews changed or did not specify the primary outcome: A PROSPERO register study. Journal of Clinical Epidemiology, 79, 46-54. doi: 10.1016/j.jclinepi.2016.03.025.
  3. Tricco, A. C., Straus, S. E., & Moher, D. (2011). How can we improve the interpretation of systematic reviews? BMC Medicine, 9(31). doi: 10.1186/1741-7015-9-31.


Examples of Systematic Reviews conducted by our team

  1. Ashoor, H. M., Lillie, E., Zarin, W., Pham, B., Khan, P. A., Nincic, V., … & Tricco, A. C. (2017). Effectiveness of different compression-to-ventilation methods for cardiopulmonary resuscitation: A systematic review. Resuscitation, 118, 112-125. doi: 10.1016/j.resuscitation.2017.05.032.
  2. Tricco, A. C., Soobiah, C., Hui, W., Antony, J., Struchkov, V., Hutton, B., … & Straus, S. E. (2015). Interventions to decrease the risk of adverse cardiac events for patients receiving chemotherapy and serotonin (5-HT3) receptor antagonists: A systematic review. BMC Pharmacology and Toxicology, 16(1). doi: 10.1186/2050-6511-16-1.