• Population of interest

• Intervention to examine

• Comparator(s)

• Outcome of interest

• Study design(s)

• Time limitations

Additional Resources

Cochrane Handbook Section 2.5: Defining the review question and developing criteria for including studies

Popping the (PICO) question in research and evidence- based practice. Stone PW. 2002. Appl Nurs Res.

The well-built clinical question: a key to evidence based decisions. Richardson WS et al. 1995. ACP J Club.

Formulating questions and locating primary studies for inclusion in systematic reviews. Counsell C. 1997. Ann Intern Med. 

• Develop a research question based on PICOST

Additional Resources

Cochrane Handbook Section 2.5: Defining the review question and developing criteria for including studies

Centre for Reviews and Dissemination: 1.2.2.2 Review question and inclusion criteria and 1.2.2.3 Defining inclusion criteria

• Ask a librarian to develop the search to ensure relevant keywords and search strategies are used for a comprehensive search; peer-review by another librarian

Additional Resources

Cochrane Handbook Section 2.6.4: Designing search strategies

PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. McGowan J et al. 2016. J Clin Epidemiol.

• Search at least two databases, difficult to locate or “grey” literature, hand search key journals, ask experts, and scan reference lists

Additional Resources

Cochrane Handbook Section 2.6.2: Sources to search

CADTH: Grey matters: a practical search tool for evidence-based medicine

Analysis of the reporting of search strategies in Cochrane systematic reviews. Yoshii A et al. 2009.  J Med Libr Assoc.

Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. Golder et al. 2008. J Clin Epidemiol.

Influences on the outcome of literature searches for integrative research reviews. Cooper et al. 1989. Sci Commun.

• Identify knowledge gap and research question with knowledge users

• Involve key stakeholders during the review process (i.e. integrated knowledge translation)

Additional Resources

Cochrane Handbook Section 1.2.1: The need for systematic reviews

Cochrane Handbook Section 1.2.2: What is a systematic review

Centre for Reviews and Dissemination: 1.1.1 Is a review required?

CIHR Guide to Knowledge Translation

The art and science of knowledge synthesis. Tricco AC et al. 2011. J Clin Epidemiol.

Systematic reviews from astronomy to zoology: myths and misconceptions. Petticrew M. 2001. BMJ. 

Stakeholder engagement opportunities in systematic reviews: knowledge transfer for policy and practice. Keown K et al. 2008. J Contin Educ Health Prof.

• Pilot test to assess accuracy and usability of eligibility criteria developed for the Level 1 screening process with a small sample of studies

• Screening of all titles and abstracts by two independent reviewers. Conflicts usually resolved by discussion

Additional Resources

Cochrane Handbook Section 2.7.2: Selecting studies 

Centre for Reviews and Dissemination: 1.3.2 Study Selection

Identifying relevant studies for systematic reviews. Dickersin et al. 1994. BMJ.

Oxman AD, Guyatt GH. The science of reviewing research. 1993. Ann N Y Acad Sci.

• Pilot test to assess accuracy and usability of eligibility criteria developed for the Level 2 screening process with a small sample of studies

• Screening of all full text articles by two independent reviewers. Conflicts usually resolved by discussion

Additional Resources

Cochrane Handbook Section 2.7.2: Selecting studies 

Identification of randomized controlled trials in systematic reviews: Accuracy and reliability of screening records. Edwards P et al. 2002. Statist Med.

Many scenarios exist for selective inclusion and reporting of results in randomized trials and systematic reviews. Page MJ et al. 2013. J Clin Epidemiol. 

• Pilot test the form on a small sample of studies to ensure it captures all relevant information from the studies. Revise forms as necessary

Additional Resources

Cochrane Handbook Section 2.7.3: What data to collect

Cochrane Handbook Section 2.7.5: Data collection forms

Centre for Reviews and Dissemination: 1.3.3 Data extraction

Innovations in data collection, management, and archiving for systematic reviews. Li et al. 2015. Ann Intern Med.

• Two reviewers abstract data independently and assess study quality. Conflicts usually resolved by discussion

Additional Resources

Cochrane Handbook Section 2.7.6: Extracting data from reports

Cochrane Handbook Section 2.7.7: Extracting study results and converting to the desired format 

Cochrane Handbook: Part 2, Chapter 7, Sections 7.3-7.9: Collecting data

Single data extraction generated more errors than double data extraction in systematic reviews. Buscemi N et al. 2006. J Clin Epidemiol.

Choice of data extraction tools for systematic reviews depends on resources and review complexity. Elamin MB et al. 2009. J Clin Epidemiol. 

Data extraction errors in metaanalyses that use standardized mean differences. Gøtzsche et al. 2007. JAMA.

Systematic review data extraction: Cross-sectional study showed that experience did not increase accuracy. Horton J et al. 2010. J Clin Epidemiol. 

• Critical appraisal is the assessment of evidence by systematically reviewing its relevance, validity, and results.

Additional Resources

Cochrane Handbook Section 2.12.2: Assessing the quality of a body of evidence

Cochrane Handbook Section 2.8: Assessing risk of bias in included studies

Centre for Reviews and Dissemination: 1.3.4 Quality Assessment

Critical Appraisals Skills Programme (CASP)

Cochrane Effective Practice and Organisation of Care Group

Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods. Dixon WM et al.  2007. J Health Serv Res Policy.

Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Sanderson S et al. 2007. Int J Epidemiol.

The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. Higgins JP et al. 2011. BMJ.

The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Wells G et al. 2000.

AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Shea BJ et al. 2009. J Clin Epidemiol.

Few systematic reviews exist documenting the extent of bias: a systematic review. Tricco AC et al. 2008. J Clin Epidemiol.

• Use tables and text to describe study and patient characteristics

Additional Resources

Cochrane Handbook Section 2.11.2.2: Characteristics of included studies tables

• Based on your research question and the available data, choose the appropriate method to synthesize your findings

• This may include:

Additional Resources

Cochrane Handbook Section 2.9: Analysing data and undertaking meta-analysis

Cochrane Handbook Section 3.16.6: Indirect comparisons and multiple-treatments meta-analysis

Cochrane Handbook Section 3.20: Qualitative research and Cochrane reviews

Centre for Reviews and Dissemination: 1.3.5 Data Synthesis Module 12.4

The statistical basis of meta-analysis. Fleiss JL. 1993. Stat Methods Med Res. 

Meta-analysis: formulating, evaluating, combining, and reporting. S-L. Normand. 1999. Stat Med.

Bias due to changes in specified outcomes during the systematic review process. Kirkham JJ et al. 2010. PLoS One.

Bias in meta-analytic research. Felson DT. 1992. J Clin Epidemiol.

GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. Guyatt GH et al. 2008. BMJ.

Additional Resources

Cochrane Handbook Section 2.11: Presenting results and ‘Summary of findings’ tables

Cochrane Handbook: Part 2, Chapter 12: Interpreting results and drawing conclusions file

Centre for Reviews and Dissemination: 1.3.6.3 Formulating the discussion file

Centre for Reviews and Dissemination: 1.3.6.4 Conclusions, implications, recommendations file

How can we improve the interpretation of systematic reviews? Tricco et al. 2011. BMC Med.

• Include background, methods, results, and discussion; may involve key stakeholders/audience

Additional Resources

PRISMA Statement

Cochrane Handbook Section 2.11: Presenting results and ‘Summary of findings’ tables

Cochrane Handbook Section 2.12: Interpreting results and drawing conclusions

• For example: Conference presentations, Key opinion meetings, Peer-reviewed publications

Additional Resources

CIHR Guide to Knowledge Translation

Moving knowledge to action through dissemination and exchange. Gagnon ML. 2011. J Clin Epidemiol.

Effectiveness and efficiency of guideline dissemination and implementation strategies. Grimshaw JM et al. 2004. Health Technol Assess.

Systematic Review Protocol Development based on the PRISMA statement

Additional Resources

Cochrane Handbook Section 2.1: Rationale for protocols 

Cochrane Handbook Section 2.2.2 Outline of a protocol for a Cochrane review