Skip to Main Content

Systematic Reviews

Processing Data

Reference management software such as Zotero, EndNote and Procite allow for the manual entry of references and automated importing of search results from many databases. It makes the production of reference lists easier.

By creating a 'library' (database) of references, information can be shared by the whole review team, duplicated references can be identified and deleted more easily, and customised fields can be created where ordering decisions can be recorded. It is possible to construct a database of references using a database package such as Microsoft Access, a word processing package or a more specialised programme as EPPI-Reviewer, a web-based review management programme which incorporates reference management functions.

EndNote, Procite and EPPI-Reviewer are fee based softwares whereas Zotero is free.

In this step, there are at least two reviewers who choose which articles are relevant or not for the systematic review. Text mining tools can help to select the articles, but a model must be built based on previous eligibility decisions, which is able to predict whether a given citation is likely to be relevant or irrelevant in a given review. The technology thus cannot automate the whole process, as it needs a representative sample of included and excluded citations.

Text mining has the potential to reduce screening workload in systematic reviews and to make the screening process more efficient. It should be safe to use for ranking the order in which citations are screened to facilitate the early identification of relevant studies.

Tools for article screening:

- Covidence
A web-based software platform that streamlines the production of systematic reviews, including Cochrane Reviews.  Nonprofit organization, open source software. First systematic review free for 2 reviewers, then fee-based

- Rayyan
A free web application developed at Qatar Computing Research Institute to help with systematic reviews. Authors create systematic reviews, collaborate on them, maintain them over time and get suggestions for article inclusion.

For more information on Rayyan, see this guide from the Mcgill University Library

- Abstrackr
A free, open-source tool for facilitating the citation screening process. Upload your abstracts, invite reviewers, and get to screening.

- EPPI reviewer
Developed and maintained by the EPPI-Centre at the Social Science Research Unit of the UCL Institute of Education, University of London. Fee based. Free trial.

An important step of systematic reviews is the evaluation of the studies selected. It consists on checking the quality, reliability and relevance of the studies in the review.

The critical appraisal process will consider:

  • Question – Does this study address a clearly focused question?
  • Methodological quality – Did the study use valid methods to address this question?  To what extent do the study design and conduct eliminate the potential for systematic error (bias)?
  • Precision – What is the likelihood of random errors? (Often depicted as the confidence interval around the result)
  • External validity – Are these valid, important results applicable to my patient or population?

Standardised tools available for critical appraisal

- Critical Appraisal Skills Programme (CASP) has checklists for Systematic Reviews, Randomised Controlled Trials, Cohort Studies, Case Control Studies, Economic Evaluations, Diagnostic Studies, Qualitative studies and Clinical Prediction Rule.

- BMJ Best Practice - Critical Appraisal Checklists

- Cochrane RoB 2 - A revised tool to assess risk of bias in randomized trials  

- The software EPPI reviewer includes also standardised tools (checklists) which can help you to evaluate the quality of each study.

To know more about critical appraisal, see : Burls, A. (2009). What is critical appraisal? Hayward Medical Communications

What information to record?

  • Use the review's research question and inclusion and exclusion criteria as a guide
  • Elements recorded will vary according to your project, but standard elements include:
    • full article citation
    • study objectives
    • study design
    • participant characteristics
    • intervention
    • comparison
    • results/outcome
    • comments related to study quality

Data extraction tools

- Covidence: A web-based software platform. Nonprofit organisation, open source software. First systematic review free for 2 reviewers, then fee-based.  


  • ​Study Selection (Citation screening, Full text review)
  • Quality Assessment (Risk of Bias assessment)
  • Data Extraction (extraction of study characteristics, export of data into RevMan)
  • Collaboration
  • Document Management 

- Distiller: A fee-based software.


  • Protocol Development
  • Study Selection
  • Quality Assessment
  • Data Extraction
  • Automated Analysis
  • Report Write Up
  • Collaboration
  • Document Management

- EPPI reviewer: developed and maintained by the EPPI-Centre at the Social Science Research Unit of the UCL Institute of Education, University of London. Fee based. Free trial

Fonctionnalities : 

  • Study Selection
  • Data Extraction
  • Automated Analysis
  • Text Analysis
  • Meta Analysis
  • Collaboration
  • Document Management

The systematic review toolbox is a useful platform where you can look information on different systematic reviews software.

Synthesis is the process of combining the results of individual research studies to answer a systematic review question. It can take the shape of a written narrative, a table, or graphical plots, including statistical meta-analysis.

If the studies you have included in your evidence synthesis are not similar (e.g. you have included different research designs), a meta-analysis is not possible. In that case, you can synthesise the data using a process called “narrative or descriptive synthesis”.

If the studies you have included in your systematic review are sufficiently similar, you can synthesise the data using a process called “meta-analysis”, which is a statistical method to combine the results of primary studies.

Tools for conducting a meta-analysis:

- RevMan Web: Free version available. Software used for preparing and maintaining Cochrane Reviews. It can perform meta-analysis of the data entered, and present the results graphically.

- EPPI reviewer: developed and maintained by the EPPI-Centre at the Social Science Research Unit of the UCL Institute of Education, University of London. Fee based. Free trial. It can perform meta-analysis of the data entered.

Tool for teaching or learning  how to conduct a meta-analysis:

- MetaLight: a software application designed to support the teaching and learning of meta-analysis. It is freely available and can be run on about any PC.

Systematic Evidence Map:

The systematic evidence map is a visual representation that describes the nature of research in an area. It can be a component of a systematic review that helps to identify where little or no relevant research has been done and where new studies are needed.

To go further on the synthesis of qualitative research:

Barnett-Page, E., James, T. "Methods for the synthesis of qualitative research: a critical review". BMC Medical Research Methodology 9, 59 (2009).