Reference management software such as Zotero, EndNote and Procite allow for the manual entry of references and automated importing of search results from many databases. It makes the production of reference lists easier.
By creating a 'library' (database) of references, information can be shared by the whole review team, duplicated references can be identified and deleted more easily, and customised fields can be created where ordering decisions can be recorded. It is possible to construct a database of references using a database package such as Microsoft Access, a word processing package or a more specialised programme as EPPI-Reviewer, a web-based review management programme which incorporates reference management functions.
EndNote, Procite and EPPI-Reviewer are fee based softwares whereas Zotero is free.
In this step, there are at least two reviewers who choose which articles are relevant or not for the systematic review. Text mining tools can help to select the articles, but a model must be built based on previous eligibility decisions, which is able to predict whether a given citation is likely to be relevant or irrelevant in a given review. The technology thus cannot automate the whole process, as it needs a representative sample of included and excluded citations.
Text mining has the potential to reduce screening workload in systematic reviews and to make the screening process more efficient. It should be safe to use for ranking the order in which citations are screened to facilitate the early identification of relevant studies.
Tools for article screening:
- Covidence
A web-based software platform that streamlines the production of systematic reviews, including Cochrane Reviews. Nonprofit organization, open source software. First systematic review free for 2 reviewers, then fee-based.
- Rayyan
A free web application developed at Qatar Computing Research Institute to help with systematic reviews. Authors create systematic reviews, collaborate on them, maintain them over time and get suggestions for article inclusion.
For more information on Rayyan, see this guide from the Mcgill University Library https://libraryguides.mcgill.ca/rayyan/gettingstarted
- Abstrackr
A free, open-source tool for facilitating the citation screening process. Upload your abstracts, invite reviewers, and get to screening.
- EPPI reviewer
Developed and maintained by the EPPI-Centre at the Social Science Research Unit of the UCL Institute of Education, University of London. Fee based. Free trial.
An important step of systematic reviews is the evaluation of the studies selected. It consists on checking the quality, reliability and relevance of the studies in the review.
The critical appraisal process will consider:
Standardised tools available for critical appraisal
- Critical Appraisal Skills Programme (CASP) has checklists for Systematic Reviews, Randomised Controlled Trials, Cohort Studies, Case Control Studies, Economic Evaluations, Diagnostic Studies, Qualitative studies and Clinical Prediction Rule.
- BMJ Best Practice - Critical Appraisal Checklists
- Cochrane RoB 2 - A revised tool to assess risk of bias in randomized trials
- The software EPPI reviewer includes also standardised tools (checklists) which can help you to evaluate the quality of each study.
To know more about critical appraisal, see : Burls, A. (2009). What is critical appraisal? Hayward Medical Communications
What information to record?
Data extraction tools
- Covidence: A web-based software platform. Nonprofit organisation, open source software. First systematic review free for 2 reviewers, then fee-based.
Fonctionnalities:
- Distiller: A fee-based software.
Fonctionnalities:
- EPPI reviewer: developed and maintained by the EPPI-Centre at the Social Science Research Unit of the UCL Institute of Education, University of London. Fee based. Free trial
Fonctionnalities :
The systematic review toolbox is a useful platform where you can look information on different systematic reviews software.
Synthesis is the process of combining the results of individual research studies to answer a systematic review question. It can take the shape of a written narrative, a table, or graphical plots, including statistical meta-analysis.
If the studies you have included in your evidence synthesis are not similar (e.g. you have included different research designs), a meta-analysis is not possible. In that case, you can synthesise the data using a process called “narrative or descriptive synthesis”.
If the studies you have included in your systematic review are sufficiently similar, you can synthesise the data using a process called “meta-analysis”, which is a statistical method to combine the results of primary studies.
Tools for conducting a meta-analysis:
- RevMan Web: Free version available. Software used for preparing and maintaining Cochrane Reviews. It can perform meta-analysis of the data entered, and present the results graphically.
- EPPI reviewer: developed and maintained by the EPPI-Centre at the Social Science Research Unit of the UCL Institute of Education, University of London. Fee based. Free trial. It can perform meta-analysis of the data entered.
Tool for teaching or learning how to conduct a meta-analysis:
- MetaLight: a software application designed to support the teaching and learning of meta-analysis. It is freely available and can be run on about any PC.
Systematic Evidence Map:
The systematic evidence map is a visual representation that describes the nature of research in an area. It can be a component of a systematic review that helps to identify where little or no relevant research has been done and where new studies are needed.
To go further on the synthesis of qualitative research:
Barnett-Page, E., James, T. "Methods for the synthesis of qualitative research: a critical review". BMC Medical Research Methodology 9, 59 (2009). https://doi.org/10.1186/1471-2288-9-59.