Top LitReview Systematic Screening
On this page:
“There is no 'perfect' number of papers to include in your systematic review. If your searching, screening and selection processes have been sufficiently rigorous, then you should feel confident that you have identified all relevant evidence”
Once you've conducted your search you will be ready to screen your results against your inclusion and exclusion criteria in order to determine their eligibility for the review. You can then extract your data, summarise and discuss your findings and make your conclusions.
Firstly you need to de-duplicate your total results set. Some databases such as EbscoHost will remove these automatically for you, but if you are using a variety of search platforms you will need to export to reference management software to remove the duplicates.
The next stage is to screen the titles and abstracts of the papers for relevance, also using your inclusion and exclusion criteria. It is good practice to place these into a folder if the database has that option (such as EbscoHost), if not you can export to reference management software. RefWorks or EndNote allow you to export citation information from databases and organise the articles in folders around themes before exporting them to your documents saving you a lot of work typing, copying and pasting.
For anything other than an undergraduate review, there should ideally be at least two people screening the results to avoid bias. However, time and resource may necessitate that this is not possible.
Next you should try and obtain the full text of the selected papers. If the University does not have a subscription then you may need to use the Interlibrary loan service. Contacting the authors or trying to locate a legal version via Google or networks such as Academia.edu or ResearchGate.net is another option.
Step four:Finally you need to decide whether these full text papers definitely meet your inclusion criteria. If you decide to exclude then make a note of the reasons why as you may need to document this further down the line.
The results of the screening and selection exercise should be reported in your methods section and the PRISMA flow diagram can be used to document these decisions.
Once you have identified the papers to include, you need to recognise and then extract relevant data from each individual study. You can then create relevant extraction tables which might be in the form of a summary table or synthesis matrix - see our page on The process of reviewing for examples of these.
The data you might extract includes: authors & publication year; study design; number of participants; interventions; and study outcomes.
You may use data analysis tools such as NVivo and SPSS to extract and analyse data depending on the study type.
Once you've extracted the data it's important to examine the quality of the studies. You need to assess whether the studies have been designed, conducted and reported in a way that they can be considered as reliable (rigour) and whether they provide meaningful answers to your research questions (relevance).
There are many tools available for you to use. The Critical Appraisal Skills Programme (CASP) at the Centre for Evidence Based Medicine (CEBM) in Oxford has produced a number of appraisal tools for use when appraising different types of study. CASP offers a set of 8 critical appraisal checklists which can be used for study types such as systematic reviews, randomised controlled trials and qualitative studies:
CEBM also provides this further set of Critical Appraisal Guides.
Our page on The process of reviewing also has some advice on evaluating a source's worth and usefulness to your study.
The discussion section of your review should provide a critical interpretation of the results in relation to the review question that you set out to answer. If these sections don't reflect the nature and limitations of the research process and the evidence you have presented, then you have not appropriately addressed the research question. The main components include:
What are the main findings?
How do they fit with previously published research?
What are the strengths and limitations of the included studies?
Can the findings be generalised?
What are the implications of the review?
What conclusions can be drawn from the review?