Skip to main content

Literature reviews: The process of reviewing

“Don’t let other authors hog the lime-light. It’s your work and you need to tell it how it is for you..”

Pat Thomson, Patter blog

When reviewing the literature, it is important to make sure you do not produce a "she said, he said" list of what the authors of the papers did and found. Reviewing involves searching, selecting, evaluating and synthesising the literature not just reporting on what is says.


Searching the literature

Whether you are undertaking a traditional or systematic literature review it is important that you approach your searching in an organised and methodical manner.

  • Search appropriate academic journal databases.* These can be found on the 'Articles & Journals' tab in your appropriate Subject LibGuide.
  • Develop a search strategy that will look for the key concepts in your question. This includes deciding and refining your search terms but also initial filtering such as by language or date published. Help with Planning and saving a search strategy is found in the next section of this guide. 

Selecting the papers to use

You will not need to read all of the papers that your search finds, so don't worry if you find hundreds. If you are undertaking a systematic review, you will have particular criteria for including and excluding papers (which you will have to document in your methodology - see the specific guidance on this). For a traditional review, you can be more flexible. With both types of review, the process usually involves:

  • Reading the title of the paper - often this is enough for you to decide if you are going to include it or not. It could be out of the scope of your question due to criteria like location, sector or target group which may be mentioned in the title.
  • Scanning the abstract of the paper - this is the author or publisher's own summary of the paper and should be enough for you to decide whether it is of interest to your question or not. At this stage you may wish to filter by more specific criteria such as methodology which is also usually mentioned in the abstract. This should hopefully get you down to a reasonably number of papers to read. There is no correct number of papers, for undergraduate reviews it could be anything from 5 to 100—you just have to be confident that your selection criteria will have identified the relevant papers.
  • Reading the selected papers - for systematic reviews, you will include all of your selected papers; for a traditional review you will decide on your final papers to use following your reading by looking for themes etc. We suggest you look at our four step approach to reading journal articles guide for the most effective way to read a journal article.

Visual representation of previous text


*For postgraduate students and academic staff undertaking a systematic review, you will also need to search grey literature (see the Systematic Reviews Guide). This is not necessary for undergraduate reviews, even systematic ones. 


Evaluating and synthesising the literature

Although these are given here as two different processes, they are often done together. There is no right or wrong order to do this in either. Some times you will evaluate several sources and then bring them together through synthesis, other times you will synthesise several sources together and then evaluate them as a group.

Evaluating the literature

There are two main ways of evaluating the literature:

  1. Evaluating the worth of an individual paper
  2. Evaluating how useful a paper is to your project or argument

Both are important but many students do too much of the first and not enough of the second.

Evaluating the worth of an individual paper

Not all published or even peer reviewed papers are perfect. Evaluating a paper means looking for both strengths and possible weaknesses or limitations. Sometimes the author has helped you by pointing out what they see as the limitations of their own work (usually after the discussion section) but often you have to decide on these yourself. Approach this with confidence, your thoughts matter. It is part of the process of academic research and authors expect it to happen.

When evaluating a paper, ask yourself the following questions:


person with a halo

Who is the author and what are their credentials/affiliations? Here you are looking to see if the author is credible and making sure they do not have any biases because of their personal interests. It is worth quickly googling them to see if they are still at the same institution (or have moved into industry or something where their findings may mean they gain financially for example). It is also worth googling their institution. Not all have the best reputations or are without their own agendas.


journal with a halo

How reputable is the journal and how well received is the article? Check out the journal's impact factor/ranking (see either SJR or Scopus) to see if it is prestigious or influential. Check it out in relation to other journals in your subject area rather than overall (some disciplines such as medicine have much higher impact factors). Also check how many times the article has been cited (you can do this in Web of Science or get a rough idea in Google Scholar). Don't use this as a definitive guide however - some poor papers are cited a lot to point out how bad they are and older papers have obviously had more time to be cited.


thumbs up icon

Are the results reliable? Are the conclusions backed up by the evidence? Was the methodology suitable? Were there any design or analysis flaws? In quantitative studies, is the sample size big enough for generalisation? Was there a control group? Were participants assigned to groups appropriately? In qualitative studies, were interviews piloted? Does the researcher declare their relationship with the participants? Were results triangulated for verification?


Evaluating a paper's usefulness to your argument or study

One of the most important parts of a literature review is deciding which papers are the most relevant to your overall argument (for stand alone reviews) and/or to position your study within the rest of the literature (for research papers, independent projects, dissertations or theses). Remember for a literature review your argument is about the literature rather than supported by it as in an essay.

Here are some questions you could ask yourself: 


calendar icon

Is the article current (or timeless)? Decide if the article is up-do-date and check to see if has it been superseded by more recent research. In Google Scholar or Web of Science for example, you can see which other articles have cited a particular paper so you may be able to quickly see if there is more recent research on the topic. In some disciplines, older, seminal works may be considered timeless and still relevant. If you are structuring your review chronologically you will be specifically looking for papers from particular time frames.


Venn diagram icon

How relevant is it? This is very subjective and can depend on how much literature there is on your topic. If there is a lot, you can afford to be very specific about the context you are looking for. For example, in business, is it concerned with the right sector? In education, is it concerned with the appropriate level of study? If there is very little specific literature, you will need broader criteria. For example, is it in a related field? Is it using a similar methodology?


hand supporting box icon

Does it/could it support the argument I am developing? As you scope out the literature you will start to develop an idea of the argument you are going to make about it. Decide if the paper is useful in supporting that wholeheartedly or whether it is adding extra points and nuances which will need building into the argument. Also consider, is this a better paper for supporting my argument than others I have read? In traditional reviews, some relevant papers will not make it into your final review if others are better examples.


thumbs down icon

Does it/could it refute my argument? It is not always necessary (despite common misconceptions), but sometimes it can be good to find a paper that disagrees with your developing argument. If this is particularly persuasive, you could change your mind and alter the idea you have for your overall argument. If you don't find it persuasive then it will be useful for showing you have considered other possible perspectives but then explain why you do not agree.


magnifying glass with exclamation mark

Does it/could it provide examples I can use? Can you use its methods (scientific/interviews/surveys etc) to back up your own choices? Can you use particular parts of its findings to support a particular part of your argument?


​Synthesising the literature

A review is more than a list of summaries of the papers you have found. Instead, you need to look for the connections and relationships between the sources. As you read, you are looking for: 


Agreements. What do the sources agree on? What key sources come up regularly? What key theories or concepts occur in several papers? Are there common methodological choices?

Disagreements. What are the main points of contention? How do these impact on your topic? Which side do you agree with and why? 

Themes. What topics appear across more than one paper? Which of these are the most relevant to your research or position? What are the differences/nuances in the way the topics are presented?

Gaps. Is there anything that is not being covered? Are there areas that are less well covered than others? Are there perspectives that are not being considered?

Developments. How do sources build on each other? Do they refer to each other, if so how? Do they follow similar methodologies or adapt them?

Directions. How has the topic developed over time? What direction does it look like research is heading? What new avenues are opening up? What are current trends?


Using a synthesis matrix

To help with the process of sythesising you can use a table or a spreadsheet to organise your information and your thoughts. There are different ways to organise them but here are a few examples:

Thematic matrix

This is organised by theme - you may already have some in mind when you begin or you may just add more as you work. Add things when you think they may be a theme - you can always remove one if you realise it was actually only in one paper.

Theme Smith, 2015 Jones et al. 2017 Green & Blackwell, 2016 White, 2012
A

What Smith said about theme A. 
Your thoughts on it (why this is relevant/useful).

What is said about theme A.
Your thoughts on it.

What is said about theme A.
Your thoughts on it.

What White said about theme A.
Your thoughts on it.

B

What Smith said about theme B.
Your thoughts on it.

What is said about theme B.
Your thoughts on it.

What is said about theme B.
Your thoughts on it.

What White said about theme B.
Your thoughts on it.

C

What Smith said about theme C.
Your thoughts on it.

What is said about theme C.
Your thoughts on it.

What is said about theme C.
Your thoughts on it.

What White said about theme C.
Your thoughts on it.

Source matrix

If you are doing a more systematic style review, then something like this may be more helpful:

Source Problem/purpose Design Sample Methods Findings Implications Limitations
Paper 1              
Paper 2              
Paper 3              
Personal matrix

This is your review, so you can actually make your columns fit the needs of your topic or study. For example:

Source Country Context Focus Type Methods Main findings
Chan (2013) China Higher Education, Nursing Learner perspectives Qualitative Focus groups with concept maps Meaning of creativity similar ot literature (out of the box, uncommon, non-traditional. Also associated with happiness. Mixed ideas of need for practicality.
Maksić &
Pavlović (2011)
Serbia

Secondary School & Higher Education, Multidisciplinary

Expert perspectives Qualitative Survey with open ended questions Meaning of creativity similar to literature (originality, novelty and difference). Everyday descriptions given rather than 'eminent'. Associated with creative person rather than product.
Dewitt & Gruys (2007) USA

Higher Education, Business (MBA)

Student perceptions and abilities Quantitative Before and after creativity tests Perceptions of importance and beliefs in creative abilities rose after intervention as did creativity scores.

Synthesised writing

Once you have decided on themes, you need to make sure you are writing in a way that truly synthesises rather than lists papers under specific themes. Look at the following two examples:

Listing under a theme (bad)

The effect of the environment on creative potential has been recognised by several different researchers. Amabile et al. (1996) identified that the workplace could be adapted to encourage creative growth. Schawlaw, (1997) suggested that motivation is the key factor and Dewett (2007) agreed with these findings. McCoy and Evans (2002) recognized the physical environment as playing the most significant role and Schepers and Van den Berg (2007) argued that social factors were more important.

Notice that in above example, the sentences and clauses begin with the authors, making them the subjects. This is typical of poor synthesis and your review becomes little more than he said/she said. The example below only has one sentence that starts with an author and that is a leading author in the field whose voice therefore carries more weight.

Synthesis (good)

The effect of the environment on creative potential has been recognised by several different researchers. In a much quoted study, Amabile et al. (1996) identified that the workplace could be adapted to encourage creative growth and later went on to identify that the role of leadership as the most significant factor (Amabile et al., 2004). However, other researchers have argued that alternative factors are more important; motivation (Schawlaw, 1997; Dewett, 2007), physical environment (McCoy & Evans, 2002) and social factors (Schepers & Van den Berg, 2007) have all been identified. These aspects clearly play a role, but it can be argued that leadership influences each of these and therefore is still the single most significant factor. The leadership team of CLD Ltd will therefore be interviewed...

Another example:

Author as subject of sentence (bad)

Cammaerts et al. (2015) and Dahl et al. (2018) suggested young people are intimidated by and distrust the formal model of political engagement. However, according to Henn and Foard (2014) this is not uniform and varies by gender, and socio-economic status and level of education.

Topic/theme as subject (good)

The formal model of political engagement is thought to intimidate young people (Cammaerts et al., 2015) and create distrust (Dahl et al., 2018). However, the picture is not uniform and is influenced by gender, socio-economic status and education level (Henn & Foard, 2014). This suggests that…