10: Interpreting the findings

Once you have described the different aspects of the studies in a review you need to think about ‘the bigger picture’. Exactly what this is will depend on the studies included in the review and the question the review set out to answer. In other words, the next step is to provide an interpretation of what the findings might mean, and your understanding of what you have found. There are two main aspects to this:

• addressing the review’s question or objectives
• contextualising the review within the existing evidence.

The following questions should help you think about how to interpret a review’s findings.

Can you answer the research question?

You must address your research question (or research objectives). If you can’t answer the question, it is important to say so, and to say why you can’t.

You may be only able to answer the question tentatively, so you should include something about any uncertainties in your answer. If you are fortunate, you may be able to provide a definitive answer to your question. Frequently it is not possible, particularly if you have had to restrict the number of included studies for practical reasons.

It is much more likely that you will need to give a qualified answer in some way. In doing so, you need to consider the constraints on the review and the quality of the evidence (determined through the critical appraisal of the included studies). This is important information that can be used to guide practice and future research.

What similarities and differences did you identify?

Consider each characteristic recorded in your grid, such as the study design, the participants, how an intervention (or exposure) was defined, the way outcomes were measured, the study findings and the quality appraisals.

What are the overall findings?

Keep in mind that the overall findings of the review are not restricted to the findings reported in the included studies. For example, your findings could include the identification of important gaps in existing research or practice.

• Did you identify any gaps? (What information was missing that you needed to answer your research question?)
• How might the findings affect current practice? Should existing practices change or be modified in any way?
• What should future research on this topic consider to address these gaps?

What is the nature of the evidence?

• What are the strengths of the review? (What is good about the review?)
• What are the limitations of the review? (What are the shortcomings of the review?)

Contextualising the findings

Ideally the findings of any review should be compared to what is already known about the topic. The best way to do this is to look at findings of existing reviews that have explored the same or similar questions. While there can be distinct differences in how reviews on the same topic are conducted, even when the same approach has been used more recent searches are likely to find newer studies that may affect the overall evidence.

How does the published systematic review compare
to your proposed review?

The following questions can help you decide if a published review has been done well, what is unclear and what could have been done better.

• What is the research question or research objectives? How does it compare with the question or objectives of your review?
• How are the PICO elements described?
  • What study populations were included? Do you know how many study participants were involved across all studies?
  • What interventions or exposures of interest were included?
  • What comparisons were made?
  • What outcomes were measured or reported?
• What study designs did the authors include? How many studies were included? How many of these provided data?
• Were the included studies appraised? If so, how? How does this differ from your approach?
• Did the authors of the published systematic review answer their question (or meet the research objectives)?
• Apart from the review question were there any other important findings?
• What did the review authors conclude? Did they identify any important gaps in research?
• Does the review raise any issues you hadn’t thought about?

Strengths and limitations of reviews

The strengths and limitations of the studies included in a review may be reported in the review findings, particularly if study quality is affected. However, these are different from the strengths and limitations of the review itself. Reviews are considered one of the strongest sources of evidence, so it is critical for review authors to reflect and report on the strengths and weaknesses of their actual reviews.

Strengths commonly reported in reviews include:

• Extensive search strategies
• The inclusion of high quality studies
• Findings consistent with or building on results of previous reviews
• Findings consistent across included studies
• Transparent review processes that ensure robust and replicable approach
• Provides important overview of topic
• Identifies important policy or methodological issues

Limitations of reviews frequently refer to:

• Inadequate number of study participants
• Differences in definitions and the outcomes measured across included studies
• Poor study quality