Credit: Hemingway, P., & Brereton, N. (2009). What is a systematic review. What is, 1-8.
A systematic review summarises the results of available carefully designed healthcare studies (controlled trials) and provides a high level of evidence on the effectiveness of healthcare interventions. Judgments may be made about the evidence and inform recommendations for healthcare.
These reviews are complicated and depend largely on what clinical trials are available, how they were carried out (the quality of the trials) and the health outcomes that were measured. Review authors pool numerical data about effects of the treatment through a process called meta-analyses. Then authors assess the evidence for any benefits or harms from those treatments. In this way, systematic reviews are able to summarise the existing clinical research on a topic.
The review plan
Review authors set about their task very methodically following, step by step, an advance plan called a protocol. The protocol describes the steps that will be followed when preparing a review. Cochrane protocols are published in the Cochrane Library so that people can provide comments to improve them before the actual review has been conducted. A protocol describes:
The review question
The purpose of the review is generally stated as: To assess the effects of [intervention or comparison] for [health problem] in [types of people, disease or problem], and healthcare setting if appropriate. The parts of the review question are often referred to as ‘PICO’ (Participants, Interventions, Comparisons and Outcomes). The included studies generally randomly assign participants to the intervention under investigation or the control or comparative intervention.
The review title
Titles of Cochrane reviews also have a set layout: Intervention for problem in a disease or population, and sometimes an outcome. An example is: Surgical excision margins for primary cutaneous melanoma. This is a statement of the types of population (participants in controlled clinical studies), types of interventions (and what they are compared to, even if it is no treatment), and the types of outcomes that are of interest.
Cochrane Handbook of Systematic Reviews of Interventions (Version 6, September 2018)
Step 1: Framing questions for a review
The problems to be addressed by the review should bespecified in the form of clear, unambiguous and structuredquestions before beginning the review work. Once the reviewquestions have been set, modifications to the protocol shouldbe allowed only if alternative ways of defining the populations,interventions, outcomes or study designs become apparent
Step 2: Identifying relevant work
The search for studies should be extensive. Multiple resources(both computerized and printed) should be searched withoutlanguage restrictions. The study selection criteria should flowdirectly from the review questions and be specifiedapriori.Reasons for inclusion and exclusion should be recorded
Step 3: Assessing the quality of studies
Study quality assessment is relevant to every step of a review.Question formulation (Step 1) and study selection criteria (Step2) should describe the minimum acceptable level of design.Selected studies should be subjected to a more refined qualityassessment by use of general critical appraisal guides anddesign-based quality checklists (Step 3). These detailedquality assessments will be used for exploring heterogeneityand informing decisions regarding suitability of meta-analysis(Step 4). In addition they help in assessing the strength ofinferences and making recommendations for future research(Step 5)
Step 4: Summarizing the evidence
Data synthesis consists of tabulation of study characteristics,quality and effects as well as use of statistical methods forexploring differences between studies and combining theireffects (meta-analysis). Exploration of heterogeneity and itssources should be planned in advance (Step 3). If an overallmeta-analysis cannot be done, subgroup meta-analysis maybe feasible
Step 5: Interpreting the findings
The issues highlighted in each of the four steps above shouldbe met. The risk of publication bias and related biases shouldbe explored. Exploration for heterogeneity should helpdetermine whether the overall summary can be trusted, and, ifnot, the effects observed in high-quality studies should beused for generating inferences. Any recommendations shouldbe graded by reference to the strengths and weaknesses of systematic review.
Hierarchies of evidence for questions of therapy, prevention, aetiology or harm:
Level 1a Systematic review (with homogeneity) of randomised controlled trials (RCTs)
Level 1b Individual RCT (with narrow confidence interval)
Level 1c All-or-none studies
Level 2a Systematic review (with homogeneity) of cohort studies
Level 2b Individual cohort study (including low quality RCT; eg <80% follow-up)
Level 2c ‘Outcomes’ research; ecological studies
Level 3a Systematic reviews (with homogeneity) of case-control studies
Level 3b Individual case-control study
Level 4 Case series (and poor quality cohort and case-control studies)
Level 5 Expert opinion without explicit critical appraisal, or based on physiology, bench research on 'first principles'