Emiliano Rossi ID
Cardiologist. Department of Investigation. Hospital Italiano de Buenos
Aires.
Ciudad Autónoma de Buenos Aires, Argentina.
Acta Gastroenterol Latinoam 2023;53(1):11-14
Received: 14/01/2023 / Accepted: 06/03/2023 / Published online 30/03/2023 / https://doi.org/10.52787/agl.v53i1.291
A systematic review uses explicit and documented methods to collect and synthesize the findings of studies with the aim of answering a clearly formulated research question.1
Systematic reviews have several functions, including: a) providing a synthesis of current knowledge in a disciplinary field, allowing the identification of future research priorities, b) addressing questions that could not otherwise be answered by individual studies, c) identifying research problems that should be addressed in future studies and d) generating or evaluating theories about how or why the phenomena occur.2
The publication of systematic reviews has grown exponentially in the last decades. Although they are considered high quality evidence, they are subject to biases and errors. For this reason, several organizations, including the Cochrane Collaboration, have developed methodological guides to improve quality.1, 3, 4
The process of conducting a systematic review involves a number of steps, which have been described in detail by Muka et al.5
1) Define the research question. Before beginning, the purpose of the review should be established and the research question should be clearly stated. In the case of intervention studies, using the PICO methodology will help to clearly define the population (P) included, the intervention (I) evaluated, the comparator (C), and the results (outcomes) obtained.
2) Establish the work team. The group should have members with skills in conducting literature searches and research methodology, and knowledge of the disciplinary area.
3) Define search strategies. A comprehensive search is the basis of a systematic review. Search strategies need to be written and run on different online databases to retrieve potentially eligible studies. Inadequate searches may mean not including relevant articles, which will lead to biased results. In this regard, searches must be carried out in multiple databases (e.g., MEDLINE, Embase, Google Scholar, Cochrane CENTRAL, etc.).
4) Define selection criteria. The selection criteria serve as a guide for reviewers and allow relevant studies to be identified during the article screening process. They should take into account the study design, the population included, the exposure of interest, the results and the analysis methodology.
5) Design the data collection forms. Data extraction from primary studies is a key stage, and standardized forms must be used for this purpose. The general characteristics of the study, the included population, the exposures, the methods used and the results must be recorded.
6) Write and record the review protocol. The protocol must contain the research question and the objectives of the review, the inclusion and exclusion criteria, the search strategies, and the analysis plan. To promote transparency, it is recommended that the review is registered before it is carried out on a public access platform such as Prospero (http://www.crd.york.ac.uk/PROSPERO).
7) Execute the search strategies. The search strategies applied in each database must be documented to allow their reproducibility.
8) Collect all references and abstracts. This collection must be done in a single file.
9) Remove duplicate quotes
10) Review the titles and abstracts of the articles found. Two reviewers working independently must assess the relevance of each citation and make the selection following the previously established inclusion criteria. The process of citation management during the review can be quite cumbersome, so the use of specific applications (e.g., Covidence, EPPI- Reviewer, CADIMA, etc.) is recommended.
11) Select the quotes to be included. The list of citations selected by each reviewer is compared and those that coincide are included. In case of disagreement, this can be resolved by consensus between two reviewers or by the intervention of a third reviewer.
12) Retrieve the full texts of the articles and evaluate their inclusion criteria. Once the full text of the selected articles has been obtained, either through a local institutional library, an online library or by direct request to the authors, two independent reviewers must assess whether the article meets the established inclusion criteria. In case of discrepancy between the reviewers, a procedure similar to the previous step is applied. In this step, the reasons for exclusion of each article have to be recorded.
13) Contact thematic experts. It is advisable to contact experts in the thematic area of study, in order to identify missing studies, retrieve relevant data not published in full or recalculate any necessary summary measure.
14) Look for additional references. The references of the selected articles should be reviewed in search of any relevant article that was not identified in the initial search. Also, articles that cite articles already included can be searched. Another round of checking the relevance of new citations, selecting the matching ones, retrieval of full text and evaluation of their eligibility is required.
15) Make the flowchart. A flowchart should be made detailing the number of relevant citations found through database searches, expert recommendations, and additional references; the number of studies excluded during title and abstract evaluation; the number of full texts evaluated, and the reasons for their exclusion.
16) Collect the information. Once the articles to be included in the review have been identified, the information must be extracted from each one using the previously defined form. This step should be performed independently by two reviewers.
17) Evaluate the quality of the studies and their risk of bias. Two reviewers are also involved in this process. The risk of presenting different types of biases should be considered in each primary study. For this purpose, there are tools such as RoB-2 for clinical trials, ROBINS-1 for non-randomized intervention studies, Newcastle-Ottawa Scale for prospective observational studies, QUADAS-2 for the precision of diagnostic studies, QUIPS for prognostic factors and PROBAST for studies to develop, validate or update predictive models. The quality of evidence from the included studies should be reported and interpreted to allow the reader to assess how reliable the conclusions reached are.
18) Prepare the database for analysis. The information collected from each study must be prepared to allow its descriptive synthesis or meta-analysis to be carried out.
19) Prepare the descriptive synthesis. The included studies should be described by means of tables in which the author, year, place, characteristics of the included population, exposures or interventions and observed results are mentioned. If it is not possible to carry out a meta-analysis, the direction and size of the effect observed and its consistency between the different studies should be mentioned.
20) To meta-analyze. Meta-analysis is a statistical technique used to synthesize results when the effect estimates from the studies and their variances are available, generating a quantitative summary of the results.2 Its potential advantages are 1) to improve precision, since the combination of different studies increases the sample size allowing for greater statistical power to be achieved, 2) to answer questions not raised by individual studies by making it possible to investigate the consistency of the effect in a wide range of populations and interventions, and 3) to resolve controversies arising from apparently contradictory studies, or to generate new hypotheses.1
The most widely used procedure in a meta-analysis is the inverse variance method. In it, the weight given to each study is weighted by the inverse of its variance (1/standard error2). Therefore, larger studies, which have smaller standard errors, will carry more weight than smaller studies, which have larger standard errors. As a consequence, the imprecision of the estimate of the pooled effect is minimized.1
Meta-analysis results are presented in a diagram called a forest plot. It shows the effect estimates, with their confidence intervals, both for the individual studies and for the overall study.1 The variability in the effects of the intervention evaluated in the different studies is called statistical heterogeneity and is a consequence of clinical and/or methodological diversity. Clinical diversity is the variability in the participants, interventions, and outcomes studied. Methodological diversity is the variability in study design, outcome measurement tools, and risk of bias.1
The decision to combine findings from different studies through meta-analysis depends on the degree of heterogeneity found. If there is considerable variation, especially if there is inconsistency in the direction of the effect, it can be misleading to quote an average value for this effect.1
It is important to use the same estimators and to standardize their definition and coding across all included studies. Statistical heterogeneity can be assessed by different methods, of which the most widely used are the Cochrane’s Q test and the Higgins’ I2.
There are different software packages that allow meta-analysis to be performed (e. g. RevMan, Metafor, Stata, MetaXL, etc.)
21) Explore heterogeneity. Subgroup analysis, which should be predefined in the meta-analysis protocol, allows the sources of heterogeneity to be explored. Meta-regression analysis can be used to explore whether the observed heterogeneity is due to a given study characteristic. Heterogeneity should be taken into account when interpreting the results.
22) Evaluate the existence of publication biases. Publication bias occurs when the decision to publish a study is associated with its result. It is known that there is a tendency not to publish studies with negative results. This bias can be detected by means of asymmetries in the funnel plot and the Egger test.
23) Evaluate the quality of the evidence. The quality of the evidence for each outcome of the systematic review is assessed using the GRADE guidelines. Clinical trials start with the highest ranking and observational studies with the lowest. The assessment is lowered by considering study limitations, publication biases and inconsistency of results and is increased by considering if the magnitude of the effect is large and if there is a dose-response phenomenon.
24) Publish the review. There are methodological guidelines that recommend how to prepare a systematic review for publication in a transparent, complete and accurate way. These also allow readers to assess the adequacy of the methods used and the reliability of the results obtained.2
Considering if the necessary steps were followed in the elaboration of a systematic review will allow us to assess its quality.
Intellectual Property. The author declares that the data presented in the manuscript are original and were carried out at his belonging institution.
Funding. The author declares that there were no external sources of funding.
Conflict of interest. The author declares that he has no conflicts of interest in relation to this article
Copyright
© 2023 Acta Gastroenterológica latinoamericana. This is an
open-access article released under the terms of the Creative Commons Attribution (CC BY-NC-SA 4.0) license,
which allows non-commercial use, distribution, and reproduction, provided the original author and source are
acknowledged.
Cite this article as: Rossi E. Introduction to Systematic Reviews and Meta-analysis. Acta Gastroenterol Latinoam. 2023;53(1):11-14. https://doi.org/10.52787/agl.v53i1.291
Correspondence: Emiliano Rossi
Email: emiliano.rossi@hospitalitaliano.org.ar
Acta Gastroenterol Latinoam 2023;53(1):11-14