The Meta-Analysis Internship

A Practical and Interactive Course on Meta-Analaysis




 

Systematic reviews and meta-analyses present results by combining and analyzing data from different studies conducted on similar research topics. Systematic reviews and meta-analyses have been actively performed in various fields from cancer to infectious disease or social sciences. These research methods are powerful tools that can substitute the difficulties in performing large-scale randomized controlled trials. However, the inclusion of studies with any biases or improperly assessed quality of evidence in systematic reviews and meta-analyses could yield misleading results.  Meta analysis has been ranked in top of the research methods in terms of quality and validity.

It is easy to confuse systematic reviews and meta-analyses. A systematic review is an objective, reproducible method to find answers to a certain research question, by collecting all available studies related to that question and reviewing and analysing their results. A meta-analysis differs from a systematic review in that it uses statistical methods on estimates from two or more different studies to form a pooled estimate. Following a systematic review, if it is not possible to form a pooled estimate, it can be published as is without progressing to a meta-analysis; however, if it is possible to form a pooled estimate from the extracted data, a meta-analysis can be attempted. Systematic reviews and meta-analyses usually proceed according to the flowchart:

The aim of this internship program is to help the prospective intern go through all the steps of systematic review with the help of our dedicated teaching assistants and lecturer. The program takes three to 5 months. Each intern will choose a topic, the topic is evaluated for novelty by a peer in the field of the topic. If the topic is approved, a teaching assistant will be assigned to the intern to help him/her go through all the steps, including preparing the final report and the scientific article ready for submission to the proper journal.

This course is designed for health services researchers, epidemiologists, statisticians, systematic reviewers and decision analysts who are planning to perform a meta-analysis of their own concern.

The systematic review and met-analysis internship program is design to help the prospect intern to go through the all steps of choosing a proper topic, running a systematic review and meta-analysis and publishing the result. The program is divided into two main modules 1) Teaching and lecture-based modules, and 2) interactive and assistant-based modules.  The lecture-based modules include 18 hours of lectures covering the principles of systematic review, and meta-analysis and the interactive based modules include the main part of the course in which an experienced teaching assistant is assigned to intern to help him through the process of choosing a real-world topic, do the search, evidence analysis, statistical analysis and ultimately reporting and preparing the final product “a manuscript ready for submission to a proper journal.

A basic understanding of common statistical concepts (e.g., confidence intervals and hypothesis tests) and, though not essential, some prior knowledge of SR and meta-analytic concepts.

The whole program is divided into following components:


The Teaching Modules:

Principles of systematic review and evidence analysis: 

The focus of this course is on advanced topics in evidence synthesis, including state-of-the-art methods such as GRADE. The workshop consists of a mixture of interactive presentations, small-group discussions, and hands-on computer exercises.

The course aims to introduce participants to the methodology of systematic review (SR). By the end of the course participants should be able to:

  • Explain the need for systematic review 
  • Distinguish between and mention the advantages and disadvantages of the various types of literature review
  • Formulate a systematic review question
  • Appreciate the role of tools to assess risk of bias (RoB), including their application to RCT
  • Summarize the findings of a systematic review;
    Assess the quality of evidence in a SR using GRADE approach;

 

This module includes six lectures as follows:

  • Lecture 1: Explain the need for systematic review;
  • lecture 2: Distinguish between the advantages and disadvantages of the various types of literature review;
  • Lecture 3: Formulate a systematic review question;
  • Lecture 4: Appreciate the role of tools to assess risk of bias (RoB), including their application to RCT;
  • Lecture 5: Summarize the findings of a systematic review;
  • Lecture 6: Assess the quality of evidence in a systematic review using GRADE approach

Recommended teaching materials:

  1. Egger M, Davey Smith G, and Altman D. Systematic Reviews in Health Care: Meta-analysis in Context, 2nd Edition, BMJ Books, 2001.
  2. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011.
  3. GRADE working group. GRADE Series. Journal of Clinical Epidemiology. Available at: http://www.jclinepi.com/content/jce-GRADE-Series

 

The Principals of meta analysis

This module includes three parts

 Part I- Meta-analysis – Meta Analysis Basic

This part includes the following lectures’ topic:

  • Basic concepts of meta-analysis
  • Effect measures, commonly used in meta-analysis of intervention studies
  • Modeling strategies for meta-analysis: fixed-effect models (I-V, M-H, Peto) vs. random-effects model [D-L, P-L, H-K-S-J]
  • Assessment of heterogeneity

Suggested Readings:

  • White IR, Schmid CH, and Stijnen T, Choice of effect measure and issues in extracting outcome data, in Handbook of Meta-Analysis, C.H. Schmid, T. Stijnen, and I.R. White, Editors. 2020, Chapman and Hall/CRC Press: USA. p. 27-38.
  • Röver C, Knapp G, and Friede T. Hartung-Knapp-Sidik-Jonkman approach and its modification for random-effects meta-analysis with few studies. BMC Med Res Methodol, 2015; 15: 99. DOI: 10.1186/s12874-015-0091-1.
  • Harris RJ, Deeks JJ, Altman DG, et al. Metan: Fixed- and Random-Effects Meta-Analysis. Stata J, 2008; 8(1): 3-28. DOI: 10.1177/1536867×0800800102.
  • Thompson SG and Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med, 1999; 18(20): 2693-708.

Part II- Meta-analysis (Subgroup analysis)

This part includes the following lectures topic

  • Subgroup analysis vs. sensitivity analysis and credibility of subgroup effects (ICEMAN tool)
  • Meta-regression and aggregation bias in meta-analysis
  • Publication bias and small study effect (statistical tests and visual inspection with funnel and contour-enhanced funnel plots)
  • Meta-analysis of prevalence and proportions

Suggested Readings:

  • Schandelmaier S, Briel M, Varadhan R, et al. Development of the Instrument to assess the Credibility of Effect Modification Analyses (ICEMAN) in randomized controlled trials and meta-analyses. CMAJ, 2020; 10; 192(32): E901-E906.
  • Baker WL, White CM, Cappelleri JC, et al. Understanding heterogeneity in meta-analysis: the role of meta-regression. Int J Clin Pract, 2009; 63(10): 1426-34. DOI: 10.1111/j.1742-1241.2009.02168.x.
  • Thompson SG and Higgins JP. How should meta-regression analyses be undertaken and interpreted? Stat Med, 2002; 21(11): 1559-73. DOI: 10.1002/sim.1187.
  • Peters JL, Sutton AJ, Jones DR, et al. Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J Clin Epidemiol, 2008; 61(10): 991-6. DOI: 10.1016/j.jclinepi.2007.11.010.
  • Peters JL, Sutton AJ, Jones DR, et al. Performance of the trim and fill method in the presence of publication bias and between-study heterogeneity. Stat Med, 2007; 26(25): 4544-62. DOI: 10.1002/sim.2889.
  • Nyaga VN, Arbyn M, and Aerts M. Metaprop: a Stata command to perform meta-analysis of binomial data. Arch Public Health, 2014; 72(1): 39. DOI: 10.1186/2049-3258-72-39.

Part III- Meta-analysis – Network Meta-analysis (NMA)

This part includes the following lectures

  • Understand what indirect comparisons and NMA are and why they are used;
  • Perform NMA using Stata, with continuous and dichotomous data;
  • Understand the assumptions made in NMA and use Stata to examine consistency;
  • Be aware of different techniques to present the results from NMA;
  • Be able to critically appraise a paper that uses NMA;
  • Evaluate the quality of evidence in a NMA.
    •  

Reporting and writing manuscript of systematic review and meta-analysis:

This part of the internship is lecture based, interns will develop the skills on how to write a scientific paper and how to report the result if systematic review and meta-analysis. The part includes the following lecture’ topics:

  • Writing and Publication Process
  • Basic Writing Principles
  • Abstract
  • Introduction Section
  • Methods Section
  • Results Section
  • Discussion Section

 

Topic Selection:

Each intern is supposed to choose his own topic of systematic review and meta-analysis.   Interns may share a topic with each other and agree on the share authorship. This will help the intern to have more product and publication at the end of its internship.

In order to have a successful outcome, the topic needs to be a novel topic and carry a substantial degree of novelty.  The topic will be scrutinized by our faculty and the interns need to defend his topic and make sure the novelty the topic, the needs of systematic review and meta-analysis is out there, etc.   There will be one lecture on how to choose a topic of scientific worthy for systematic review or meta-analysis. We encourage intern to spend time in choosing a proper topic.

Performing a real-world systematic review and meta-analysis (Interactive with help of experience teaching assistant).

This part of the internship is, in fact, the main part, the intern who has chosen his topic will b have a teaching assistant assign to him/her. The teaching assistant will go through the whole process of systematic review (right database, search strategies, evidence analysis, statistical analysis).  Our teaching assistant are experts in meta-analysis, and they have published and contributed to many systematic reviews.  

Writing report and preparation of manuscript for publication with the help of experience teaching assistant.

This part of the internship complements the whole process of lectures, search, evidence analysis, statistical analysis.   The same teaching assistant who has been with intern during the course, will help the intern to prepare his report5 and write proper manuscript for publication.

 

 

 

The Systematic review and meta-analysis internship program is organized by the West Asia Organization for Cancer Prevention APIOCP’s west Asia Chapter.  The program is implemented by very experienced lecturers in the field of evidence analysis from different universities. The program best fit to health services researchers, epidemiologists, statisticians, systematic reviewers and decision analysts who are planning to perform a meta-analysis of their own concern.

The prospect intern needs the basic understanding of common statistical concepts (e.g., confidence intervals and hypothesis tests) and, though not essential, some prior knowledge of SR and meta-analytic concepts.  Normally, young and early career scientists are the main beneficiaries of the internship however, low- and high-level graduate student can participate. The program is in English and the prospect intern needs to have English writing and listening skills to participate in the lectures and teaching assistant sessions.

To participate, you need to register and send your resume and a statement of purpose. The statement of purpose would describe the interns aim and goal as well as its capability and better include a topic for doing a systematic review.  All the participants will receive an official certificate for participation in the internship.

The program may take up to 5 months (though the course organizer tries to help the intern to finish the program in three months).

While the lectures are set at fixed time (the intern must be ready to attend), the interactive part is flexible to arranging by intern and its teaching assistant.

To register, please click and follow the instruction