Checklist — Basic meta-analysis

A student-friendly checklist for Your First Meta-Analysis

Refer to Chapter 20.15 in Mike’s Biostatistics Book and links for discussion about how to conduct and interpret meta-analysis work.

In addition to this guideline, see

From data collection → analysis → lab-report write-up

  1. Planning and Setting Inclusion Criteria
    1. Before searching any literature, decide and write down:
      • Research question
      • What effect or outcome are you summarizing?
      • Example: “Does treatment X reduce symptom Y compared to control?”
    2. Inclusion criteria (must be set before searching)
      1. Study type: e.g., clinical trials, field studies, experiments.
      2. Population/species: humans, mice, dogs, etc.
      3. Outcome requirement: study must report p-values, effect sizes, or enough data to compute them.
      4. Date range: last 5 years, last decade, etc.
      5. Language limitations: usually English only for beginners.
      6. Control/comparison group required? Yes/no.
      7. Minimum sample size? Optional but helpful.
    3. Exclusion criteria
      1. Wrong study design (e.g., review papers).
      2. Missing outcome data (no p-values or data drops).
      3. No control group (if required).
      4. Study is duplicated (conference abstract + paper).
      5. Write these criteria clearly; they go in your Methods section later.
  2. Searching the Literature
    1. Choose a database (eg, PubMed or Google Scholar).
    2. Write the search terms exactly as you used them.
      • Example: “botox migraine” + filters: Clinical Trial, last 5 years.
    3. Record how many papers were returned.
    4. Download abstracts or open them in tabs.
      • Tip: Keep everything in a spreadsheet (Excel/Sheets). It keeps the workflow transparent.
  3. Screening and Data Extraction
    1. For each abstract:
      • Screen for inclusion criteria: Does it meet all criteria? Yes/No
      • If No, record why (e.g., “no control group,” “not a clinical trial”).
      • If Yes, move to data extraction.
    2. Extract the following variables (make a column for each in your spreadsheet):
      • Citation (authors, year).
      • Study design (trial, experiment, etc.).
      • Sample size(s).
      • p-value(s) for the outcome of interest.
      • Effect size if available (Cohen’s d, mean difference, log odds; optional for beginners).
      • Confidence intervals
      • Any notes that help interpret results.
      • Check for duplicate studies and mark accordingly.
    3. Decide on one main outcome per study if multiple are reported.
      • Tip: The primary outcome should be stated in the abstract — if not, exclude the paper.
  4. Preparing Data for Meta-Analysis
    1. Convert all p-values to the same scale (optional but recommended):
      • Use Fisher’s method, or
      • Convert p-values to Z-scores, or
      • Convert to effect sizes (if data allow; see Chapter 3.5 and Chapter 7.6 in Mike’s Biostatistics Book).
    2. Check for missing information
      • If p-value is “p < 0.05,” students may decide to replace (impute) with 0.049 (clearly noted).
      • If p-value is “p > 0.05,” students may decide to replace (impute) with 0.51 (clearly noted).
      • If there’s no p-value, exclude the study and mark the reason.
        • Optional, use calculator methods to estimate p-value, if data allow; see Chapter 3.5 and Chapter 7.6 in Mike’s Biostatistics Book
  5. Doing the Meta-Analysis (Beginner Level)
      • Tip: Most intro meta-analyses use Fisher’s method because it works directly with p-values.
    2. Choose your method
      • Fisher’s combined p-method (most common for students).
      • Or compute effect sizes with a simple fixed-effects model (optional upgrade).
    3. Combine the p-values
      • Use R, base and R meta-analysis packages like metafor.
      • Tip: See examples in Chapter
    4. Get one overall p-value showing whether the studies collectively support the effect.
    5. (Optional) Weight the studies
      • Larger sample sizes can be weighted more heavily.
      • Beginners can skip weighting or use simple sample-size weights.
  6. Create one or two visual summaries
    1. A simple table with included studies.
    2. A basic forest plot if effect sizes were calculated (optional).

Notes:

This is a draft, written November 2025 by MD with assistance from generative AI, ChatGPT