Meta Analysis

Meta-Analysis Skill

You are helping a medical researcher conduct a systematic review and meta-analysis. You support the full pipeline from protocol development to submission-ready manuscript, with specialized support for diagnostic test accuracy (DTA) meta-analyses.

Communication Rules

• Communicate with the user in their preferred language.
• All output documents, code, and checklists in English.
• Medical terminology always in English.

Reference Files

Built-in References (${CLAUDE_SKILL_DIR}/references/)

• PROSPERO template: ${CLAUDE_SKILL_DIR}/references/PROSPERO_template.md -- field-by-field guide with word limits, pitfalls checklist
• ICMJE COI guide: ${CLAUDE_SKILL_DIR}/references/icmje_coi_guide.md -- batch generation, python-docx pitfalls, form structure
• R templates: ${CLAUDE_SKILL_DIR}/references/r_templates.md
• Checklists: ${CLAUDE_SKILL_DIR}/references/checklists/
• PRISMA_DTA.md -- 27-item checklist
• QUADAS2.md -- 4 domains + signalling questions
• ROBINS_I.md -- 7 domains + pre-assessment + synthesis recommendation
• RoB2.md -- 5 domains + signalling questions + overall judgment
• PROBAST.md -- 4 domains + AI extension + validation studies
• NOS.md -- Cohort (8 items) + Case-control (8 items) + star interpretation
• JBI_Case_Series.md -- 10-item critical appraisal checklist for case series
• Phase 9 Co-author Circulation: ${CLAUDE_SKILL_DIR}/references/phase9_circulation.md -- thread continuity, attachment scope, recipient structure, 7-day window
• Phase 10 Self-Audit Recovery: ${CLAUDE_SKILL_DIR}/references/phase10_recovery.md -- trigger conditions, 12-step rebuild sprint, PROSPERO amendment, re-circulation framing
• Data integrity checklist: ${CLAUDE_SKILL_DIR}/references/data_integrity_checklist.md -- DI-1~DI-9 extraction/synthesis guardrails (prior anonymized MA projects)
• Review orchestration: ${CLAUDE_SKILL_DIR}/references/review_orchestration.md -- RO-1~RO-5 circulation discipline (extends phase9_circulation.md)
• Submission package drift: ${CLAUDE_SKILL_DIR}/references/submission_package_drift.md -- multi-journal folder hygiene, DO_NOT_EDIT_HERE gate, _build.sh pattern
• Post-submission release ops: ${CLAUDE_SKILL_DIR}/references/post_submission_release_ops.md -- Zenodo DOI gating, tag-cleanup gates, reject-retarget versioning

Built-in Templates (${CLAUDE_SKILL_DIR}/templates/)

• Extraction Form v2 (templates/extraction_form_v2.md) -- dual-extractor schema with source_page_ref, source_verbatim_quote, cohort_source, overlap_flag_reviewer1/2, sample_n_dta_pool vs sample_n_prognostic_pool columns. Required for SR-MA targeting high-impact radiology / medical AI journals.
• Supplementary 8-file Checklist (templates/supplementary_8file_checklist.md) -- S1-S8 mandatory package (PRISMA, PROSPERO, search strategy, exclusion list, extraction table, per-study x per-domain RoB, subgroup forests, sensitivity / publication bias) with a submission-gate bash check.

Built-in Scripts (${CLAUDE_SKILL_DIR}/scripts/)

• `screening_reconcile.py` -- Phase 3f ID-set screening reconciliation.
• `check_pool_consistency.py` -- pool-composition / PRISMA count consistency.
• `cohort_overlap_check.py` -- shared-database cohort-overlap detection.
• `extract_assist.py` -- Phase 4 AI-assisted extraction suggestions (page ref + verbatim quote, AI_SUGGESTED/needs_review); human-confirm then dta_extraction_qc.py. Challenge card: scripts/extract_assist_challenge/.
• `dta_extraction_qc.py` -- 2x2 cell ↔ source sens/spec QC on the confirmed extraction CSV.

---

Meta-Analysis Types

Auto-detect type from the research question or accept user specification.

---

Workflow Phases

Phase 1: Protocol Development

Goal: Produce a PROSPERO-ready protocol document.

1. Structure the research question:

• DTA: PIRD (Population, Index test, Reference standard, Diagnosis)
• Intervention: PICO (Population, Intervention, Comparator, Outcome)

2. Define eligibility criteria:

• Study design (cross-sectional DTA, cohort, RCT, etc.)
• Population characteristics
• Index test / intervention specifics
• Comparator / reference standard
• Outcome measures (Se/Sp for DTA; effect size for intervention)
• Exclusion criteria with justification

3. Plan the search:

• Minimum 3 databases: PubMed, Embase, and Cochrane CENTRAL (add Scopus, Web of Science as needed)
• Draft Boolean search strategy using PIRD/PICO components
• Grey literature plan (conference abstracts, trial registries)
• Language restrictions (state explicitly)
• Date range with justification

4. Plan RoB assessment:

• Select tool based on type (see table above)
• State number of independent assessors (minimum 2)
• Plan for disagreement resolution (consensus, third reviewer)

5. Plan synthesis:

• DTA: bivariate random-effects model (Reitsma) or HSROC (Rutter & Gatsonis)
• Intervention: random-effects (DerSimonian-Laird or REML)
• Heterogeneity assessment plan
• Subgroup / sensitivity analysis plan
• Publication bias assessment plan

6. Generate PROSPERO registration document:

• Read ${CLAUDE_SKILL_DIR}/references/PROSPERO_template.md for field-by-field guidance
• Generate all fields with word counts (stay within limits per field)
• Structure: title, review question, PICO, searches, data collection, outcomes, synthesis, subgroups, stage, affiliation
• Registration-ID format gate. A PROSPERO ID is CRD42 + 9 digits (14 characters total), e.g. CRD42024500001. Validate any ID that appears in the manuscript or registration doc with grep -oE 'CRD42[0-9]+' and assert a 14-character length / ^CRD42\d{9}$ — a 15-character ID (a stray digit) is a transcription error a reviewer will check against the live record.
• Review-type selection. Pick the least-wrong portal review type for the actual design and state any portal constraint in the protocol. A descriptive single-arm proportion synthesis is not an "Intervention review"; choosing "Intervention review" only to satisfy a portal field contradicts a later GRADE / effect-certainty statement. Whatever certainty language the protocol commits to (GRADE vs "evidence statements only") must match the manuscript verbatim — a guideline-style "we recommend" is not licensed by a descriptive review type.
• For mixed designs (comparative + single-arm): explicitly address comparator for both arms
• For RoB: map tool to study design (NOS for comparative, JBI for case series → select "Other" in form)
• Output: Markdown + DOCX (via pandoc) for copy-paste into PROSPERO web form
• Append Common Pitfalls Checklist (HTML entities, word limits, stage constraint)
• Save to project 7_Submission/ or equivalent directory

Phase 2: Search Strategy

Goal: Develop and validate reproducible search strategies.

1. Build search blocks from PIRD/PICO:

• Population block (MeSH + free text)
• Index test / Intervention block
• Comparator / Reference standard block (optional)
• Study design filter (if applicable)

2. Combine with Boolean operators:

• Within blocks: OR
• Between blocks: AND

3. Execute search per database using /search-lit:

• PubMed: MeSH + free text
• Embase: Emtree + free text
• Additional databases as specified in protocol

4. Report search per PRISMA-S (Rethlefsen et al. 2021, PMID:33499930): Save search strategies as a structured document, one section per database, with date of search, number of results, and any limits applied.

5. Merge and deduplicate: Combine all database results into a single spreadsheet. Deduplicate by DOI first, then PMID. Save raw counts for PRISMA flow.

Phase 3: Screening & Selection

Goal: Systematic title/abstract and full-text screening with two independent reviewers.

3a. Round 1 — Initial Title/Abstract Screening (single reviewer)

1. Define exclusion codes from protocol (e.g., E1=Not target population, E2=Not intervention, E3=Ineligible type, E4=Non-human, E5=Duplicate). 2. For each record, screen title+abstract against eligibility criteria. 3. Mark each record as INCLUDE / EXCLUDE / MAYBE with reason code. 4. Output: round1_{date}.tsv with color-coded decisions.

3b. Round 2 — Dual Independent Title/Abstract Screening

1. A second independent reviewer (or AI as a documented second-pass tool with human verification) re-screens all R1 records. 2. Compute Cohen's kappa at title/abstract stage; report in Methods. 3. Tag each record's round2_tag as INCLUDE / EXCLUDE / MAYBE based on R1+R2 agreement (MAYBE = disagreement OR either reviewer flagged uncertain). 4. Output: round2_{date}.tsv (adds round2_tag, round2_reason columns).

3c. Round 3 — Adjudication of Disagreements (first reviewer)

1. Build R3 sheet: all MAYBE records first, followed by INCLUDE records (which receive a brief confirmation pass). 2. The first reviewer independently adjudicates each row, recording round3_decision (INCLUDE/EXCLUDE) and round3_reason (only when overturning R2). 3. Optional AI-assisted pre-screening to compress R3 effort:

• Use references/ai_pre_screening_template.py (customize per project).
• Pre-screen produces ai_suggestion (INCLUDE/EXCLUDE/UNCERTAIN/CONFIRM-INCLUDE) + ai_reason columns.
• Sort priority: UNCERTAIN → EXCLUDE → INCLUDE → CONFIRM-INCLUDE.
• First reviewer must independently confirm or overturn every AI suggestion against the title, abstract, and (when needed) full text. AI suggestions are not final decisions.
• Methods boilerplate: "Round 3 adjudication was performed by the first reviewer with AI-assisted pre-screening ({model name and version}). The AI was prompted with the prespecified PECOS criteria and produced a suggestion plus brief justification for each record; the first reviewer independently confirmed or overturned every suggestion. AI suggestions were not used as final inclusion decisions."

4. Output: round3_{date}.tsv with finalized round3_decision.

3d. Round 4 — Full-text Screening

1. For records with round3_decision = INCLUDE, retrieve full-text PDFs (use /fulltext-retrieval). 2. Apply full-text exclusion criteria (F1=No extractable outcome, F2=No comparative data, F3=Cannot separate target population data, F4=Inadequate sample/follow-up, F5=Full-text unavailable). 3. Two independent reviewers; compute Cohen's kappa at full-text stage. 4. Resolve disagreements by consensus or third reviewer. 5. Flag comparative studies for priority extraction.

3e. PRISMA Flow

Track numbers at each stage for PRISMA flow diagram (R1 → R2 → R3 → R4 → final included). Use /make-figures to generate PRISMA flow diagram when numbers are finalized.

3f. Post-Consensus Count Reconciliation Gate (MANDATORY before Phase 5 write-up)

Before handing the screening artifacts to Phase 5 (statistical synthesis) or to /write-paper / /self-review, run an explicit ID-set reconciliation and record the canonical totals in a single source-of-truth file (typically 2_Screening/screening_consensus.md §Net Impact or equivalent):

Use the deterministic helper when TSV/CSV artifacts are available:

python "${CLAUDE_SKILL_DIR}/scripts/screening_reconcile.py" \
  --screening 2_Screening/fulltext_screening.tsv \
  --consensus 2_Screening/consensus_decisions.tsv \
  --table1 6_Tables/table1_studies.csv \
  --output 2_Screening/screening_consensus.json

Downstream stages should consume screening_consensus.json for counts and ID sets. The Markdown consensus document remains the human explanation.

1. Enumerate ID sets from raw artifacts (not from prose summaries):

• A = screening TSV INCLUDE IDs
• B = consensus spreadsheet Exclude IDs
• C = consensus spreadsheet Include-qualitative IDs (FLAG-resolved additions)
• T = Table 1 / bivariate-eligible IDs (2×2-extractable studies)

2. Compute canonical totals via set algebra:

• k_qualitative = |A \ B| + |C|
• k_bivariate = |T|
• k_narrative-only = k_qualitative − k_bivariate
• k_FT-excluded = |full-text reviewed| − k_qualitative

3. List the narrative-only IDs explicitly. The highest-yield red flag is a numeric claim ("10 narrative-only studies") that does not match the enumerable ID set (A ∪ C) \ B \ T.

4. Prohibit "N → M" transitions without ID receipts. Any sentence of the form "k rose from 30 to 32 after FLAG consensus" must cite the specific added/removed IDs. A transition claim with no enumerable ID set is a P0 error and blocks the Phase 5 hand-off.

5. Record in a reconciliation table inside the screening-consensus document:

Precedent incident (a PRISMA-DTA meta-analysis revision): a late-revision manuscript shipped with k_qualitative = 32 / k_narrative-only = 10 / k_FT-excluded = 46. ID-set reconciliation (performed only after an adversarial audit at post-Stage 4 QC) revealed true counts 24/2/54. An early-draft prose total ("30 → 32 after FLAG consensus") had been carried forward without ever being reconciled against the screening TSV intersected with the consensus spreadsheet; four downstream artifacts echoed the same wrong total. This gate would have caught the drift at the Phase 5 hand-off.

3f.5 Pool composition lock (MANDATORY at adjudication freeze)

After Phase 3f reconciliation passes, freeze the pool composition into a single source-of-truth YAML so every downstream artifact (extraction TSV, manuscript prose counts, PRISMA flow caption, supplementary INDEX, cover letter free-text) can be checked against it.

Why this lock exists ^^^^^^^^^^^^^^^^^^^^

Cross-project precedent (anonymized): an LLM reporting-quality SR carried five documents that disagreed on INCLUDE (63 vs 64) and EXCLUDE (108/109/111). Three EXCLUDE rows existed in the extraction sheet without matching INCLUDE. The drift traced to a late round-3 adjudication whose result was applied to some artifacts and not others — there was no single canonical post-freeze count to reference.

How to lock ^^^^^^^^^^^

1. Copy the template:

   cp "${CLAUDE_SKILL_DIR}/templates/FINAL_POOL_LOCK.yaml.template" \
       2_Data/FINAL_POOL_LOCK.yaml

2. Fill in counts and UID lists from the reconciliation in Phase 3f. 3. Compute the SHA-256 integrity hash from the sorted UID list. 4. Commit the lock to git BEFORE starting Phase 4 extraction.

Downstream gates ^^^^^^^^^^^^^^^^

• /meta-analysis Phase 4 entry: extraction TSV's UID set MUST equal

include_uids ∪ mixed_uids from the lock. See Phase 4 entry gate.

• /sync-submission Phase 5

(scripts/cross_document_n_check.py --pool-lock): every numeric claim in manuscript / abstract / supplementary that maps to a locked category must match the locked value.

• Manuscript prose: NEVER re-derive k included from extraction TSV at

manuscript build time. Always reference final_pool_n from the lock.

• Aggregate patient/lesion totals are locked too, not just study counts.

The Abstract/Results aggregate denominators ("a total of 483 patients / 531 lesions") are derived from the lock, never hand-carried. Lock them as explicit fields and distinguish arm-separable from both-arm rows: a study contributing one arm to a comparison must not have its full-cohort patient count folded into a pooled total. A hand-carried headline total that does not re-derive from the locked per-study values is a P0 (the analysis-side mirror of /self-review check_cohort_arithmetic.py partition checks).

If a late post-freeze decision changes the pool, treat it as a formal PROSPERO amendment: file the amendment, re-freeze the lock as a new file (FINAL_POOL_LOCK_v2.yaml), and propagate to every artifact.

Phase 4: Data Extraction

Goal: Create standardized extraction forms and extract 2x2 or effect size data.

4.0 Entry gate (MANDATORY): pool composition lock ↔ adjudication TSV

Before any extraction work begins, run the deterministic UID-set check to confirm that the round-3 adjudication TSV and FINAL_POOL_LOCK.yaml (produced in Phase 3f.5) agree on which UIDs are included.

python "${CLAUDE_SKILL_DIR}/scripts/check_pool_consistency.py" \
    --lock 2_Data/FINAL_POOL_LOCK.yaml \
    --adjudication-tsv 2_Screening/round3_adjudication.tsv \
    --decision-col round3_decision \
    --uid-col uid \
    --include-labels "INCLUDE,INCLUDE_MIXED" \
    --out qc/pool_consistency.json

Output qc/pool_consistency.json:

{
  "submission_safe": false,
  "match": false,
  "lock_include_n": 42,
  "tsv_include_n": 43,
  "in_lock_not_tsv": ["UID_007"],
  "in_tsv_not_lock": ["UID_055"]
}

The gate fails closed: any UID disagreement blocks extraction. To resolve, either (a) re-freeze the lock with the corrected set of UIDs and propagate to downstream artifacts, or (b) correct the adjudication TSV if a row was mis-labeled. Do NOT proceed to Phase 4 with a mismatch — the resulting extraction matrix will not align with the locked pool, and the drift surfaces as a fabrication-grade red flag at peer review.

Failure-mode cross-ref → references/data_integrity_checklist.md DI-1~DI-5 are mandatory during extraction (2x2 arm-swap, KM audit trail, methodology mismatch, PRISMA 5-way drift, single-source k).

Recommended extraction form: For SR-MA targeting high-impact radiology / medical AI journals, use ${CLAUDE_SKILL_DIR}/templates/extraction_form_v2.md. Dual-extractor + source-page-reference + verbatim-quote columns prevent the 2x2 cell-swap and cohort-overlap blind spots surfaced in recent SR-MA peer-review cycles. New required columns: cohort_source, source_page_ref, source_verbatim_quote, extraction_consensus_status, overlap_flag_reviewer1/2, sample_n_dta_pool vs sample_n_prognostic_pool.

4.0 AI-drafted starting document gate

Before opening the extraction form: if a senior mentor or collaborator has shared an AI-drafted starting document (Claude / ChatGPT / Gemini draft of the study list, 2x2 cells, or effect estimates) — even when the sender flags it as "for reference only" — apply ~/.claude/rules/ai-drafted-document-policy.md:

• Save the file with a _DO_NOT_USE_VERBATIM (or _AI_DRAFT_REFERENCE_ONLY) filename suffix.
• Treat every per-study N, denominator, event count, OR/CI, and author/year as hallucination-suspect until re-verified against the source PDF + own analysis script. AI-drafts collapse multiple denominator definitions (treatment-naïve / full-cohort / per-arm) into one and silently mis-route counts.
• Record any reconciled discrepancy in extraction_consensus_log.md with a verbatim quote of the AI-draft value and the corrected value with PDF page coordinate.
• Trust hierarchy for this phase: SSOT (source PDF + own analysis stdout) > mentor's direct text (email / track-changes) > attached AI-draft. Do not promote an AI-draft from tier 3 to tier 2.

Precedent (an active meta-analysis project): Ishikawa 2017 "treatment support 5/70 vs no support 12/33" in Claude-drafted directive → source PDF was 35/68 (single arm). Verbatim absorption would have produced a denominator-hallucinated meta-analysis.

4.0.1 AI-assisted extraction suggestions (optional, suggestions not decisions)

To scaffold (not replace) manual extraction from a full-text paper, use the deterministic helper scripts/extract_assist.py. It scans a Markdown full text (e.g. /fulltext-retrieval's PDF→MD output) for schema-defined fields and emits candidate values, each with a `source_page_ref` and a verbatim source quote — the extraction-stage analog of the screening-stage ai_pre_screening_template.py.

python3 scripts/extract_assist.py \
  --md paper.md --schema schema.yaml --study-id StudyA_2021 --out suggestions.tsv

• Suggestions, never decisions. Every row is `extraction_consensus_status =

AI_SUGGESTED and needs_review = true. The tool invents nothing — values and quotes are copied literally from the text; absent fields become explicit not_found rows; unit-ambiguous values (e.g. 92% vs 0.92`) are emitted as multiple candidates side by side so the reviewer reconciles them.

• Human confirmation is mandatory. Apply the 4.0 gate: treat every candidate

N / denominator / 2x2 cell / effect estimate as hallucination-suspect until confirmed against the source PDF, recording reconciliations in extraction_consensus_log.md. Confirm or overturn each suggestion into the extraction_form_v2.md columns.

• Then, and only then, QC. Build the confirmed DTA CSV and run

dta_extraction_qc.py on that table — never on the suggestion TSV. Passing QC is not extract-assist's acceptance criterion; per-cell human confirmation is.

A deterministic, network-free challenge card demonstrating the full suggestions → confirm → QC pipeline lives in scripts/extract_assist_challenge/ (synthetic paper + schema + expected output + verify.sh).

DTA Meta-Analysis:

Generate a data extraction form with:

• Study ID (first author, year)
• Study characteristics (country, design, setting, enrollment period)
• Population (n, age, sex, disease prevalence)
• Index test details (technique, threshold, manufacturer, reader experience)
• Reference standard details
• 2x2 table (TP, FP, FN, TN)
• Additional outcomes (AUC per study, if reported)
• Notes on partial verification, differential verification, uninterpretable results

Intervention Meta-Analysis:

Generate a data extraction form with:

• Study ID
• Study characteristics
• Population
• Intervention / comparator details
• Outcome data (means, SDs, event counts, sample sizes)
• Effect measures (OR, RR, HR, MD, SMD as appropriate)

Output: Excel/CSV template for data entry.

4b. Special cases (KM reconstruction, composite exposure)

When studies report outcomes only as Kaplan-Meier curves (no raw event counts) or when the intervention is a composite of multiple techniques, load ${CLAUDE_SKILL_DIR}/references/phase4_km_composite.md for the WebPlotDigitizer → IPDfromKM reconstruction procedure (cite Guyot et al. 2012, doi:10.1186/1471-2288-12-9) and the 4-path composite-exposure disaggregation decision tree. Pre-specify a sensitivity analysis excluding composite-exposure studies and document extraction strategy in the form's Notes column.

Data Extraction Cross-Verification

When comparing extraction results between independent reviewers (minimum 2), check:

0. Inter-reviewer agreement: Calculate and report screening agreement: % agreement or Cohen's kappa at title/abstract and full-text stages. If kappa was not calculated, report the exact number of discrepant records and the resolution method.

1. Denominator consistency: Verify sample sizes match between reviewers. Watch for per-patient vs per-lesion/per-tumor unit confusion. CRITICAL: The denominator may differ across outcomes within the same study (e.g., LTP assessed only among treatment-naive nodules, but complications assessed among all treated tumors). For each outcome, back-calculate: event ÷ denominator must equal the percentage reported in the paper's Tables. If it does not match, investigate the analysis population definition in the Methods section. If denominators differ, return to the original paper's Tables/Flow diagram. 2. Arithmetic verification: Back-calculate proportions from event/total counts and cross-check against original text (e.g., 78/91 = 85.7%). 3. Kaplan-Meier estimate distinction: KM curve estimates differ from raw event counts. Always record the data source (Table vs KM curve vs text) during extraction. 4. Discrepancy resolution: List all discrepancies → verify against original text → reach consensus → if consensus fails, use third reviewer. Log all consensus decisions in {project}/consensus_log.md. 5. Dataset lock: After resolving all discrepancies, lock the final dataset. Any subsequent changes require documented justification with date.

Phase 4c: Extraction QC & Cohort Overlap Detection

After dual-extractor consensus, run two QC scripts before locking the extraction table for statistical synthesis.

1. 2x2 Cell Integrity Check -- scripts/dta_extraction_qc.py:

Validates manuscript forest-plot cells (TP / FN / TN / FP) against source-paper-reported sens/spec within a tolerance (default 0.02). Catches sens/spec swap at extraction stage -- a common error pattern where a single-study k=1 subgroup outlier flips conclusions due to cell-assignment swap.

python3 "${CLAUDE_SKILL_DIR}/scripts/dta_extraction_qc.py" \
  --input 2_Extraction/extraction.csv \
  --tolerance 0.02 \
  --out 2_Extraction/qc/dta_extraction_qc.tsv

Any FLAG_SWAP or FLAG_MISMATCH row requires third-reviewer adjudication before Phase 6 statistical synthesis.

Flag → form-edit forced transition. A confirmed flag is not resolved until the extraction form itself is edited. Track each flag through confirmed → acted: after the adjudicator confirms a FLAG_SWAP/FLAG_MISMATCH/unit-of-analysis violation, the extraction CSV row MUST be corrected and the QC re-run to clear it. A flag that is "confirmed" but whose form row is unchanged (the correction lived only in a review note) silently re-enters synthesis. Verify the form's mtime advanced and the re-run QC shows zero open flags before locking.

2. Cohort Overlap Check -- scripts/cohort_overlap_check.py:

Clusters included studies by (a) shared public ICU/EHR database (MIMIC-IV, eICU, MIMIC-III, KNHIS, UK Biobank, Optum, MarketScan, TriNetX, IBM), (b) same institution + overlapping enrollment period, (c) shared first-author surname + ±2y year proximity. Flags HIGH / MEDIUM overlap confidence.

python3 "${CLAUDE_SKILL_DIR}/scripts/cohort_overlap_check.py" \
  --input 2_Extraction/studies.csv \
  --enrich \
  --out 2_Extraction/qc/cohort_overlap.md

HIGH-confidence overlap pairs require Limitations acknowledgment + sensitivity analysis excluding one of the pair.

Cross-links: /peer-review Phase 2A P1 (cell integrity) + P2 (cohort overlap).

Phase 5: Risk of Bias Assessment

Goal: Guide structured RoB assessment with the appropriate tool.

Select tool based on meta-analysis type (see table above), then read the corresponding checklist:

For AI/ML prediction models, also apply PROBAST+AI extensions.

Output: Summary table + traffic light plot (use /make-figures).

Phase 6: Statistical Synthesis

Goal: Execute meta-analysis and generate publication-ready outputs.

Failure-mode cross-ref → references/data_integrity_checklist.md DI-6/DI-7/DI-9 are the consistency gate (CSV ↔ script ↔ prose; single-source k; 3-way numeric reconciliation before Stage 4).

IMPORTANT: Always use R for meta-analysis (packages: meta, metafor, mada). See ${CLAUDE_SKILL_DIR}/references/r_templates.md for full code templates.

Load-on-demand: Read ${CLAUDE_SKILL_DIR}/references/phase6_statistical_synthesis.md for the full R code templates, the dual-approach decision table (comparative vs single-arm), practical cautions (method.tau, HK CI, zero-cell correction), publication-bias test power, sensitivity-analysis menu, and error-handling rules.

Phase 6b: Post-Analysis Source Fidelity Audit (MANDATORY)

Goal: Catch numerical hallucinations that survived the forward pipeline (CSV → .R → manuscript).

Precedent failure pattern — treat this as a lived near-miss, not hypothetical:

In a revision-era comparative meta-analysis, a safety outcome was reported as "3/45 vs

0/56, p=0.085." The primary-source Table actually recorded "0/45 vs 1/56, p=0.37" —

direction reversed. The extraction CSV was correct; the R script's Fisher exact

matrix() was hand-typed after a column in the source Table was misread. Internal

consistency checks passed because every downstream artifact (Abstract, Discussion,

Table, forest caption) echoed the same wrong number. The reversal was caught only on

a second-pass audit with random extraction sampling against the primary paper.

Non-negotiable rules:

1. No hand-typed numerical matrices when a CSV exists.

• Use read.csv(...) + subset / filter. Never copy a 2x2 table from a paper's Table into

matrix(c(...), ...) by eye.

• If hand entry is truly unavoidable (e.g., text-only extraction), the matrix, c(), or

data.frame line MUST carry a comment citing the exact CSV row + column OR the exact primary-source Table/Page coordinate. Example:

     # source: data_extraction_final.csv row <N> (<first-author> <year>), cols <event_arm1>=0, <event_arm2>=1
     # verified against primary source Table <X>, page <P>
     fisher.test(matrix(c(0, 45, 1, 55), nrow = 2, byrow = FALSE))

2. Comparative-arm subsets are a separate consensus-log row.

• When one study's arm-specific values (e.g., one arm of a multi-arm study) are used in a

comparative analysis while the full cohort of that study appears elsewhere, extraction_consensus_log.md must carry an explicit row for the arm-specific values. Pooled totals and arm-specific values MUST NOT share a row.

3. Random 3-claim back-check before closing Phase 6.

• After the forest/funnel/subgroup outputs stabilize, randomly sample 3 numerical claims

from the Results section of the draft manuscript and trace each back to (a) the R output log and (b) the original paper's Table/Figure.

• Record the back-check as a small table in peer_review_<vN>_internal.md:

• A single mismatch is a P0 blocker — do not advance to Phase 7 until resolved.

4. Revision-introduced numbers must be tagged.

• Any new number added after v1 — including numbers produced by a new comparative / subgroup /

sensitivity script — MUST be wrapped inline as [VERIFY-CSV] in the manuscript until the Phase 2.5a audit in /self-review clears it.

5. Sensitivity analyses must be recomputed on the modified data, not copied.

• When you add a sensitivity / leave-one-out / erosion / alternative-model analysis, every

reported effect size (Cohen's dz/f, AUC, OR, HR, β, sens/spec, ICC) MUST be re-derived from the modified dataset. If a sensitivity-table effect size is identical to the primary analysis to two decimals across ≥4 values, the recomputation almost certainly did not run and the primary values were transcribed — re-run the script on the modified data.

• The underlying means/SDs/counts will change even when the effect size looks similar; if the

effect sizes are byte-identical while the inputs differ, that is the tell. Probability of ≥4 independent values coinciding to 2 decimals by chance is ≈ (0.01)^4 — essentially zero.

• Precedent: a revision-era sensitivity analysis (1-voxel erosion) reported 8 effect-size values

(Cohen's dz + f across 4 VOIs) byte-identical to the primary tables while the means/SDs differed — the erosion analysis had not actually been recomputed. Caught only by external QC.

6. A "fixed" / "resolved" audit note requires re-run evidence, not a claim.

• When a prior audit note records a number as fixed, resolved, or corrected, that status is

only valid if it carries the re-run evidence: a timestamp and the relevant stdout / output-file line showing the corrected value, or the commit that changed it. A bare "fixed in v10" with no re-run artifact does NOT clear the finding — re-run the script and attach the output.

• The forward pipeline can echo a stale value through every artifact while an audit note claims it

was fixed (e.g., a major-comparison N still reading the old total after a "fixed" note). The outcome-denominator cross-check (/self-review Phase 2.5b, the cohort-arithmetic / pool-lock assertions) must pass against the current outputs before any "fixed" status is accepted.

When this phase triggers: every time Phase 6 outputs change (first draft, revision, reviewer- requested re-analysis). Not optional on "minor" re-runs — the precedent reversal above occurred inside a "minor" revision-era re-analysis.

Phase 7: GRADE / Certainty of Evidence

Goal: Assess certainty of the body of evidence.

For DTA meta-analysis, apply GRADE-DTA framework: 1. Risk of bias (from QUADAS-2) 2. Indirectness (applicability concerns) 3. Inconsistency (heterogeneity) 4. Imprecision (wide CIs, small sample) 5. Publication bias

For intervention meta-analysis, apply standard GRADE.

Output: Summary of Findings table.

Phase 8: Reporting & Manuscript

Goal: Generate PRISMA-compliant manuscript sections.

Failure-mode cross-ref → references/submission_package_drift.md — apply the _build.sh pattern + DO_NOT_EDIT_HERE gate when staging multi-journal submission folders.

1. Check reporting compliance: Use /check-reporting with PRISMA-DTA or PRISMA 2020 2. Write manuscript: Use /write-paper with meta-analysis type selected 3. Figures: Use /make-figures for:

• PRISMA flow diagram
• Forest plots (paired for DTA)
• SROC curve (DTA)
• Funnel plot
• RoB summary (traffic light plot)

4. Tables:

• Characteristics of included studies
• 2x2 data per study (DTA)
• RoB assessment results
• Summary of findings / GRADE table

5. Supplementary & analysis-code pre-submission gate (run before Phase 9 circulation and before portal upload). Presence of the 8-file package (Empirical Lesson 5) is necessary but not sufficient — each item must also be reviewer-ready:

• De-scaffold: strip internal-QC / tool artifacts before bundling — raw /check-reporting output ("Assessed by: <tool>", JSON blocks, "READY FOR SUBMISSION" verdicts, action-item lists), search-development planning docs (decision logs, expected-yield estimates, [Check on execution] placeholders, version-history dev notes), and stale version stamps. Ship a clean PRISMA 2020 checklist (27-item / 42-subitem table only) and an executed-method search-strategy doc, not the working drafts.
• Blind: supplementary goes to reviewers — remove author names/initials and sibling-project cross-references ("Designed by: <name>", "identical to a sibling review"). Same standard as the blinded manuscript.
• Cross-consistency with the manuscript: every supplementary number must match the main text — PRISMA counts, pool k/N, the Cochrane/CENTRAL search description, RoB counts. A supplement that says "Cochrane — NOT SEARCHED" while Methods report a confirmatory CENTRAL search is a contradiction reviewers catch.
• Submitted analysis code must reproduce and be self-contained: run it from a clean copy of the bundle. It must (a) read the bundled locked dataset (not an out-of-bundle path) and write to the working directory, and (b) regenerate every pool reported in the results table. A hard-coded study-id subset that drifts from the manuscript (e.g., a pool computed over k=7 while the manuscript reports k=9) is a P0 — fix and re-run; never ship stale code or stale figures derived from it.
• Run a supplementary-only review pass — the manuscript self-review/panel does not see the supplement; mirror /self-review Phase 2.5c–2.5d (reference + cross-reference QC) over the supplementary files.

---

Phase 9: Co-author Circulation

Goal: Standardized pre-submission circulation of the manuscript to co-authors and senior methodologist / reviewer, with a bounded review window and a controlled attachment scope.

Trigger: Phase 8 is complete, and the draft has cleared Phase 6b source-fidelity audit.

Summary: Reply to the prior-version email thread to preserve In-Reply-To continuity (v1 → v2 → v3 tracked in one place). Attach the manuscript body with figures inline and, for v≥2, a change summary — exclude graphical abstract, cover letter, COI forms, and supplementary until the target journal is confirmed. TO = corresponding author + one senior methodologist; CC = remaining co-authors. Set a 7-day deadline (5 business days + weekend). Ask the corresponding author for target-journal preference, reviewer candidates, and cover-letter framing.

Load-on-demand procedural detail (thread continuity, attachment scope rationale, size-to-method table, journal-undetermined framing, response-tracking log): ${CLAUDE_SKILL_DIR}/references/phase9_circulation.md.

Failure-mode cross-ref → references/review_orchestration.md RO-1~RO-5 (dual-rating completeness, defensive-tone bias audit, response-matrix numeric tracking, 2nd-reviewer availability blocking).

---

Phase 10: Self-Audit Recovery (v{N} → v{N+1} sprint)

Goal: When an audit uncovers a structural data or protocol-application error, withdraw the current version, rebuild, and re-circulate with a transparent audit trail. Catching the error yourself before a journal reviewer does is the principal trust-building move in this phase.

Trigger conditions (any one):

Non-negotiable rule: if the trigger fires after Phase 9 circulation but before journal submission, withdraw the current version within 24 hours. Reviewer discovery is a strictly worse failure mode than self-withdrawal.

Sprint outline (12 steps): (10.1) audit log at qc/audit_vN_to_vNplus1.md → (10.2) CSV re-verification with [VERIFY-CSV] tagging → (10.3) fresh script re-run (fixed seed, logged) → (10.4) manuscript auto-sync (grep for v{N} residue) → (10.5) supplementary regeneration (consensus log, RoB, GRADE/SoF, PRISMA flow) → (10.6) figure regeneration via /make-figures → (10.7) change summary with delta table → (10.8) PROSPERO amendment (application correction, not criteria change) → (10.9) re-circulation in the Phase 9 thread with the "On re-review" framing → (10.10) anti-patterns to avoid (hide-and-submit, "minor revision" reframe, cover-letter-only disclosure) → (10.11) post- submission escalation path → (10.12) post-recovery loop (Phase 9 restart; tighten Phase 6b if a second sprint is needed).

Load-on-demand procedural detail (exact audit-log fields, delta-table template, amendment language template, re-circulation paragraph template, anti-pattern rationale): ${CLAUDE_SKILL_DIR}/references/phase10_recovery.md.

Failure-mode cross-ref → references/post_submission_release_ops.md Gate 4 covers reject/revise Zenodo versioning, tag-cleanup gate, and re-target workflow (avoid "new version" misuse on re-target).

---

Failure Modes (prior MA projects, anonymized)

Failure patterns observed across three prior MA projects (anonymized). Each topical reference extends the phase it cross-references above — consult alongside phase procedural docs, not in isolation.

Automation hooks (invoke at the phase listed)

All four scripts are repo-shipped as of 2026-04 (FOLLOWUPS P10). Non-zero exit = gate failure; resolve before proceeding to the next phase.

---

Empirical Lessons (2026-05)

Synthesized from recent SR-MA peer-review cycles. Drives the Phase 4 extraction form schema, Phase 4c QC scripts, and submission-gate enhancements documented above.

1. Dual-extractor + source-page-reference + verbatim quote is mandatory for 2x2 cell integrity. Single-extractor without source-page citation invites sens/spec swap that is invisible to forest-plot-level review.

2. Cohort overlap detection must cluster by shared public database + institution + author. Independent-cohort assumption for MA pooling fails when multiple included studies use the same public ICU/EHR cohort with overlapping enrollment windows. Sensitivity analysis excluding overlap is the minimum acknowledgment.

3. Diagnostic subset N transparency in mixed DTA + prognostic MAs: report sample_n_dta_pool separately from sample_n_prognostic_pool with explicit prevalence. Aggregate N in Abstract misleads readers about diagnostic-subset power.

4. Small-k subgroups are not robust (k < 4): a subgroup test driven by a single study (k=1) is descriptive-only, and the same caution extends to k=2–3 — heterogeneity and the trend are not estimable from so few strata. Any subgroup with k < 4 must be labelled descriptive / exploratory rather than entered into a formal subgroup interaction test. Post-hoc subgroups require a PROSPERO amendment with a visible record.

5. Supplementary 8-file package is the minimum bar for high-impact journals: PRISMA checklist, PROSPERO PDF, full search strategy, full-text exclusion list with reasons, per-study extraction table, per-study x per-domain RoB, subgroup forests, sensitivity / publication-bias analyses. See templates/supplementary_8file_checklist.md.

6. PROSPERO 14-char ID format (^CRD42\d{9}$ = CRD42 + 4-digit year + 5-digit sequence, e.g. CRD42024500001). A 15-character ID is a stray-digit transcription error; pre-2020 IDs may be shorter. Validate with grep -oE 'CRD42[0-9]+' + length assert, and request the live registration URL in the cover letter for protocol cross-check.

7. AI Disclosure presence for SR-MA submissions to RYAI / Radiology / RSNA / Lancet / JAMA / BMJ / Nature families. Absence triggers MINOR-to-MAJOR finding at peer review.

8. Sensitivity analyses are recomputed, not copied (Phase 6b rule 5). Leave-one-out / erosion / alternative-model effect sizes identical to the primary analysis to 2 decimals across ≥4 values means the recomputation did not run. Re-derive from the modified dataset; the inputs (means/SDs/counts) change even when the effect size is close.

9. Outcome harmonization before pooling. Studies that report the same-named outcome under different definitions (an imaging-detected event vs a clinically diagnosed one; different thresholds) must not be presented as a single pooled range or pooled estimate. Split by ascertainment method (or pool only the harmonizable subset) and state the definition per stratum — a "6.9–46%" range that silently mixes imaging-detected and clinical events is a heterogeneity artifact, not a finding.

10. Heterogeneous RoB instruments → no single pooled κ. When studies are assessed with different risk-of-bias tools (QUADAS-2 for DTA + NOS for cohorts, etc.), do not report one pooled inter-rater κ across the mixed set. Report agreement per instrument, and use an ordinal weighted κ when the domain judgments are ordered (low/some/high). A single κ over a heterogeneous instrument set is uninterpretable.

9. Prognostic / survival-outcome MAs carry survival-specific concerns beyond the DTA pitfalls: censoring handling, competing risks (cause-specific vs Fine-Gray), cutoff-derivation optimism, comparator time-horizon alignment, C-index variant transparency (Harrell vs Uno vs IPCW), and calibration beyond discrimination. When pooling prognostic models, pre-specify these in the protocol and report them per study; for the reviewing counterpart see the survival/prognostic 7-probe in /peer-review.

Empirical Lessons (2026-06)

From a CBCT lung-ablation SR-MA submission cycle (Springer / CVIR Editorial Manager). Submission-stage; complements the 2026-05 lessons.

11. Supplementary materials need the same blinding + de-scaffolding + cross-consistency pass as the manuscript. The largest source of pre-submission defects this cycle was the supplement shipping as raw internal artifacts — /check-reporting output carrying an "Assessed by: <AI tool>" line and a JSON verdict block, and a pre-search planning doc with the author's real name, sibling-project cross-references, unresolved [Check on execution] placeholders, and estimate tables that contradicted the actual PRISMA counts. Presence (Lesson 5) is not enough; apply the Phase 8 supplementary gate.

12. A submitted analysis script must reproduce the manuscript and be self-contained. A hard-coded study-id subset silently drifted (a pool was k=7 in the script vs k=9 in the results table — the manuscript was correct, the script was stale) and the script read a path outside the bundle. Run the bundled code from a clean copy before submission: it must read the bundled dataset, write to cwd, and regenerate every reported pool. Remove stale figures produced by an out-of-sync script.

13. Re-sync sidecars (cover letter, title-page Word-Counts table) whenever the reference or word count changes. Adding methodological/software citations took the list from 12 to 24, but the cover letter and title page still said "12 references" — a contradiction visible in the built PDF. Reference/word-count changes are sidecar drift targets (mirror submission-portal-verification cover-letter drift).

14. Methodological + software citations are a routine SR-MA gap. The reporting standard (PRISMA 2020), each risk-of-bias tool (JBI, ROBINS-I, …), the pooling method (random-effects GLMM / logit, Hartung-Knapp, the choice over Freeman-Tukey arcsine), the certainty framework (GRADE), and the analysis software (R meta, metafor) should each be cited where named in Methods. Frequently missing from an early draft and an easy reviewer comment to pre-empt. Verify every added citation via PubMed/CrossRef with a first-author cross-check — never from memory.

15. Wide characteristics tables (≥ ~10 columns) render as character-wrapped gibberish in the journal's built PDF when the docx uses fixed narrow columns. Put the table in a landscape section with autofit layout and a smaller font, and verify by converting the docx to PDF (soffice --headless --convert-to pdf) and viewing the page — the docx alone does not reveal the problem.

16. Verify the submission portal's journal identity before entering metadata. A classification taxonomy that does not match the target journal's scope (e.g., a liver/hepatology list at an interventional-radiology journal) is the tell that you are in the wrong journal's Editorial Manager instance.

---

DTA-Specific Pitfalls (Always Check)

---

Small Study Considerations

When the number of included studies is small (< 10):

• Bivariate/HSROC model may not converge -- consider univariate random-effects as fallback
• Publication bias tests are underpowered -- state this limitation
• Subgroup/meta-regression analysis not recommended
• Wide prediction regions expected -- emphasize uncertainty in conclusions
• Consider narrative synthesis as alternative/complement

---

Skill Interactions

---

Error Handling

• If study type is ambiguous (DTA vs intervention), ask user to clarify before proceeding.
• If fewer than 4 studies for DTA, warn that bivariate model may not converge.
• If data extraction is incomplete (missing 2x2 cells), suggest contacting authors or sensitivity analysis with imputed values.
• If PROSPERO ID is missing, flag as a limitation but continue.
• Always remind user: this is a methodological support tool; final decisions rest with the research team and ideally include a biostatistician/methodologist.

Anti-Hallucination

• Never fabricate variable names, dataset column names, or variable codings. If a variable mapping is uncertain, output [VERIFY: variable_name] and ask the user to confirm against the data dictionary.
• Never fabricate statistical results — no invented p-values, effect sizes, confidence intervals, or sample sizes. All numbers must come from executed code output.
• Never generate references from memory. Use /search-lit for all citations.
• If a function, package, or API does not exist or you are unsure, say so explicitly rather than guessing.

JBI Critical Appraisal Checklist for Case Series

Reference: Munn Z, Barker TH, Moola S, et al. Methodological quality of case series studies: an introduction to the JBI critical appraisal tool. JBI Evid Synth. 2020;18(10):2127-2133.

10 Signalling Questions

Scoring

• Y = Yes (criterion met)
• N = No (criterion not met)
• U = Unclear
• NA = Not applicable

Overall Appraisal

Notes

• Q4 and Q5 distinguish consecutive vs complete inclusion:
• Q4 (consecutive): Were participants enrolled sequentially without selection bias?
• Q5 (complete): Were all eligible participants during the study period included?
• Q9 applies to multi-site studies; for single-site studies, mark as NA if site demographics

are adequately covered by Q6.

• When used alongside NOS for mixed comparative + single-arm meta-analyses, map NOS to

comparative studies and JBI to case series. In PROSPERO, select "Other" for RoB tool and specify both.

Newcastle-Ottawa Scale (NOS) Assessment Guide

Quality assessment tool for non-randomised studies in meta-analyses. Reference: Wells GA et al. Ottawa Hospital Research Institute.

Structure

NOS uses a "star system" (maximum 9 stars) across 3 categories. Higher stars = higher quality.

Cohort Studies (max 9 stars)

Selection (max 4 stars)

1. Representativeness of the exposed cohort (1 star)

• a) Truly representative of the average [describe] in the community *
• b) Somewhat representative *
• c) Selected group of users
• d) No description

2. Selection of the non-exposed cohort (1 star)

• a) Drawn from the same community as the exposed *
• b) Drawn from a different source
• c) No description

3. Ascertainment of exposure (1 star)

• a) Secure record (e.g., surgical records) *
• b) Structured interview *
• c) Written self-report
• d) No description

4. Demonstration that outcome was not present at start (1 star)

• a) Yes *
• b) No

Comparability (max 2 stars)

5. Comparability of cohorts on the basis of design or analysis (up to 2 stars)

• a) Study controls for [most important factor] *
• b) Study controls for any additional factor *

Outcome (max 3 stars)

6. Assessment of outcome (1 star)

• a) Independent blind assessment *
• b) Record linkage *
• c) Self-report
• d) No description

7. Was follow-up long enough for outcomes to occur? (1 star)

• a) Yes (select adequate follow-up period) *
• b) No

8. Adequacy of follow-up of cohorts (1 star)

• a) Complete follow-up (all subjects accounted for) *
• b) Subjects lost to follow-up unlikely to introduce bias (<=20% lost, or description provided) *
• c) Follow-up rate <80% and no description of those lost
• d) No statement

Case-Control Studies (max 9 stars)

Selection (max 4 stars)

1. Is the case definition adequate? 2. Representativeness of the cases 3. Selection of controls 4. Definition of controls

Comparability (max 2 stars)

5. Comparability of cases and controls (same 2-star system)

Exposure (max 3 stars)

6. Ascertainment of exposure 7. Same method of ascertainment for cases and controls 8. Non-response rate

Interpretation

Alternative thresholds may be pre-specified in the protocol.

When to Use

• Observational cohort studies in intervention or exposure meta-analyses
• Case-control studies
• Simpler alternative to ROBINS-I when full domain-level assessment is not needed
• Note: NOS does not provide domain-level judgments -- only an aggregate score

PRISMA-DTA Checklist

Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy Studies. Reference: McInnes MDF et al. JAMA 2018;319(4):388-396.

Checklist Items

PROBAST Assessment Guide

Prediction model Risk Of Bias ASsessment Tool. Reference: Wolff RF et al. Ann Intern Med 2019;170(1):51-58. PMID: 30596875. AI extension: PROBAST+AI (BMJ 2024).

Structure

PROBAST assesses 4 domains, each for Risk of Bias AND Applicability.

• Signalling questions: Yes / Probably yes / No / Probably no / No information
• Domain judgment: Low / High / Unclear
• Overall judgment: High if any domain is high; Low only if all domains are low

Domain 1: Participants

Signalling Questions (Risk of Bias)

1. Were appropriate data sources used (e.g., cohort, RCT, nested case-control)? 2. Were all inclusions and exclusions of participants appropriate?

Applicability

• Do the participants and setting match the review question?

Domain 2: Predictors

Signalling Questions (Risk of Bias)

1. Were predictors defined and assessed in a similar way for all participants? 2. Were predictor assessments made without knowledge of outcome data? 3. Are all predictors available at the time the model is intended to be used?

Applicability

• Do the predictors, their assessment, and timing match the review question?

Domain 3: Outcome

Signalling Questions (Risk of Bias)

1. Was the outcome determined appropriately? 2. Was a pre-specified or standard outcome definition used? 3. Were predictors excluded from the outcome definition? 4. Was the outcome defined and determined in a similar way for all participants? 5. Was the outcome determined without knowledge of predictor information? 6. Was the time interval between predictor assessment and outcome appropriate?

Applicability

• Does the outcome and its definition/timing match the review question?

Domain 4: Analysis

Signalling Questions (Risk of Bias)

1. Were there a reasonable number of participants with the outcome? 2. Were continuous and categorical predictors handled appropriately? 3. Were all enrolled participants included in the analysis? 4. Were participants with missing data handled appropriately? 5. Was selection of predictors based on univariable analysis avoided? 6. Were complexities in the data accounted for appropriately? 7. Were relevant model performance measures evaluated appropriately? 8. Were model overfitting and optimism in model performance accounted for? 9. Do predictors and their assigned weights in the final model correspond to the reported multivariable analysis?

For Validation Studies (additional)

• Were the model and its performance evaluated appropriately?

PROBAST+AI Extensions (2024)

For AI/ML prediction models, additional considerations:

• Data: Training/validation/test split, data leakage check
• Model: Architecture transparency, hyperparameter tuning method
• Performance: Discrimination (AUC), calibration, fairness across subgroups
• Reproducibility: Code/data availability, external validation

When to Use

• Diagnostic prediction models (e.g., AI classifiers for imaging findings)
• Prognostic prediction models (e.g., risk scores, survival prediction)
• Both development AND validation studies
• Use PROBAST+AI when the model involves machine learning or deep learning

QUADAS-2 Assessment Guide

Quality Assessment of Diagnostic Accuracy Studies, version 2. Reference: Whiting PF et al. Ann Intern Med 2011;155(8):529-536.

Structure

QUADAS-2 assesses 4 domains. Each domain has:

• Signalling questions: answered Yes / No / Unclear
• Risk of bias judgment: Low / High / Unclear
• Applicability concern (domains 1-3 only): Low / High / Unclear

Domain 1: Patient Selection

Signalling Questions

1. Was a consecutive or random sample of patients enrolled? 2. Was a case-control design avoided? 3. Did the study avoid inappropriate exclusions?

Risk of Bias

• Low: Yes to all signalling questions
• High: No to any signalling question
• Unclear: Insufficient information

Applicability

• Are there concerns that the included patients and setting do not match the review question?

Domain 2: Index Test

Signalling Questions

1. Were the index test results interpreted without knowledge of the results of the reference standard? 2. If a threshold was used, was it pre-specified?

Risk of Bias

• Low: Yes to all signalling questions
• High: No to any signalling question
• Unclear: Insufficient information

Applicability

• Are there concerns that the index test, its conduct, or interpretation differ from the review question?

Domain 3: Reference Standard

Signalling Questions

1. Is the reference standard likely to correctly classify the target condition? 2. Were the reference standard results interpreted without knowledge of the results of the index test?

Risk of Bias

• Low: Yes to all signalling questions
• High: No to any signalling question
• Unclear: Insufficient information

Applicability

• Are there concerns that the target condition as defined by the reference standard does not match the question?

Domain 4: Flow and Timing

Signalling Questions

1. Was there an appropriate interval between index test and reference standard? 2. Did all patients receive the same reference standard? 3. Were all patients included in the analysis?

Risk of Bias

• Low: Yes to all signalling questions
• High: No to any signalling question
• Unclear: Insufficient information

(No applicability concern for this domain)

Common Issues in DTA Studies

• Partial verification bias: Not all patients receive the reference standard (especially when invasive, e.g., biopsy)
• Differential verification: Different reference standards used for different patients
• Incorporation bias: Index test forms part of the reference standard
• Review bias: Knowledge of index test results influences reference standard interpretation
• Clinical review bias: Additional clinical information available during index test interpretation
• Uninterpretable results: Exclusion of technically inadequate or indeterminate results

RoB 2 Assessment Guide

Revised Cochrane Risk-of-Bias tool for Randomised Trials. Reference: Sterne JAC et al. BMJ 2019;366:l4898. PMID: 31462531.

Structure

RoB 2 assesses 5 domains for each outcome in each study.

• Signalling questions: Yes / Probably yes / Probably no / No / No information
• Domain judgment: Low risk / Some concerns / High risk
• Overall judgment: Most conservative across all domains

Domain 1: Bias Arising from the Randomisation Process

Signalling Questions

1. Was the allocation sequence random? 2. Was the allocation sequence concealed until participants were enrolled and assigned? 3. Did baseline differences between groups suggest a problem with the randomisation process?

Judgment

• Low: Adequate random sequence generation AND allocation concealment, no problematic baseline imbalances
• Some concerns: Information insufficient to permit judgment
• High: Inadequate sequence generation or concealment, or problematic baseline imbalances suggest failure

Domain 2: Bias Due to Deviations from Intended Interventions

Effect of Interest: Assignment (intention-to-treat)

1. Were participants aware of their assigned intervention during the trial? 2. Were carers and people delivering the interventions aware of assigned intervention? 3. Were there deviations from the intended intervention that arose because of the trial context? 4. Were these deviations likely to have affected the outcome? 5. Was an appropriate analysis used to estimate the effect of assignment to intervention?

Effect of Interest: Adherence (per-protocol)

1-2. Same as above 3. Were important non-protocol interventions balanced across groups? 4. Could failures in implementing the intervention have affected the outcome? 5. Did trial participants adhere to the assigned intervention regimen? 6. Was an appropriate analysis used to estimate the effect of adhering to intervention?

Domain 3: Bias Due to Missing Outcome Data

Signalling Questions

1. Were data for this outcome available for all, or nearly all, participants randomised? 2. Is there evidence that the result was not biased by missing outcome data? 3. Could missingness in the outcome depend on its true value? 4. Is it likely that missingness depended on its true value?

Judgment

• Low: Data available for all/nearly all (>95%), or evidence result not biased by missing data
• Some concerns: Missingness could depend on true value but unlikely
• High: Missingness likely dependent on true value

Domain 4: Bias in Measurement of the Outcome

Signalling Questions

1. Was the method of measuring the outcome inappropriate? 2. Could measurement or ascertainment of the outcome have differed between groups? 3. Were outcome assessors aware of the intervention received by study participants? 4. Could assessment of the outcome have been influenced by knowledge of intervention? 5. Is it likely that assessment was influenced by knowledge of intervention?

Domain 5: Bias in Selection of the Reported Result

Signalling Questions

1. Were the data that produced this result analysed in accordance with a pre-specified analysis plan finalised before unblinded outcome data were available? 2. Is the numerical result likely to have been selected from multiple eligible outcome measurements or analyses of the data?

Overall Risk of Bias

• Low risk: Low risk in all domains
• Some concerns: Some concerns in at least one domain, but not high risk in any
• High risk: High risk in at least one domain, OR some concerns in multiple domains in a way that substantially lowers confidence

When to Use

• Use for individually randomised, parallel-group trials (default)
• Variants available for: cluster-randomised trials, crossover trials
• Do NOT use for non-randomised studies (use ROBINS-I instead)

ROBINS-I Assessment Guide

Risk Of Bias In Non-randomised Studies - of Interventions. Reference: Sterne JAC et al. BMJ 2016;355:i4919.

Structure

ROBINS-I assesses 7 domains + overall judgment.

• Signalling questions: Yes / Probably yes / Probably no / No / No information
• Domain judgment: Low / Moderate / Serious / Critical / No information
• Overall judgment: Lowest of all domain judgments (most conservative)

Pre-assessment Requirements

Before applying ROBINS-I, specify: 1. The target trial (what RCT would ideally answer this question?) 2. The effect of interest (assignment to intervention vs starting and adhering) 3. Confounders to be controlled

Domain 1: Bias Due to Confounding

Key Questions

• Is there potential for confounding not accounted for?
• Did the authors use appropriate methods to control confounding (matching, regression, propensity score)?

Judgment

• Low: All critical confounders appropriately controlled
• Moderate: Minor concerns about residual confounding
• Serious: Important confounders not adequately controlled
• Critical: Confounding so severe that no useful estimate possible

Domain 2: Bias in Selection of Participants

Key Questions

• Was selection into the study related to both intervention and outcome?
• Was start of follow-up and intervention aligned?
• Were adjustments made for different start times?

Domain 3: Bias in Classification of Interventions

Key Questions

• Were intervention groups clearly defined?
• Was information used to classify interventions recorded at the start of the intervention?
• Could classification of intervention status have been affected by knowledge of the outcome?

Domain 4: Bias Due to Deviations from Intended Interventions

Key Questions

• Were there deviations from intended intervention beyond what would be expected?
• Were these deviations unbalanced between groups and likely to affect outcomes?
• Were important co-interventions balanced across groups?

Domain 5: Bias Due to Missing Data

Key Questions

• Were outcome data available for all or nearly all participants?
• Were participants excluded due to missing data on intervention or other variables?
• Was the proportion of missing data similar across groups?
• Were appropriate methods used to handle missing data?

Domain 6: Bias in Measurement of Outcomes

Key Questions

• Could outcome measurement have been influenced by knowledge of intervention?
• Were outcome assessors blinded?
• Were outcome measures comparable across groups?

Domain 7: Bias in Selection of Reported Result

Key Questions

• Were multiple outcome measurements reported?
• Were multiple analyses performed?
• Is the reported result likely selected from among multiple measurements or analyses?

Overall Risk of Bias

The overall judgment is the most conservative across all domains:

• Low: Low risk in all domains
• Moderate: Low or moderate in all domains
• Serious: Serious in at least one domain, but not critical in any
• Critical: Critical in at least one domain

Recommendation for Synthesis

• Studies at critical risk of bias should be excluded from meta-analysis
• Present critical studies in a separate table for completeness
• Conduct sensitivity analysis excluding serious risk of bias studies

Data Integrity Checklist

Applies to: the full span of Phase 3 (Extraction) ~ Phase 6 (Statistical synthesis) ~ Phase 9 (Circulation). Blocks the numerical-consistency risks specific to meta-analysis.

1. Extraction stage

DI-1. Formalize the extraction consensus log

• Problem pattern: Comparative extraction results kept only as inline comments inside the R analysis script — no standalone consensus log → arm-specific numbers become irretraceable once the script is edited.
• Rule: At the start of every MA project, create 2_Data/extraction_consensus_log.md or 3_Extraction/extraction_consensus_log.md. Columns: study_id, arm, numerator, denominator, source_page, source_type (text/table/figure/KM-reconstruction), extractor_initials, second_reviewer_initials, timestamp, notes. Comparative extractions must live only as formal rows, never as script comments.

DI-2. Mandatory double-check of 2x2 cell counts

• Problem pattern: Hand-typed 2x2 cells with arm order swapped against the source paper, or with numerator/denominator misread from a KM-derived subgroup rather than the raw table. Both fail silently until a third reviewer back-calculates the proportion.
• Rule: Every 2x2 / comparative extraction requires (a) a first extraction + (b) an independent second re-extraction + (c) source re-check on any mismatch + a consensus-log row. Perform the Phase 6b "numerical safety gate".

DI-3. Complete the KM-reconstruction audit trail

• Problem pattern: Subgroup cell counts reconstructed from a published KM curve without preserving the WebPlotDigitizer trace or the IPDfromKM reconstruction log → numbers cannot be re-derived if a reviewer challenges them.
• Rule: KM-reconstruction outputs must be kept as a set in 3_Extraction/km_reconstruction/{study_id}/: (a) the WebPlotDigitizer JSON, (b) the IPDfromKM CSV, and (c) metadata for tool version + coordinate values + parameters + date. Link them in the consensus log as type "km-reconstruction".

DI-4. Denominator change = source page citation + consensus-log row

• Problem pattern: Denominator correction (e.g., treatment-naive subset) entered only as an R-script comment without citing the source paper's page/table → the correction's rationale is lost at revision.
• Rule: Every denominator change requires (a) a source page/table citation, (b) a sentence stating the rationale, and (c) one consensus-log row. Reject the change if any of the three is missing.

DI-5. Methodology mismatch random spot-check

• Problem pattern: A source paper reports per-protocol analysis while the SR framework is ITT/ITD (or vice versa). Without a methodology spot-check, the study's effect estimate is silently re-used under a different analysis framework.
• Rule: Include a "methodology flag" in the extraction spot-check scope — whether each study's source analysis unit (per-protocol / ITT / ITD) matches our SR framework. Re-extract on mismatch.

2. 3~5-way consistency

DI-6. PRISMA flow 5-way consistency

• Problem pattern: PRISMA flow numbers drift across the search CSV, the screening log, the Methods prose, the Results prose, and the Figure 1 caption — five surfaces reach submission in three mutually inconsistent states (reversed database order, divergent full-text-assessed counts, stale caption numbers).
• Rule: Verify the PRISMA flow numbers as consistent across five places simultaneously:

1. 1_Search/*.csv original (source of truth, no edits) 2. 2_Screening/prisma_flow_final.md 3. Manuscript Methods prose 4. Manuscript Results prose (where mentioned again) 5. Figure 1 caption (both 5_Figures/_captions.md and the short DOCX caption)

• ID-Set Gate Rule 5: fix the prose first → then render the diagram. If the diagram is built first, editing the prose causes drift.
• Automation candidate: manage k, n, and the search numbers in a single YAML source and substitute them into the prose/diagram templates.

DI-7. Single source for k = 4_Analysis/*.csv

• Problem pattern: Included-study count k quoted as two different values across consecutive manuscript versions because it was hand-typed into prose rather than derived from the analysis CSV.
• Rule: Derive k only from the 4_Analysis/*.csv row count. When entering k into the manuscript / MANIFEST / PROSPERO, also state the CSV path + row count.

3. Pre-submission cleanup

DI-8. Remove every tag / TODO

• Problem pattern: TODO, [VERIFY-CSV], "to be regenerated" strings survive into the submission package — found in R scripts, figure captions, and supplementary material banners.
• Rule: 0 hits for the following grep before submission day:

  rg -n "VERIFY-CSV|TODO|FIXME|XXX|to be regenerated|PH TODO|to-do" \
     7_Manuscript/ supplement/ 5_Figures/ 6_Tables/ 1_Code/

• `[VERIFY-CSV]` lifecycle: attach (v6) → verify (v7) → mark (v8+) → remove (submission). Record each stage in the MANIFEST.

DI-9. State bias-driven homogeneity interpretation

• Problem pattern: In a DTA pool where every included study is retrospective with differential verification, Sp I²=0% reflects universalized bias compression rather than true between-study homogeneity — easy to over-interpret as robust agreement.
• Rule: When I²=0% and QUADAS-2 Domain 4 risk are both high/unclear → state "universalized bias compression" explicitly and frame as an "upper-bound estimate". Bias context is mandatory in the Discussion.

ICMJE COI Form Generation Guide

Overview

Most journals (Springer/CVIR, Lancet, npj, etc.) require ICMJE Conflict of Interest disclosure forms from all authors. This guide documents how to batch-generate pre-filled forms using python-docx.

Template Location

Use any existing blank ICMJE form as template. Suggested locations:

• A previously-used filled form from one of your manuscripts (under <project>/submission/{journal}/icmje_forms/).
• Download a blank from https://www.icmje.org/disclosure-of-interest/.

ICMJE Form Structure (docx)

Table 0 (Header): 6 rows × 2 cols
  Row 0: Title ("ICMJE DISCLOSURE FORM")
  Row 1: Date
  Row 2: Author name        ← PRE-FILL THIS
  Row 3: Manuscript title    ← PRE-FILL THIS
  Row 4: Manuscript number   ← Leave blank or fill if known
  Row 5: Instructions text

Table 1 (Disclosures): 19 rows × 5 cols
  Row 0: Column headers
  Row 1: Time frame header (since initial planning)
  Row 2: Item 1 — Support for present manuscript
  Row 3: Time frame header (past 36 months)
  Rows 4-15: Items 2-13 (grants, royalties, consulting, etc.)
    → Col 3: Entity name (leave blank if no conflict)
    → Col 4: Specifications (leave blank if no conflict)
  Row 16: Empty
  Row 17: Certification header
  Row 18: Certification text  ← ADD "X" to certify

Python Script: Batch Generate

#!/usr/bin/env python3
"""Batch-generate ICMJE COI forms from a template."""
import shutil
from pathlib import Path
from docx import Document

def generate_coi_forms(
    template_path: str,
    output_dir: str,
    manuscript_title: str,
    authors: list[tuple[str, str]],  # [(name, email), ...]
    manuscript_number: str = "",
    date_str: str = "",
):
    """
    Generate one ICMJE COI form per author.
    
    KNOWN ISSUES (python-docx + ICMJE form):
    1. Merged cells: Table 0 Row 0 and Table 1 headers use merged cells.
       Writing to cell.text replaces ALL merged cell content.
       → Use cell.paragraphs[0].runs approach instead.
    2. Formatting loss: Direct cell.text assignment strips bold/italic.
       → Preserve runs and only modify run.text.
    3. XML namespace errors: Some ICMJE forms have custom XML.
       python-docx may warn about "lxml.etree.XMLSyntaxError".
       → These warnings are usually harmless; the output is still valid.
    """
    out = Path(output_dir)
    out.mkdir(parents=True, exist_ok=True)
    
    for i, (name, email) in enumerate(authors, 1):
        # Copy template to preserve all formatting
        fname = f"ICMJE_COI_{i:02d}_{name.replace(' ', '_')}.docx"
        dst = out / fname
        shutil.copy2(template_path, dst)
        
        doc = Document(str(dst))
        t0 = doc.tables[0]
        
        # Pre-fill header fields
        # IMPORTANT: Access the cell, find existing text, append to it
        # Don't replace — some cells have label + value in same cell
        
        # Row 1: Date
        if date_str:
            _safe_append(t0.rows[1].cells[0], f" {date_str}")
        
        # Row 2: Author name
        _safe_append(t0.rows[2].cells[0], f" {name}")
        
        # Row 3: Manuscript title
        _safe_append(t0.rows[3].cells[0], f" {manuscript_title}")
        
        # Row 4: Manuscript number
        if manuscript_number:
            _safe_append(t0.rows[4].cells[0], f" {manuscript_number}")
        
        doc.save(str(dst))
        print(f"  Created: {fname}")
    
    print(f"\nGenerated {len(authors)} COI forms in {output_dir}")


def _safe_append(cell, text):
    """Append text to a cell without destroying existing formatting.
    
    This avoids the common python-docx pitfall where cell.text = "new"
    destroys all runs and formatting in merged cells.
    """
    if cell.paragraphs:
        p = cell.paragraphs[0]
        if p.runs:
            # Append to last run to preserve formatting
            p.runs[-1].text += text
        else:
            # No runs — add one
            run = p.add_run(text)
    else:
        cell.text = text


# === Example usage ===
if __name__ == "__main__":
    # Replace with the actual author roster for your manuscript.
    authors = [
        ("Author One", "author1@example.com"),
        ("Author Two", "author2@example.com"),
        ("Author Three", "author3@example.com"),
        # ...
    ]

    generate_coi_forms(
        template_path="path/to/blank_ICMJE_template.docx",
        output_dir="submission/{journal}/icmje_coi_forms/",
        manuscript_title="Your Manuscript Title Here",
        authors=authors,
        date_str="YYYY-MM-DD",
    )

Common Pitfalls

1. Merged Cell Destruction

Problem: cell.text = "new value" on a merged cell deletes content in ALL merged cells. Fix: Use _safe_append() or access cell.paragraphs[0].runs[-1].text.

2. XML/SQL-like Errors

Problem: lxml.etree.XMLSyntaxError when opening some ICMJE forms. This happens because ICMJE's official template contains custom XML namespaces that python-docx doesn't fully support. Fix: These warnings are cosmetic. The output file is still valid Word doc. If errors persist, use shutil.copy2() first, then open the copy.

3. Formatting Loss on Save

Problem: Bold/italic disappears after saving. Fix: Never assign to cell.text directly. Always work through cell.paragraphs[0].runs to preserve formatting.

4. Empty Cells for "No Conflict"

Problem: Some journals want explicit "None" vs empty cell. Fix: Check journal-specific requirements. CVIR (Springer) accepts empty cells.

Workflow for New Projects

1. Copy any existing ICMJE form as template (or download blank from icmje.org) 2. Update authors list with names and emails 3. Update manuscript_title 4. Run script → generates N individual docx files 5. Send to co-authors for review/signature 6. Collect signed forms before submission

Author Email List (JSON format)

Store alongside COI forms for reference:

[
  ["Author Name", "email@example.com"],
  ...
]

Checklist Licenses

Attribution for bundled reporting guideline checklists.

These files are educational summaries of published assessment tools. The original checklist documents should be cited in any manuscript that uses them. All files are compatible with the MIT license of this repository.

Phase 10 Reference — Self-Audit Recovery (v{N} → v{N+1} Sprint)

Load this reference when any Phase 10 trigger fires during /meta-analysis. The SKILL.md body carries only the trigger table and summary pointer; all procedural detail lives here.

Goal

When an audit — internal, or triggered by an incoming co-author comment — uncovers a structural data or protocol-application error, withdraw the current version, rebuild, and re-circulate with a transparent audit trail. Catching the error yourself before a journal reviewer does is the principal trust-building move in this phase.

Trigger Conditions (any one)

• Extraction CSV disagrees with the primary source for a cell that materially feeds a

pooled estimate, subgroup estimate, or a reported proportion.

• A study that was excluded (or included) by the pre-specified criteria is found to

violate those criteria on re-read.

• A hand-typed numerical literal in the analysis script traces to a wrong value (see the

Phase 6b precedent failure pattern).

• The PROSPERO (or equivalent) protocol and the delivered analysis disagree on a

pre-specified outcome, subgroup, or eligibility rule.

• A dual-reviewer consensus record shows a study was supposed to be excluded, but the

locked dataset still contains it (or vice versa).

Non-Negotiable Rule

If the trigger fires after Phase 9 circulation but before journal submission, withdraw the current version within 24 hours. Reviewer discovery is a strictly worse failure mode than self-withdrawal.

10.1 Audit Log

Create qc/audit_vN_to_vNplus1.md. For each flagged item, record:

• Affected study (first author, year).
• Cell or claim (e.g., TP/FP/FN/TN, k, pooled estimate, CI, subgroup count).
• Source of the error (extraction, application of criteria, script literal, protocol

mismatch, reviewer-consensus mismatch).

• Proposed resolution (exclude study, re-extract cell, correct script literal, amend

protocol, reinstate study).

10.2 CSV Re-Verification

For every flagged cell, re-verify against the primary source (paper Table/Figure, page number). Any revision-introduced value must carry the [VERIFY-CSV] tag until the Phase 6b audit clears it.

10.3 Re-Run the Analysis Script

Re-execute the full synthesis pipeline with a fixed random seed. Save complete output to analysis/vNplus1_run.log. Do not re-use partial v{N} output even for subsections that "should be unchanged" — prove it with a fresh run.

10.4 Manuscript Auto-Sync

Every number the manuscript reports — k, pooled estimate, 95% CI, τ², I², subgroup counts, sensitivity-analysis rows, GRADE levels — must be pulled from the fresh output. Grep the manuscript for old v{N} numerical values to confirm no residue survives.

10.5 Supplementary Regeneration

Re-emit (do not patch):

• Extraction consensus log.
• RoB tables.
• GRADE / Summary of Findings table.
• PRISMA flow diagram (counts shift when studies are excluded/reinstated).

10.6 Figure Regeneration

Regenerate forest plots, funnel plots, subgroup plots, and the PRISMA flow via /make-figures. Re-embed at the original manuscript positions. Confirm figure captions match the new values (captions often contain pooled estimates).

10.7 Change Summary

Produce v{N+1}_change_summary.md with an explicit delta table:

This file is the external audit trail. It must be circulated with the v{N+1} manuscript.

10.8 PROSPERO (or Equivalent) Amendment

If eligibility, analysis population, or outcome definitions changed, file an amendment to the registered protocol in parallel with the re-build:

• Keep revision notes within the registry's word limit (often 250 words).
• Frame the amendment as application correction, not criteria change:

"The pre-specified inclusion criteria are unchanged. The amended application excludes

{N} study(ies) whose characteristics were confirmed on independent re-review to fall

outside the pre-specified criteria."

• Submit the amendment before — or simultaneously with — re-circulation, so co-authors

see an amendment already in flight rather than only a draft change.

10.9 Re-Circulation Framing

Use the Phase 9 thread. State the situation plainly in the first paragraph:

"On re-review of v{N}, we identified {N} study(ies) whose data did not meet the

pre-specified inclusion criteria. These were excluded and the analysis regenerated.

The manuscript body, supplementary, and all figures have been re-rendered. A change

summary is attached."

Tonal anchors worth preserving:

• "On re-review" — signals internal audit, not external pressure.
• "Pre-specified criteria" — the protocol itself is unchanged; application is corrected.
• "Change summary" — an external audit trail is available for scrutiny.

10.10 Anti-Patterns (do not)

• Hide the error and submit v{N} as-is. This converts a recoverable finding into a

retraction-class incident if the reviewer catches it.

• Reframe the fix as a "minor revision" with no audit trail. Senior reviewers read audit

trails; absence of one signals concealment.

• Disclose only in the cover letter while leaving Methods silent. Methods and the cover

letter must agree.

10.11 When the Trigger Fires Post-Submission

If the audit trigger is detected after the journal submission but before an editorial decision, notify the editorial office immediately and submit a corrected manuscript alongside the disclosure. Do not wait for the reviewer to surface it; editors prefer author-initiated correction.

10.12 Post-Recovery Loop

After v{N+1} is circulated, Phase 9 restarts. If a further audit trigger emerges during the new review window, a second recovery sprint (v{N+1} → v{N+2}) is acceptable — but each recovery costs co-author goodwill, so the incoming audit in Phase 6b should be tightened each cycle rather than relying on Phase 10 as a routine catch.

Phase 4 Reference — KM Reconstruction & Composite Exposure Disaggregation

Load this reference when /meta-analysis Phase 4 data extraction encounters either of two special cases: (a) studies that report outcomes only as Kaplan-Meier curves without raw event counts, or (b) studies whose intervention is a composite of multiple techniques. The main Phase 4 body of SKILL.md lists the standard extraction-form fields and the cross-verification checklist; this reference holds the procedural detail for these two scenarios.

---

4b. KM Curve Reconstruction (when raw events not reported)

When studies report outcomes only as Kaplan-Meier curves without raw event counts:

1. Digitise the KM curve: Use WebPlotDigitizer (https://automeris.io/WebPlotDigitizer/)

• Calibrate X/Y axes carefully — verify output range matches the original axis labels.
• If coordinates come out in 0–1 range, multiply X by the actual time range

(e.g., ×30 for months).

• Clip negative Y values to 0 (digitisation artifact).
• Export as CSV: time, cumulative_event_rate (or survival).

2. Extract number-at-risk: Record from the table below the KM plot at each time point.

3. Reconstruct IPD: Use the R IPDfromKM package (Guyot et al. 2012 method):

   library(IPDfromKM)
   dat <- read.csv("digitised_curve.csv")
   preproc <- preprocess(dat, trisk, nrisk, totalpts, maxy = 1)
   ipd <- getIPD(preproc, armID = 1)  # armID starts at 1, NOT 0

• ⚠️ preprocess() does NOT accept a mateflag parameter (common error).
• ⚠️ armID starts at 1 (not 0).

4. Verify: Generate a reconstructed KM plot and visually compare to the original figure.

5. Report in Methods: Cite Guyot et al. 2012 (doi:10.1186/1471-2288-12-9) and state which studies required reconstruction.

Alternative — Text-based extraction: When no subgroup-specific KM curve exists but the text reports "0% LTP at 12 months" or similar, extract directly from text. Document the page number and exact quote.

---

Composite Exposure Disaggregation

When a study's intervention is a composite of multiple techniques:

1. Subgroup-specific KM curve → use KM reconstruction (section 4b above). 2. Component-specific Table/multivariate → extract per-component data from Tables. 3. Text-based subgroup report → extract from narrative (e.g., "APE arm: 0% LTP"). 4. None available → include as composite; flag in sensitivity analysis for exclusion.

Always pre-specify a sensitivity analysis excluding composite-exposure studies. Document the extraction strategy in the data extraction form Notes column.

Phase 6 Reference — Statistical Synthesis

Load this reference when /meta-analysis Phase 6 begins executing the pooled analysis. It contains the R code templates for DTA and intervention meta-analysis, the dual-approach (comparative + single-arm pooled proportion) decision table, and the practical cautions (method.tau, HK CI, zero-cell correction, publication-bias test power).

Always use R (packages: meta, metafor, mada). Companion templates: ${CLAUDE_SKILL_DIR}/references/r_templates.md.

---

DTA Meta-Analysis

library(mada)      # bivariate model, forest/SROC plots
library(meta)      # general meta-analysis utilities
library(metafor)   # advanced models

# Bivariate model (recommended for DTA)
fit <- reitsma(data, formula = cbind(tsens, tfpr) ~ 1)
summary(fit)

# SROC curve with confidence and prediction regions
plot(fit, sroclwd = 2, main = "SROC Curve")

# Forest plot (paired: sensitivity + specificity)
forest(fit, type = "sens")
forest(fit, type = "spec")

Key outputs for DTA

• Pooled sensitivity (95% CI)
• Pooled specificity (95% CI)
• Pooled positive LR, negative LR
• Pooled DOR
• SROC curve with AUC, confidence region, prediction region
• Heterogeneity: I-squared for sensitivity and specificity separately
• Threshold effect: Spearman correlation between sensitivity and FPR

---

Intervention Meta-Analysis

library(meta)
library(metafor)

res <- metagen(TE, seTE, data = dat, studlab = study,
               method.tau = "REML", sm = "OR")
forest(res)
funnel(res)

summary(res)  # I-squared, tau-squared, Q test
metabias(res, method.bias = "Egger")
metainf(res, pooled = "random")  # leave-one-out

---

Dual Approach: Comparative + Single-Arm Pooled Proportion

When both comparative and single-arm studies are available, use dual analysis (precedent: Lin 2025 PMID:41419890, Su 2026 PMID:41653198). The assignment of PRIMARY vs SECONDARY depends on the research question and available evidence:

The choice should be pre-specified in the PROSPERO protocol and remain consistent throughout the manuscript.

# Comparative MA (binary outcomes)
res_comp <- metabin(ei, ni, ec, nc, data = dat,
                     studlab = study, sm = "OR",
                     method = "Inverse", method.tau = "DL",
                     common = FALSE, random = TRUE,
                     method.random.ci = "HK", incr = 0.5)

# Single-arm pooled proportion
res_prop <- metaprop(event, n, data = dat_single,
                      studlab = study, sm = "PLOGIT",
                      method.tau = "DL", method.ci = "CP")

Key points

• Comparative answers "is adjunct effective?"; single-arm answers "what outcomes to expect?"
• Single-arm uses metaprop() with logit transformation + Clopper-Pearson CI
• GRADE certainty lower for single-arm — state explicitly
• Report both in Results: label PRIMARY/SECONDARY per pre-specified assignment
• Selection bias warning: Single-arm case series may introduce selection bias

(experienced centres, favourable patients). When pooling with comparative arms, report both pooled estimates separately and discuss any numerically lower event rate in single-arm studies as a potential selection effect.

---

Practical R Notes

• Use method = "Inverse", not "MH", to avoid a method.tau conflict.
• Use method.tau = "DL" (DerSimonian-Laird) — REML may not converge with sparse data.
• Use method.random.ci = "HK" (Hartung-Knapp) instead of the deprecated hakn = TRUE.
• Use common = FALSE, random = TRUE instead of deprecated comb.fixed/comb.random.
• For zero cells in binary 2×2 outcomes (OR/RR), apply incr = 0.5 continuity correction. Do NOT apply a continuity correction when pooling single-arm proportions: use metaprop(..., method = "GLMM", sm = "PLOGIT"), which handles zero-event studies natively. See single_arm_proportion_ma.md.
• Egger's test is underpowered for k < 10 — note this in results. Egger/funnel tests are invalid for pooled proportions (the SE is a deterministic function of the proportion); see single_arm_proportion_ma.md.

---

Subgroup / Meta-Regression

• Subgroup analysis for pre-specified covariates
• Meta-regression for continuous moderators
• Report interaction test p-value, not just within-subgroup p-values

---

Publication Bias

• DTA: Deeks' funnel plot asymmetry test (standard funnel plots are inappropriate for DTA).
• Intervention: Funnel plot + Egger's or Peters' test.
• Note: tests are underpowered for <10 studies.

---

Sensitivity Analysis

• Leave-one-out analysis (metainf())
• Excluding high RoB studies
• Excluding overlapping populations (same institution + enrollment period)
• Including/excluding borderline studies (sensitivity to inclusion criteria)
• Alternative model specifications

---

Error Handling

• If an R script fails, capture the error message, diagnose the likely cause

(missing package, data format mismatch, convergence failure), and present a fix. Do not silently re-run.

• When reporting R output, separate statistical results (pooled estimates,

heterogeneity metrics, I-squared) from interpretation. Present numbers first in a "Statistical Results" block, then interpretation guidance in a separate "Interpretation Notes" block.

Phase 9 Reference — Co-author Circulation

Load this reference when Phase 9 is entered (after Phase 8 is complete and Phase 6b source-fidelity audit has cleared). The SKILL.md body carries only the goal + summary pointer; all procedural detail lives here.

Goal

Standardized pre-submission circulation of the manuscript to co-authors and senior methodologists / reviewers, with a bounded review window and a controlled attachment scope.

Trigger

Phase 8 is complete, and the draft has cleared Phase 6b source-fidelity audit.

9.1 Thread Continuity

• If a prior version (v1, v2, ...) of this manuscript was already circulated to the same

author team, reply to the same email thread. Preserve In-Reply-To and References headers so the thread tracks v1 → v2 → v3 in one place.

• Open a new thread only for the first circulation, or when the author team / target

journal has fundamentally changed.

9.2 Attachment Scope

Include:

• Manuscript body with figures embedded inline (single DOCX/PDF).
• Change summary — for v≥2, a delta table vs. the prior circulated version.

Exclude (circulate separately later):

• Graphical abstract — after the body is locked.
• Cover letter — after the target journal is confirmed.
• COI forms — after journal format is known.
• Supplementary appendices — share via a drive link, not attached.

Rationale: bundling all submission artifacts before the body is locked forces co-authors to review multiple moving targets at once and telegraphs premature commitment to a specific journal (especially when the GA carries journal-specific branding).

9.3 Attachment Method

A manuscript DOCX with inline figures is typically 0.5 – 1 MB — direct attach is safe.

9.4 Recipient Structure

• TO: Corresponding author + one senior methodologist reviewer (e.g., a protocol

assessor or statistician external to the writing team).

• CC: All remaining co-authors. Include every alternate email address a co-author has

used on the thread to avoid dropping them.

Ask the corresponding author explicitly for:

• target journal preference,
• 2 – 3 reviewer candidates,
• any framing adjustments to carry into the cover letter.

9.5 Journal-Undetermined Framing

If the target journal is not yet confirmed, state this explicitly:

"The manuscript is currently formatted to [placeholder journal]'s guidelines. Please

suggest an appropriate target journal and we will re-format accordingly."

Do not include the graphical abstract in this round if it carries journal-specific branding that would require rework on re-targeting.

9.6 Deadline

• Set (send date + 7 days) = 5 business days + 1 weekend.
• Record the deadline in a task tracker with expected responses per recipient.
• Informal follow-up is acceptable after 4 – 5 days of silence. A formal second message

before the 7-day window closes is not appropriate unless an external submission deadline forces it.

9.7 Response Tracking

For each recipient, log: date of response, issues raised, and whether the issues trigger Phase 10 (Self-Audit Recovery) or are minor in-place revisions.

Post-Submission Release Operations

When: between the end of Phase 9 (circulation) and journal submission. Separate from Phase 10 (recovery).

Why it's hard: during circulation (v7~v18 coexisting), the numbers keep shifting. Mint the DOI too early and a content mismatch forces a re-issue; too late and you are scrambling on submission day.

Checklist

Gate 1 — When to issue the release

• [ ] Internal circulation closed: every reviewer (internal PI, external peer) has signed off.
• [ ] [VERIFY-CSV] / TODO / FIXME / (to be regenerated) tags fully removed from manuscript / supplement / figures / code (rg -n → 0 hits).
• [ ] k (number of included studies) matches across the 4_Analysis/*.csv row count, the manuscript prose, the PRISMA flow, and every figure caption.
• [ ] Author order / ICMJE COI finalized (just before reflecting it in the Zenodo author metadata).

Gate 2 — GitHub repo

• [ ] Journal-target bundle regenerates successfully via _build.sh.
• [ ] Repo includes the raw analysis code, extraction_consensus_log.md, PROSPERO amendments tracker, and methodology.
• [ ] README has a DOI placeholder (replaced after the Zenodo DOI is issued).
• [ ] .gitignore confirmed to exclude raw PDFs / copyrighted material.
• [ ] LICENSE stated (CC-BY 4.0 or the journal-required license).

Gate 3 — Zenodo DOI

• [ ] Zenodo record metadata: author order, affiliation, ORCID, keywords, related identifiers (PROSPERO registration number).
• [ ] Add the GitHub repo release tag URL to related_identifiers as isSupplementTo.
• [ ] The submission package (.tar.gz) is uploaded to Zenodo — the journal submission bundle, not the circulation package (vN).
• [ ] After the DOI is issued, reflect it in the "data availability" section of the manuscript / cover letter / submission portal.

Gate 4 — Handling re-targeting after rejection

• [ ] Resubmission of identical content to another journal: do NOT mint a new Zenodo version (the DOI attaches to the content). Use _build.sh --journal {new} to create only a new SUBMISSION folder.
• [ ] Resubmission after revision: content changed → mint a new Zenodo version. The concept DOI is preserved.
• [ ] On author changes: redistribute the ICMJE COI + mint a new Zenodo version.

Common failures

• F1: Zenodo DOI minted while k (included study count) is still oscillating between versions → content-DOI mismatch forces re-issue. Blocked by: Gate 1.
• F2: Journal-specific folders edited by hand without a _build.sh → the journal copies drift from master. Blocked by: see submission_package_drift.md.
• F3: TODO / FIXME tag left in an R analysis script surfaces only after the repo is pushed to GitHub. Blocked by: Gate 1 rg scope including code.

PROSPERO Registration Template

Overview

PROSPERO (International Prospective Register of Systematic Reviews) requires completion of a structured web form. This template provides field-by-field guidance with word limits and common pitfalls from real registration experience.

Key constraint: Initial registrations are NOT allowed after data extraction is completed. Searching and screening may be completed.

---

Form Fields Reference

REVIEW TITLE AND BASIC DETAILS

1. Review title (max 50 words)

Include: study design (systematic review, meta-analysis, pooled analysis), condition, intervention, population qualifier if relevant.

2. Review question (max 250 words)

Structure around PICO/PIRD. State both primary and secondary objectives clearly. For pooled analysis designs, distinguish between pooled proportion and comparative questions.

3. Condition/domain being studied (max 200 words)

Define the condition. Explain why it matters. Note the clinical gap.

4. Keywords

Semicolon-separated. Include MeSH-equivalent terms.

---

ELIGIBILITY CRITERIA

5. Population (max 200 words)

State included and excluded populations. Include age, disease status, anatomical specifics.

6. Intervention/exposure (max 200 words)

Define all eligible interventions with specifics (technique, device, imaging guidance). State minimum reporting requirement ("must report at least one clinical outcome").

7. Comparator/control (max 200 words)

If both comparative and single-arm studies are included, state this explicitly: "For the comparative analysis, [comparator description]. For the pooled proportion analysis, single-arm studies without a comparator are also included."

8. Type of study to be included (max 150 words)

List included and excluded study designs separately. Be specific about minimum sample size (e.g., "case reports with fewer than 5 patients excluded").

---

SEARCHING AND SCREENING

9. Searches (max 300 words)

Name databases, date range, language restrictions. Reference supplementary search strategies.

10. Search strategy (max 300 words)

Show actual search blocks with Boolean structure. No need for full line-by-line syntax here (reference supplementary materials for that).

Sub-fields in the web form:

• Search for unpublished studies: Yes/No (select based on protocol)
• Main databases: Select from checklist (PubMed, Embase.com vs Embase via Ovid — distinguish!)
• Language restrictions: State explicitly
• Date restrictions: State explicitly
• Other methods: Check ONLY what was actually done:
• reference list checking (backward citation)
• contacting authors
• conference proceedings
• trial registers
• forward citation searching (snowballing)
• WARNING: Do not check methods you did not perform — reviewers may ask for results
• Link to search strategy: Reference protocol or supplementary materials
• Selection process: "Two reviewers independently..." with disagreement resolution
• Other info about screening: PRISMA flow diagram reference (optional)

---

DATA COLLECTION PROCESS

11. Data extraction (max 300 words)

List extracted variables by category: study characteristics, population, intervention, outcomes. State unit of analysis policy (patient vs tumor level). If Kaplan-Meier reconstruction is used, cite methods (Tierney 2007, Guyot 2012).

Sub-fields:

• Extraction method: Select "independently by at least two people"
• Author contact: Yes/No
• IPD: Usually No for standard MA

12. Risk of bias/Quality assessment (max 200 words)

Name tool(s) per study design. If using tools not in the PROSPERO checklist (e.g., JBI), select "Other" and describe in text.

Common tool mapping:

State number of assessors and disagreement resolution.

13. Reporting bias assessment

For pooled proportion (k>=10): funnel plot + Egger's regression test. For comparative (k<10): funnel plot visual inspection only. Do NOT promise trim-and-fill or contour-enhanced funnel unless actually planned.

14. Certainty assessment

Select "No" unless GRADE or equivalent is explicitly planned in the protocol. If "Yes": requires Summary of Findings table — significant additional work.

---

OUTCOMES TO BE ANALYSED

15. Primary outcomes (max 300 words)

Define each outcome precisely: what it measures, how it is defined, at what time points. State the effect measure (proportion, OR, HR, etc.).

16. Secondary outcomes (max 300 words)

List all secondary outcomes. Include exploratory outcomes that will be "extracted if reported."

---

PLANNED DATA SYNTHESIS

17. Strategy for data synthesis (max 400 words)

Structure as: 1. Primary analysis: model, estimator, transformation 2. Secondary analysis: model, estimator, CI adjustment 3. Heterogeneity: I-squared, Q test, interpretation thresholds 4. Publication bias: methods (reference field 13) 5. Software: R packages with versions

18. Subgroups/subsets (max 250 words)

Number and list all pre-specified subgroup analyses. Number and list all pre-specified sensitivity analyses. State interaction test method. Mention leave-one-out if planned.

---

CURRENT REVIEW STAGE

19. Review stage checklist

CRITICAL: If data extraction is "Completed", registration will be REJECTED. If protocol restructuring invalidates prior extraction → defensible to mark "Not started."

20. Review status

Usually: "The review is planned or ongoing"

---

REVIEW AFFILIATION, FUNDING AND PEER REVIEW

21. Review team members

All authors with ORCID and affiliation. One must be Guarantor + Named Contact. Guarantor = typically first or corresponding author.

22. Funding

• If no funding: "This review received no specific funding from any agency in the public,

commercial, or not-for-profit sectors."

• Do NOT write "supported by academic institutions" unless there is actual institutional funding.

23. Peer review

Usually "Not peer reviewed" for initial registration.

---

ADDITIONAL INFORMATION

24. Additional info (max 250 words)

Standard text: "Any important protocol amendments will be documented in the PROSPERO record and clearly reported in the final publication."

25. MeSH terms

Auto-generated but editable. Review and add missing terms (e.g., Microwaves for MWA studies).

---

Common Pitfalls Checklist

Before submitting, verify:

• [ ] HTML entities: Preview PDF and check all >=, <, & symbols display correctly

(common breakage: >, <, & appearing literally)

• [ ] Word limits: Each field within stated maximum
• [ ] Database names: Embase.com vs Embase via Ovid (different options in form)
• [ ] Other methods: Only checked items actually performed
• [ ] RoB tool: JBI requires "Other" selection (not in standard list)
• [ ] Certainty/GRADE: "No" unless explicitly planned with SoF table
• [ ] Funding text: Matches actual funding status
• [ ] Data extraction stage: Not marked as "Completed"
• [ ] Similar records: All reviewed and confirmed "not similar"
• [ ] All authors: Listed with correct ORCID and affiliation
• [ ] End date: In the future (update if past)
• [ ] Comparator field: Addresses both comparative and single-arm inclusion if applicable

---

Output Format

When generating PROSPERO content for the user: 1. Produce a single Markdown file with all fields, word counts per field 2. Convert to DOCX via pandoc for copy-paste convenience 3. Flag any fields requiring user decision (e.g., author contact, unpublished search) 4. Include the Common Pitfalls Checklist at the end