Self Review

Self-Review Skill

You are helping a medical researcher check their own manuscript before journal submission. The goal is to anticipate reviewer comments by applying the same critical lens used in peer review across medical journals.

This is NOT about writing a review. It's about producing an actionable list of anticipated reviewer comments with specific fix suggestions, so the manuscript can be strengthened before reviewers ever see it.

Optional Flags

• --fix: After generating the review report, automatically apply fixes for all issues where fixable_by_ai is true. Edits the manuscript in place, then reports a diff summary. Does NOT fix issues marked fixable_by_ai: false (e.g., missing data, design flaws). Maximum 2 fix-and-re-review iterations.
• --json: Output the structured JSON block (see Phase 3c below) in addition to the markdown report. Default when called from /write-paper Phase 7.
• --panel: Run the multi-agent panel review (Phase 2.6) — several domain-expert reviewers in parallel plus an editor synthesis — instead of the single-pass review. Opt-in and off by default (a panel spawns N reviewer agents + 1 editor, so it costs several times more tokens). Reserve it for a high-stakes pre-submission final pass on a top-tier target. Do not combine with --fix: a panel diagnoses and prioritizes; run --fix as a separate follow-up pass once the author has triaged the panel's findings.

Severity Framing

When flagging issues, classify severity:

• Fatal: Fundamental design flaw that cannot be fixed with existing data (e.g., data leakage

that invalidates all results, absence of any reference standard, label-feature circularity). The manuscript likely needs redesign. Submission would likely result in Reject.

• Fixable: Significant but addressable with existing data (e.g., missing calibration analysis,

unclear exclusion criteria, absent CIs, incomplete reporting). These are the most actionable findings.

Most issues are Fixable. Reserve Fatal for true design-level problems.

Workflow

Phase 1: Intake

1. Get the manuscript -- PDF, Word doc, or pasted text. 2. Ask the user:

• Target journal? (affects reporting standards and scope expectations)
• Manuscript type? (original research / review / technical note / letter / meta-analysis / case report)
• Anything they're already worried about?
• Review depth? The default is a single-pass review. For a high-stakes pre-submission final pass, a multi-agent panel (--panel, Phase 2.6) is available — several domain-expert reviewers run independently, then an editor consolidates them (more thorough, but it spawns several agents so it costs several times more tokens). On an interactive run, surface this option once in one line and offer it; then proceed with the single-pass review unless the user opts in. Do not surface or auto-apply the panel when invoked with --json or from /write-paper — those stay single-pass.

3. Read the full manuscript. 4. SSOT gate — confirm there is one manuscript, not several. Self-review reads a single input file, so a divergence between a legacy working copy and the live submission copy is structurally invisible to it. Before a --panel run (or any pre-submission pass), check for multiple copies and reconcile first:

   find . \( -path '*manuscript*' -o -path '*main_document*' \) -name '*.md' | grep -v node_modules

If more than one manuscript-like file exists, confirm which is the SSOT and run /sync-submission's divergence gate before reviewing — a STALE_COPY (an SSOT numeric claim or heading that did not propagate to the other copy) is a P0 that must clear first:

   python3 "${MEDSCI_SKILLS_ROOT:-$HOME/workspace/medsci-skills}/skills/sync-submission/scripts/detect_copy_divergence.py" \
     --ssot <ssot>.md --copy <other-copy>.md

Review the SSOT copy; do not review a stale copy and pass it.

In `--panel` mode this is a blocking precondition, not advice. A panel spawns N reviewer agents + an editor, so reviewing a stale copy wastes the whole pass (a prior panel's top finding was literally "you reviewed the wrong file"). If the find above returns more than one manuscript-like .md and the SSOT is not pinned — no SSOT.yaml with truth.manuscript_md and no explicit --ssot <path> argument — STOP before spawning any reviewer and have the user name the SSOT (and clear any STALE_COPY). Do not auto-pick the longest/newest file. The single-pass review may proceed on the one file it was given, but the panel must not.

Phase 2: Systematic Check

Run the manuscript through each applicable category below. For each item, assess whether a reviewer would raise it as a Major or Minor comment.

Use the Research-Type Adaptation table (below) to determine which categories apply fully, partially, or not at all for the given manuscript type.

A. Study Design & Data Integrity

B. Reference Standard & Ground Truth

C. Validation & Statistical Reporting

D. Clinical Framing & Importance

Run the deterministic scope gate:

python3 "${CLAUDE_SKILL_DIR}/scripts/check_scope_coherence.py" \
  --manuscript manuscript.md --out qc/scope_coherence.json --strict

CROSS_SECTIONAL_PROGNOSTIC and SURROGATE_CARE_DIRECTIVE are Anticipated Major Comments (category: D. Clinical Framing). CROSS_SECTIONAL_YIELD_LANGUAGE is an Anticipated Minor Comment — a cross-sectional / prevalence design using incidence-flavored screening vocabulary ("yield", "detection rate", "number-needed-to-screen/image", "rescreen interval") without defining "yield" once as cross-sectional report-positive prevalence. The gate is conservative — it fires only when a design/endpoint signal and a conclusion-region action verb (or the yield lexicon) co-occur.

E. Reproducibility

F. Reporting Completeness

G. Reporting Guideline Compliance

Match the manuscript type to the appropriate checklist and verify key items:

For a full item-by-item audit, run /check-reporting on this manuscript. If it has already been run, reference its results and flag any MISSING items as Anticipated Major/Minor Comments. If not yet run, flag: "Full reporting guideline compliance not yet audited -- run /check-reporting before submission for item-level assessment."

H. Circularity

I. Protocol Heterogeneity

J. Method Transparency

Run the deterministic classical-style lint (these are all greps, so they belong in a gate, not eyeballing):

python3 "${CLAUDE_SKILL_DIR}/scripts/check_classical_style.py" \
  --manuscript manuscript.md --out qc/classical_style.json --strict

SECTION_SYMBOL and INBODY_AI_DISCLOSURE are Major (the § count must be 0; the AI-disclosure paragraph belongs on the title page for a classical / senior-MA target, not the body). ELIGIBILITY_PROSE, DECIMAL_INCONSISTENCY, and EM_DASH_OVERUSE are Minor. This is the self-review-side mirror of /write-paper Step 7.1's classical QC (manuscript-style-classical §5/§6/§7/§8).

K. Reviewer-team consistency (SR/MA-only; fabrication-grade)

Run the deterministic check at Phase 2 entry (pass the extraction JSON — a file or a directory of per-study JSONs — so the prose↔JSON↔confession 3-way is covered):

python "${CLAUDE_SKILL_DIR}/scripts/check_reviewer_team_consistency.py" \
    --manuscript manuscript.md \
    --prospero prospero/record.md \
    --extraction-json extraction/ \
    --out _audit_self/reviewer_team_consistency.md

Exit 1 = MAJOR red flag. The JSON sidecar carries dual_hits, single_hits, llm_reviewer_hits, and deferred_mitigation_hits. Any of the DUAL+SINGLE conjunction, an LLM reviewer field, or a deferred mitigation trips it. Either of the dual/single claims alone is fine; the conjunction is read by reviewers as fabrication. Resolution path: 1. Honest Methods/PROSPERO update (single-reviewer execution disclosed), OR 2. Limitations confession rewritten if dual review was actually completed.

Research-Type Adaptation

Not all categories apply equally to every study type. Use this routing table:

*Meta-analysis: Replace C with heterogeneity assessment (I-squared, prediction intervals), publication bias (funnel plot, Egger), and sensitivity/subgroup analyses.

Type-Specific Additional Checks:

• Observational studies: Confounding assessment (DAG or adjustment strategy), selection bias, exposure measurement validity. Run Phase 2.5e (Confounding Completeness) and apply the O1–O14 probes in references/domain-probes/observational_confounding.md — including O7 (over-adjustment: do not adjust for a consequence/mediator of the outcome, e.g. serum uric acid in an eGFR model — the opposite-direction failure to O1), O8 (analysis unit & clustering — run check_cohort_arithmetic.py --id-col for records-vs-subjects), O9 (construct validity of a report-/registry-derived outcome), O10 (an inferential effect-size gradient across overlapping/nested subsets needs a difference/interaction test, not descriptive refinement alone), and — for complex-survey data (NHANES/KNHANES/CHNS) — O11 (design-based weighting: the right weight + strata + PSU, subpopulation-not-subset) and O12 (data-driven inflection-point/'saturation' threshold mining needs a breakpoint CI + pre-specification, not a quoted cutoff), O13 (a cross-sectional mediation claim cannot establish X→M→Y order and needs an unmeasured-M–Y-confounding sensitivity), and O14 (a synergy/joint-effect/effect-modification claim needs the additive scale — RERI/AP/S with CIs — not a multiplicative-only interaction or joint-category ORs). If the manuscript develops or compares a clinical prediction model (TRIPOD / TRIPOD+AI, nested predictor-set comparison), also apply the CP1–CP4 probes in references/domain-probes/clinical_prediction_model.md (apparent-vs-optimism-corrected calibration/DCA, the incremental-value-vs-marginal-effect two-null distinction, EPV per nested model, net benefit as model comparison not policy).
• Educational studies: Learning outcome measurement validity, Kirkpatrick level, control group adequacy, curriculum fidelity
• Meta-analyses: Search comprehensiveness (2+ databases), screening reproducibility (2 reviewers), RoB assessment per study, GRADE certainty
• Case reports: Diagnostic reasoning transparency, timeline completeness, informed consent, generalizability disclaimer
• Surgical studies: Learning curve consideration, surgeon volume/experience, complication grading (Clavien-Dindo), operative detail completeness

Domain probe modules (load when the manuscript type matches):

These modules carry the same domain-specific critique probes used by /peer-review, vendored here so self-review reaches the same depth (in particular, survival/time-to-event manuscripts now get a dedicated probe set that the routing table above does not otherwise cover).

When the manuscript matches a row, read ${CLAUDE_SKILL_DIR}/references/domain-probes/<module>.md and apply each probe as an additional source of Anticipated Major / Minor Comments. The module severity words (MAJOR / MINOR) map to this skill's framing as follows: a conclusion-threatening or design-level finding becomes a Fatal Anticipated Major Comment, a reporting-level finding becomes a Fixable Anticipated Minor Comment, and each is tagged with the closest category letter (A–K). These probes complement categories A–K above; they do not replace them. (The modules are vendored byte-identical from /peer-review; do not edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync.)

Phase 2.5: Numerical Cross-Verification (Internal)

Before generating the report, verify internal consistency:

1. Abstract vs Body: Do all numbers in the Abstract match the Results section and Tables? 2. Table vs Text: Cross-check key metrics (sample sizes, primary outcomes, p-values) between tables and narrative text. 3. Figure vs Text: Do figure legends match the data described in Results? 4. Percentage arithmetic: Verify that n/N percentages are calculated correctly (e.g., 23/150 = 15.3%, not 15.0%). 5. CI plausibility: Do confidence intervals seem reasonable given sample sizes? 6. Rate back-calculation: every reported rate must invert to its own numerator/denominator — an incidence rate ≈ events / person-years × scale (±rounding). A rate that does not recompute from the stated events and person-time (or that implies more events than the cohort can supply) is a Major, not a Minor. 7. Exclusion-cascade and complete-case arithmetic (cohort/observational): the STROBE flow must balance — start N − Σ(exclusions) == final analytic N — and any complete-case statement must balance — total − missing == complete. A footnote N that does not equal the subtraction is a Major.

For cohort/observational manuscripts, run the deterministic gate instead of eyeballing it (it parses prose equations + GFM tables, and recomputes from a committed CSV when given one):

python3 "${CLAUDE_SKILL_DIR}/scripts/check_cohort_arithmetic.py" \
  --manuscript manuscript.md --data analysis/cohort.csv --id-col mockid \
  --out qc/cohort_arithmetic.json --strict

RATE_BACKCALC / CASCADE_SUM / PARTITION_OVERLAP rows are Anticipated Major Comments (category: A. Study Design & Data Integrity); the partition check is the Phase 2.5b cohort branch below. Pass --id-col (or let it auto-detect a subject-ID column) on health-screening / EMR / registry data so the gate also runs the analysis-unit check: when records > unique subjects and the manuscript states neither the analysis unit nor a one-record-per-subject sensitivity, it emits ANALYSIS_UNIT_UNDISCLOSED (Major — non-independent observations give anti-conservative CIs; probe O8). Flag any remaining internal-consistency discrepancies as Anticipated Minor Comments (category: F. Reporting Completeness).

Phase 2.5a: Numerical Source-Fidelity Audit (External)

Internal consistency (Phase 2.5) is necessary but not sufficient. Numbers can be fully self- consistent across Abstract / Table / Text and still be wrong at the source — a single transcription error propagates cleanly through every downstream stage.

Precedent failure pattern:

A revision-era comparative meta-analysis reported a safety-outcome 2x2 with the

arm-level events direction-reversed relative to the primary-source Table. Internal

consistency passed because Abstract, Discussion, Table, and the R script all echoed

the same wrong values. The reversal was caught only by an explicit second-pass audit

that randomly sampled claims and traced each back to the primary paper.

When to run: MA revisions, submissions, or any review where the user mentions "check against the source," "verify extraction," or "random sample."

Inputs the reviewer should expect:

• manuscript.md (or .docx converted to .md)
• extraction_final.csv (or equivalent data-extraction spreadsheet)
• A directory of primary-source PDFs (or equivalent accessible text)

Procedure:

1. Inventory numerical claims in Abstract, Results, and Discussion (patterns: \\d+/\\d+, \\d+\\.\\d+%, (95% CI:, p\\s*=\\s*0\\., I\\^2, n\\s*=, etc.).

2. Stratified random sample — draw 5 claims across: (a) pooled estimates, (b) subgroup / sensitivity results, (c) comparative-arm specific values, (d) study-level numbers (first-cited in narrative), (e) a claim introduced during revision if the draft is post-v1. Comparative-arm specific values and revision-introduced numbers are the two highest- yield strata — always include one of each.

3. For each sampled claim, traverse 3 layers:

• Layer 1 (Manuscript → CSV): Find the row / column in the extraction CSV.
• Layer 2 (CSV → Primary source): Locate the exact Table, Figure, or paragraph in the

original paper. Record page number.

• Layer 3 (Analysis script → CSV): If the claim came from an analysis script, read the

script and confirm its input value matches the CSV cell.

4. Record results in a table and append to the report:

5. Any mismatch is a Major Comment (M-level), not Minor. Mismatches that reverse a direction or change a significance boundary are P0 blockers for submission.

Revision-specific rule: If the manuscript contains [VERIFY-CSV] tags, treat each as a mandatory audit item regardless of the sampling size. The tag exists precisely because that number was introduced after the initial extraction pass and has not yet been independently checked.

Hand-entered analysis-script inputs are a code smell. When Layer 3 reveals a matrix(...), c(1, 2, 3), or data.frame(...) line with numerical data and no CSV-coordinate comment, escalate to a Major Comment even if the audited values happen to match — the next revision will re-introduce the same risk.

Statistic-type fidelity (not just the value). A prose sentence must match the table/CSV not only on the number but on the statistic type. A body sentence that reports a median ("median eGFR 92.8") while Table 1 reports a mean ("mean 91.3") for the same variable cannot be reconciled by a reviewer comparing the two — and the mismatch usually means one of them was not regenerated after a Table 1 rule change (see the mean/median-by-skewness rule in /analyze-stats table-types/table1_demographics.md). Treat a prose↔table statistic-type mismatch (mean vs median, SD vs IQR, n vs %) as a Minor Comment, or Major if it sits on a primary characteristic the conclusion leans on. Also re-check that any descriptive figure the prose quotes (e.g. "78.4% male") matches the current table value, not a stale earlier one.

Stale derived CSVs after a model/adjustment-set change (n mismatch). When the primary model or adjustment set changes mid-revision, every derived CSV (Table 2, sensitivity tables, supplements) must be regenerated, or a stale file silently contradicts the new primary. The fastest tell is the analytic n: if a derived CSV's n differs from the manuscript's current primary n, suspect it is stale — and the conflict can flip a result's significance (a proteinuria sensitivity CSV left at the old n = 4,914 / OR 4.52 contradicted the new primary n = 4,214 / OR 3.99, significant ↔ not). Grep each derived CSV's n against the primary n; any divergence that is not explained by a stated sub-analysis restriction is a Major Comment, requires_reanalysis (re-run, not a prose edit — see Phase 4).

Phase 2.5a-2: Design & Power Statistic Provenance (computed, not extracted)

Phase 2.5a traces data-derived numbers back to a CSV and a primary source. Design and power statistics are a different class and a common blind spot: the minimum detectable effect (MDE), a-priori or post-hoc power, the required sample size for a future trial, and the a-priori effect-size assumptions behind them are computed, not extracted, so they have no CSV row or source-paper Table to trace to. They routinely escape both the internal-consistency check and the source-fidelity audit above.

Precedent failure pattern:

A pilot study reported a minimum detectable effect of d = 1.67. No standard two-sample method

reproduces it (the correct value at the stated n, alpha, and power was about 1.24). It survived

several review rounds because no committed script computed it — the value had been hand-entered —

and one reviewer even cited the figure approvingly. In the same manuscript, a set of future-trial

sample sizes was numerically correct but had been produced with an exact noncentral-t tool, while

the committed script used a normal approximation and printed different numbers: right value, no

reproducible provenance.

Procedure:

1. Inventory design/power claims. Search for: "minimum detectable", "detectable effect", "MDE", "power" (80% / 90% / "1 − beta"), "sample size", "n = N per arm/group", "to detect", "powered to", "a priori", and any a-priori planning effect size (Cohen's d / f / OR used for sizing).

2. Require a reproducible source for each. Every such value must be produced by committed code (e.g. statsmodels TTestIndPower, a G*Power-equivalent, or an explicit noncentral-t computation), with the inputs stated in the manuscript: n per arm, alpha, power, allocation ratio, and one- vs two-sided. A value with no committed-code source is the highest-risk case.

3. Recompute independently with a standard tool, then classify:

• Not reproducible by any standard method → likely a calculation error (Major; P0 if it

is a headline claim). This is the d = 1.67-vs-1.24 case above.

• Reproducible only by a method the committed script does not implement (e.g. the

manuscript value is noncentral-t but the script is a normal approximation) → provenance / method drift. The number may be correct, but update the committed code so it reproduces the reported value (Major: reproducibility, not correctness).

4. Method-consistency across the manuscript. All power, sample-size, and MDE statistics in one paper should share a single method family (e.g. all noncentral-t). A mix of normal approximation and exact-t within one manuscript signals that some values were computed in an ad-hoc side tool.

5. Any non-reproducible design/power value is a Major Comment; a non-reproducible headline power or MDE claim is a P0 submission blocker.

Hand-entered design/power statistics are a code smell even when correct. If no committed function emits the value, flag it: the next revision will re-introduce the risk, and a reviewer who recomputes will not match the manuscript.

`POWER_MODEL_MISSPEC` — the power/MDE simulation's adjustment set must match the primary model. For cohort "negative findings," the whole conclusion leans on the MDE ("the literature effect of 1.2–1.5 cannot be excluded"), so the MDE must be computed under the same covariate set as the primary analysis. When a committed power/MDE script exists, read its model formula: if it fits y ~ exposure + age (2 covariates) while the primary model adjusts for 6, it overstates power (omitted covariates inflate the apparent precision) — the MDE is too small and the negative claim too strong. Re-running a parametric bootstrap under the full model is the fix (in one worked case MDE moved from a 2-covariate "OR 1.67" to a full-model "OR ≈ 1.70"). A power/MDE whose script omits primary-model covariates → Major (P0 when the MDE is a headline). This is requires_reanalysis (re-simulate, not a prose edit). `POWER_VALUE_INTERPOLATED` — any interpolat/approx/interp token in a power/MDE CSV's provenance column means the headline value was never simulated on the grid; treat a non-reproducible headline power/MDE as Major.

Phase 2.5b: Screening-Count Reconciliation from ID Sets (SR/MA + observational tier/stratum)

Internal consistency across Abstract/Methods/Results (Phase 2.5) + source fidelity of 2×2 and effect-size numbers (Phase 2.5a) do not cover study-count arithmetic. The latter is a separate failure mode: a prior-draft prose total ("30 → 32 after FLAG consensus") can survive every downstream pass because Abstract, Methods, Results, Discussion, Figure 1 caption, and even the supplementary consensus file all cite the same wrong number back to each other.

Precedent failure pattern (a PRISMA-DTA meta-analysis revision):

A late-revision manuscript reported study counts of k_qualitative = 32, k_narrative-only = 10,

k_FT-excluded = 46. An ID-level recount against the screening TSV and consensus sheet (with

FLAG additions reconciled) yielded k_qualitative = 24 with only 2 narrative-only studies

(k_FT-excluded = 54). The original 32/10/46 figures came from an early-draft assumption that

was never reconciled against the ID-level artifacts; downstream files (consensus markdown,

supplementary tables, edit plans) propagated the same wrong total. Caught only by an explicit

ID-set recount against the screening TSV and consensus spreadsheet, verified independently

by an adversarial audit.

When to run: any SR/MA manuscript revision, regardless of stage. Run before Phase 3.

Inputs:

• Screening TSV with one row per full-text-reviewed record and an include/exclude column
• Consensus spreadsheet (Excel/CSV) with one row per record requiring adjudication and a

Consensus column (typical values: Exclude, Include-qualitative, Include-bivariate)

• Any FLAG-adjudicated inclusion log documenting records added to the qualitative pool

outside the primary screening TSV

• The manuscript's Table 1 (or equivalent): the definitive list of studies contributing to

the primary quantitative synthesis

Procedure:

1. Enumerate the ID sets:

• A = set of IDs marked INCLUDE in the screening TSV
• B = set of IDs marked Exclude in the consensus spreadsheet
• C = set of IDs marked Include-qualitative in the consensus spreadsheet
• T = set of IDs represented in Table 1 (via author/year cross-match)

2. Derive canonical totals:

• k_qualitative = |A \ B| + |C|
• k_bivariate = |T|
• k_narrative-only = k_qualitative − k_bivariate = |(A ∪ C) \ B \ T|
• k_FT-excluded = |screening TSV rows| − |A| + |B ∩ A| + |(B \ A) encountered at FT stage|

3. List the narrative-only IDs explicitly — this is the highest-yield cross-check. A manuscript claiming "10 narrative-only studies" while the (A ∪ C) \ B \ T set contains only 2 IDs is an immediate P0 finding.

4. Compare each derived total against the manuscript's prose claim in Abstract, Methods §Study Selection, Results §Study Selection, Figure 1 caption, Discussion §Limitations, and any References §Narrative-Only heading. Any mismatch between derived total and manuscript prose = P0 Major Comment, blocking submission.

5. Record results in a short reconciliation block and append to the report:

   | Quantity | Manuscript claim | ID-derived value | Status |
   |---|---|---|---|
   | k_full-text | 78 | 78 | ✓ |
   | k_qualitative | 32 | 24 | ✗ P0 |
   | k_bivariate | 22 | 22 | ✓ |
   | k_narrative-only | 10 | 2 (IDs 120, 474) | ✗ P0 |
   | k_FT-excluded | 46 | 54 | ✗ P0 |

Any "N → M" transition claim in a consensus summary (e.g., "30 → 32 after FLAG consensus") that is not backed by an enumerable ID addition/subtraction set is itself a Major Comment, because the transition is unverifiable by downstream audit. Require conversion of every such claim to explicit ID lists before closing the report.

Observational tier/stratum branch. The same set-recount logic applies when a cohort manuscript presents an ordinal tier or mutually-exclusive stratum split. A partition that is claimed to be disjoint must satisfy Σ(stratum N) == unique total and Σ(stratum events) == total events; denominators that sum above the unique cohort double-count subjects, and a table where every stratum n equals the grand total is a stratum-total mis-entry rather than a partition. Run check_cohort_arithmetic.py (Phase 2.5 above) with the stratum CSV — its PARTITION_OVERLAP verdict is the cohort analogue of an ID-set mismatch and is a P0 Major:

python3 "${CLAUDE_SKILL_DIR}/scripts/check_cohort_arithmetic.py" \
  --manuscript manuscript.md --data analysis/strata.csv --strict

Also confirm the reference (baseline) row of any stratified hazard/odds table is present and labelled; a missing reference category makes the other strata uninterpretable.

Phase 2.5c: Reference Hallucination Scan

Numerical audits (2.5/2.5a/2.5b) cover in-text numbers; they do not cover reference-list integrity. LLM-drafted or co-author-handed-in bibliographies frequently contain fabricated DOIs, wrong author/year combinations for a real DOI, or plausible-looking references that never existed. These slip past human proofreading because the surface form looks canonical.

When to run: every manuscript at self-review, regardless of stage. Mandatory before submission and before any revision circulation to co-authors or the editor.

Procedure:

1. Locate the bibliography. From SSOT.yaml → truth.refs_bib (fallback manuscript/_src/refs.bib for legacy projects). If SSOT.yaml is absent, scan references/library.bib as a last resort.

2. Invoke `/verify-refs` on the resolved bib. The skill writes qc/reference_audit.json with a per-entry verdict (VERIFIED / FABRICATED / UNVERIFIED) and a top-level submission_safe boolean.

   # equivalent CLI form (same result as invoking the skill).
   # verify_refs.py takes a positional input (the .bib path) and writes its audit
   # to <project-root>/qc/reference_audit.json (path derived from --project-root).
   BIB="$(python3 -c "import yaml; print(yaml.safe_load(open('SSOT.yaml'))['truth']['refs_bib'])")"
   python3 skills/verify-refs/scripts/verify_refs.py "$BIB" --project-root . --strict

When both reference QC and cross-reference QC are needed in one pass, prefer the master orchestration entry point in /manage-refs — it chains check_citation_keys.py → verify_refs.py --strict → render_pandoc.sh (optional) → check_xref.py --strict and writes qc/pre_submission_gate.json as the single submission-readiness artifact:

   bash "${MEDSCI_SKILLS_ROOT:-$HOME/workspace/medsci-skills}/skills/manage-refs/scripts/pre_submission_gate.sh" \
       --md manuscript/manuscript.md \
       --bib manuscript/_src/refs.bib \
       --docx submission/<journal>/manuscript.docx \
       --allow-separate-attachments  # see Phase 2.5d for when this is appropriate

3. Read `qc/reference_audit.json`. For each entry not marked VERIFIED, add a row to the reconciliation block below. FABRICATED entries are P0 Major Comments (block submission). UNVERIFIED entries are Minor Comments unless the manuscript is at a circulation/submission gate, in which case they escalate to Major. For each duplicate_findings[] entry (category duplicate_pmid / duplicate_doi), add a Major Comment row noting the duplicated ref_ids pair and recommend cite renumbering — duplicates block submission (P0 Major) regardless of per-record VERIFIED status.

4. Cross-check placeholder + pagination drift. Run, on every round:

   grep -nE '\[@NEW:|\[N\]|\[N–N\]|e0{3}.{0,5}e0{3}|in[ .]?press|\bTBD\b|forthcoming' manuscript/

Two failure classes:

• Citation-queue placeholders ([@NEW:topic], [N], [N–N]): a citation slot that was never resolved. Any remaining at self-review is a P0.
• Pagination placeholders (e000–e000, in press, TBD, forthcoming): /verify-refs (Phase 2.5c step 2) marks these UNVERIFIED with note = "pagination_placeholder" but cannot judge centrality from the .bib alone. Here, with the manuscript in hand, decide centrality: if the unresolved reference supports a method choice or a headline claim (grep the citekey/marker against the Abstract, the Statistical Analysis subsection, and the first Results paragraph), escalate it to a P0 Major rather than a generic Minor. A method-load-bearing citation that is still "in press / e000" at submission is a blocker. Include each in the reconciliation block.

5. Record results in a short reconciliation block and append to the Phase 3 report:

   | Citekey | Verdict | Source check | Status |
   |---|---|---|---|
   | Kim_2024_Validation | VERIFIED | DOI + PubMed match | ✓ |
   | Park_2023_Radiomics | FABRICATED | DOI resolves to unrelated paper | ✗ P0 |
   | Lee_2022_DeepLearning | UNVERIFIED | No DOI/PMID, title not found | △ Major before submission |
   | [@NEW:segmentation_review] | PLACEHOLDER | unresolved citation queue | ✗ P0 |

Short-circuit rule: if qc/reference_audit.json already exists with a bib-hash match within 60s (P9 cache TTL, pending), the scan MAY reuse it; otherwise re-run. Never consume a stale audit from a prior manuscript revision.

Do NOT fabricate replacement references if any entry fails. Fix-forward belongs to /search-lit and /lit-sync, not to this skill. Self-review only reports the failure and blocks submission.

Phase 2.5c-2: Reference Adequacy Scan

Phase 2.5c covers reference integrity — are the cited references real (fabricated / unverified / duplicate / placeholder)? It does not ask whether there are enough references, in the right sections, grounding every named method. That is reference adequacy, and it is the failure mode behind a draft with thirteen references where the Statistical Analysis subsection names a competing-risk model, multiple imputation, the E-value, and an eGFR equation with zero citations. Keep the two strictly separate: an integrity failure blocks because a citation is wrong; an adequacy failure flags because a citation is missing.

When to run: every manuscript at self-review, after the integrity scan. The two share the manuscript and the resolved bib path.

Procedure:

1. Run the deterministic checker. Resolve the article type from project.yaml (passed verbatim; the script's alias map handles repo paper-type names) and the journal cap from the target journal profile when known:

   python3 "${MEDSCI_SKILLS_ROOT:-$HOME/workspace/medsci-skills}/skills/self-review/scripts/check_reference_adequacy.py" \
     --manuscript manuscript/manuscript.md --bib "$BIB" \
     --article-type "$TYPE" ${CAP:+--journal-cap "$CAP"} \
     --out qc/reference_adequacy.json --strict

It reports the cited-reference count vs the article-type target, the section distribution (Introduction / Methods / Results / Discussion), every named method found in the Methods/Statistical-Analysis block, which of them lack a citation in their paragraph, and a methods_zero_citations flag.

2. Fold `findings[]` into the review. Each finding becomes a standard issues[] entry (so /revise and downstream consumers ingest adequacy and other comments uniformly), additively carrying the machine-readable issue_type + subtype alongside the usual fields, under category: "F" / category_name: "Reporting Completeness":

   {"id":"M2","severity":"major","category":"F","category_name":"Reporting Completeness",
    "issue_type":"reference_adequacy","subtype":"methods_named_method_uncited",
    "location":"Methods - Statistical Analysis",
    "description":"Fine-Gray competing-risk model is named without a canonical citation.",
    "fixable_by_ai":false,
    "suggested_fix":"Run /search-lit for the canonical Fine-Gray competing-risk source, sync via /lit-sync, then rerun /verify-refs --strict."}

Severity: methods_zero_citations (original / AI-validation / meta-analysis) and each uncited statistical method → Major (a P0 candidate before submission when the method is central to the primary or a sensitivity analysis); each uncited reporting/diagnostic standard → Minor; a total count below the article-type target → Major when far below (under half the floor), otherwise Minor, scaled also by stage (escalate at a submission/circulation gate).

3. Fix-forward, not fabricate. As in Phase 2.5c, this skill never writes replacement references. Every adequacy finding carries fixable_by_ai: false; the remedy is /search-lit (Manuscript Paper Reference Pool mode) → /lit-sync → /verify-refs --strict, which the author runs.

Phase 2.5d: Cross-Reference QC (Manuscript ↔ rendered DOCX)

Reference-list integrity (Phase 2.5c) does not cover Table/Figure cross-references. This is a separate failure mode where in-text citations ("Supplementary Table S4 reports a sensitivity analysis") resolve to a different caption in the rendered DOCX ("Supp Table S4 = a diagnostics table") because the build script carries its own legacy SSOT. Internal consistency (Phase 2.5) cannot detect it — both the prose and the build artifact echo their own divergent truths cleanly.

Precedent failure pattern (an STROBE cohort manuscript revision):

Body prose cited Supp Table S4 as a sensitivity analysis; the rendered DOCX

S4 instead contained a diagnostics table. S1, S6, S7 also mismatched. S8 and S9

were cited in the manuscript but absent from the rendered DOCX entirely.

Caught only on co-author circulation review.

When to run: every manuscript at self-review when a rendered DOCX exists (e.g., circulation drafts, post-build pre-submission checks). Skip only if no DOCX build has occurred yet (early drafts).

Procedure:

1. Locate inputs. manuscript/manuscript.md (or the SSOT truth.manuscript_md) and the rendered DOCX (typically manuscript/manuscript_final.docx or the most recent circulation .docx).

2. Invoke the shared script (lives in /manage-refs):

   python3 "${MEDSCI_SKILLS_ROOT:-$HOME/workspace/medsci-skills}/skills/manage-refs/scripts/check_xref.py" \
     --md manuscript/manuscript.md \
     --docx manuscript/manuscript_final.docx \
     --out qc/xref_audit.json \
     [--allow-separate-attachments]

The script writes qc/xref_audit.json with per-label rows tagged OK | MISSING_DOCX | MISSING_BODY | MISMATCH | UNCITED | NOT_CITED_NO_BODY, a top-level submission_safe boolean, and a policy.allow_separate_attachments field that records which severity policy applied.

3. Translate findings to anticipated comments. Severity mapping depends on the journal's figure/table submission policy. Many radiology and medical journals (e.g., European Radiology, Radiology, AJR) accept figures and tables as separate attachment files rather than inline in the manuscript DOCX; for those workflows pass --allow-separate-attachments so MISSING_DOCX is not treated as a P0 blocker. MISSING_BODY and MISMATCH remain P0 regardless, because they indicate SSOT drift between body markdown and rendered DOCX rather than a legitimate attachment style.

4. Append a reconciliation block to the Phase 3 report:

   | Label | Status | Body caption | DOCX caption | Verdict |
   |---|---|---|---|---|
   | Supplementary Table S4 | MISMATCH | Sensitivity analysis | Diagnostics table | ✗ P0 |
   | Supplementary Table S8 | MISSING_DOCX | (defined in body) | — | ✗ P0 |
   | Figure 2 | UNCITED | Forest plot of subgroups | Forest plot of subgroups | △ Minor |

5. Emit each P0 row as a separate `M`-numbered Major Comment with category: "F" (Reporting Completeness) and fixable_by_ai: false (build script changes are out of scope for the auto-fix loop — they require pipeline-side fixes per /write-paper Step 7.6a routing).

Do NOT auto-fix cross-reference defects in `--fix` mode. Caption rewrites in the body without re-running the DOCX build will simply move the mismatch. Surface as Major Comments and let the user route to /write-paper Step 7.6a.

Phase 2.5e: Confounding Completeness (observational only)

For an observational study, the highest-yield reviewer finding is also the most mechanical, and a prose pass misses it because the manuscript text is internally consistent: a covariate that was measured, is imbalanced across exposure groups in the baseline table, and is absent from the adjustment set is residual confounding by a measured variable. Only a join of the exposure- stratified Table 1 against the Methods adjustment set exposes it. This is probe O1 of references/domain-probes/observational_confounding.md, run here as a deterministic gate so the finding lands without the --panel cost.

When to run: manuscript type is observational (cohort, case-control, cross-sectional, health-screening registry) and the central claim is an adjusted exposure–outcome association. Skip for RCTs and descriptive studies.

Precedent failure pattern:

A cross-sectional screening-cohort manuscript reported an adjusted association

while Table 1 showed uric acid, smoking pack-years, HDL, total cholesterol, and

HbA1c all significantly imbalanced across the exposure groups — none of which

were in the age/sex/BMI/hypertension/diabetes adjustment set. The single-pass

review passed it; only an epidemiology panel reviewer who read the Table 1 CSV

against the Methods caught the gap. After refitting with extended adjustment the

primary estimate held, but the manuscript had claimed robustness it had not shown.

Procedure:

1. Locate the exposure-stratified baseline table as a CSV (e.g. table1_by_<exposure>.csv from /analyze-stats) and the Methods adjustment set (the variables after "adjusted for ...").

2. Run the deterministic gate:

   python3 "${CLAUDE_SKILL_DIR}/scripts/check_confounding_completeness.py" \
     --table1 table1_by_<exposure>.csv \
     --adjusted-list "age, sex, BMI, hypertension, diabetes" \
     --exposure-defining-list "body mass index, waist, fasting glucose, triglycerides, HDL cholesterol" \
     --out qc/confounding_completeness.json --strict

It emits a reconciliation table (covariate | imbalance p | SMD | in adjustment set? | verdict) and flags each measured-but-unadjusted imbalanced covariate as an UNADJUSTED_IMBALANCED Major candidate. The gate resolves DB column codes against a prose adjustment set (alias map), and when the Table 1 has no p / SMD column it computes the SMD from per-stratum "mean ± SD" group columns (--group-cols A,B, or auto-detected). When the CSV is unavailable, apply probe O1 by hand from the published Table 1.

Guideline-defined exposures (MASLD / metabolic syndrome / CKM / sarcopenia / frailty): pass --exposure-defining-list (the components of the exposure's own diagnostic criteria). Those rows are marked EXPOSURE_DEFINING_EXEMPT, not Major — adjusting for them is over-adjustment (probe O7), not a confounding fix. Without the exemption the gate false-positives a Major on every metabolic-criteria covariate. The residual-confounding remedy is an extended-adjustment model adding only non-defining prognostic covariates.

3. Each `UNADJUSTED_IMBALANCED` covariate is an Anticipated Major Comment (category: A. Study Design & Data Integrity), with the suggested fix: report an extended-adjustment sensitivity model that adds the omitted covariates and states whether the primary estimate is materially unchanged; the original model stays primary only if the extended model agrees.

4. Then apply the rest of the observational probe set (O2 adjustment-set provenance, O3 selection/collider bias, O4 exposure measurement validity, O5 missing-data mechanism & complete-case collapse, O6 residual-confounding E-value, O7 over-adjustment, O8 analysis unit, O9 outcome construct validity, O10 overlapping-subset gradient) from references/domain-probes/observational_confounding.md — these are prose probes (O1/O7/O8 are the data-checkable ones), and complement the generic Phase 2 categories rather than replacing them.

5. Extended-adjustment frame discipline. When the extended-adjustment model adds covariates that carry missingness, its analytic n shrinks. Comparing the adjusted estimate to the full-frame unadjusted estimate confounds adjustment with case-concentrated missingness ("adjustment inflated the estimate" when the drift is who-was-dropped). The fair anchor is the unadjusted estimate refit on the reduced complete-case frame; flag any "adjustment changed the estimate" claim that compares across different frames, and route the refit to /analyze-stats (requires_reanalysis).

Adjustment-set matching is fuzzy (a table row "Smoking, pack-years" vs an adjustment token "smoking"): read the reconciliation table rather than trusting the count, and confirm each flagged covariate is a plausible cause of the outcome (not a mediator or collider, which O2 covers) before raising it.

Phase 2.5f: Claim-vs-Artifact Cross-Check

Phases 2.5–2.5e check numbers and adjustment sets. This phase checks claims against the external artifacts they should trace to — the pre-registration, the protocol, the analysis outputs. These are the errors that survive a single-pass review because the manuscript prose is internally consistent yet disagrees with the registration or the analysis it reports. The first scope is the two highest- value, deterministic instances; figure/flow-count reconciliation, Methods-promised- analysis completeness, and imputation-input integrity are separate subchecks (run /make-figures legend reconciliation and /write-paper's Methods-promised gate).

Precedent failure pattern:

A manuscript reported a null primary association from a multiple-imputation model

and described it as "pre-specified," while the registered primary had been the

complete-case model that was significant — the primary had been re-designated after

the results were known. In the same paper an E-value of 2.79 was attached to the

primary HR of 1.34, but 2.79 does not recompute from 1.34 (it came from a different,

non-primary estimate), and a second E-value bounded an exploratory cancer-specific

hazard, not the headline contrast. None of these tripped the internal-consistency

checks; all three are deterministic against the registration and the arithmetic.

Procedure:

1. Run the cross-check with the manuscript and (if available) the pre-registration / protocol / project.yaml:

   python3 "${CLAUDE_SKILL_DIR}/scripts/check_claim_artifact.py" \
     --manuscript manuscript.md --prereg prereg.md \
     --out qc/claim_artifact.json --strict

2. Estimand provenance. The script flags PRIMARY_REASSIGNED when the manuscript admits re-designating the primary after results were known, and ESTIMAND_DRIFT when the manuscript's primary statement does not match the registered one. Both are Anticipated Major Comments (category: A. Study Design & Data Integrity); a primary re-designated post-hoc is a P0 issue. The fix is to report the pre-specified and the revised models coequally and disclose the change in the Abstract and Limitations, not to silently lead with the more favourable estimate.

3. E-value. EVALUE_ARITHMETIC means the reported E-value does not recompute from its adjacent effect estimate (the value was likely produced for a different estimate); EVALUE_NON_PRIMARY means the E-value is attached to a secondary/exploratory estimate but presented as if it bounded the headline claim. Both warrant a Major/Minor comment — recompute the E-value for the declared primary estimate and its near-null confidence limit, and quote it there.

4. Primary-change guard. Independently of the script, if the manuscript reports two models for the same contrast where one is significant and the other null and the significant one is foregrounded, confirm which was pre-specified; an outcome-dependent choice of primary model is a Major comment even when each model is individually correct.

5. Headline vs own-sensitivity direction. Read the sensitivity series (S1 etc.) the manuscript itself reports. If the headline causal/association claim points the opposite way from the authors' own adjusted or sensitivity estimate — a positive lead sentence over a sensitivity model that attenuates to the null, or vice versa — that is a Major: the paper is contradicting its own robustness check. This is a prose judgement, not a script verdict.

6. Methods ↔ Results ↔ disk coverage. Run the deterministic coverage gate:

   python3 "${CLAUDE_SKILL_DIR}/scripts/check_artifact_coverage.py" \
     --manuscript manuscript.md --analysis-dir output/analysis \
     --out qc/artifact_coverage.json --strict

PROMISED_ABSENT (an analysis named in Methods that never reaches Results) and DISK_UNREPORTED (an analysis output on disk — an added-value DeLong CSV, a calibration table — never mentioned in the manuscript) are Anticipated Major Comments. The reverse direction matters because a run-but-unreported result can be the one that undercuts the headline. When an _analysis_outputs.md manifest exists the gate uses it as the source of truth; otherwise it globs --analysis-dir and only escalates analysis-bearing file names.

The script is deterministic but its provenance match is fuzzy (token overlap): read the reconciliation in qc/claim_artifact.json and confirm against the actual registration before raising ESTIMAND_DRIFT. For time-to-event manuscripts, also apply probe S8 (estimand provenance) of references/domain-probes/survival_prognostic.md.

Phase 2.6: Multi-Agent Panel Review (--panel, opt-in)

Run this phase only when `--panel` is passed. The default single-pass review (Phases 2–2.5d) stays the fast path; the panel is the high-cost, high-precision option for a pre-submission final pass on a top-tier target. Run it after the numerical audits (Phases 2.5–2.5d) so the reviewers see source-verified numbers, and before the Phase 3 report, which it feeds.

Precondition (blocking): the SSOT must be singular. Before spawning any reviewer, enforce the Phase 1 step 4 SSOT gate: if more than one manuscript-like .md exists and none is pinned (no SSOT.yaml truth.manuscript_md, no explicit --ssot), halt and ask the user which file is the SSOT — a panel is too expensive to spend on a stale copy. Clear any STALE_COPY from detect_copy_divergence.py first.

The panel simulates independent peer reviewers who do not see each other's comments, then an editor who consolidates them — the same structure a journal uses. It reuses the vendored domain-probe modules so every reviewer applies the same criteria.

Step 1 — Compose the reviewer set by research type. Auto-detect the manuscript type (Phase 1 input + the Research-Type Adaptation table). Each reviewer loads the matching domain-probe module so the panel's criteria are single-sourced.

If the type is ambiguous, ask the user before composing the set.

Step 2 — Run the reviewers (portable execution). When the host provides a parallel subagent / Task capability (Claude Code, or any harness exposing an Agent tool), spawn the reviewer set as independent parallel subagents, each blinded to the others, then run the editor as a final synthesis agent. Fallback (no subagent capability — e.g. a minimal Codex/Cursor harness): a single agent role-plays each reviewer sequentially and in isolation — it completes and writes out reviewer R1's full structured review before reading the manuscript "fresh" as R2, so a later reviewer never sees an earlier reviewer's comments. The panel is defined by these instructions; it does not depend on the Workflow tool or any Claude-Code-only orchestration.

A reusable reviewer schema, a generic harsh-but-fair reviewer prompt skeleton with per-domain focus checklists, and the editor synthesis prompt skeleton live in ${CLAUDE_SKILL_DIR}/references/panel_review_template.md.

Each reviewer returns: reviewer_id, expertise_area, an overall_assessment (name the single biggest threat to the conclusions), strengths (2–3), major[] (each with heading, comment, location, severity, suggested_fix), and minor[]. Map severity onto this skill's own scale — a conclusion-threatening / design-level finding is Fatal, a reporting-level finding is Fixable — rather than introducing a separate vocabulary.

Step 3 — Editor synthesis. One editor pass (a final agent, or the main agent in the fallback) consolidates the reviews: 1. Dedupe findings by theme across reviewers. 2. Flag CONSENSUS for any theme raised by ≥2 reviewers, with R1/R2/R3 attribution (e.g., [CONSENSUS: R1+R3]); single-reviewer findings are attributed to the one reviewer. 3. Decide an internal readiness verdict (this sets the Phase 3c verdict / overall_score; it is not printed as a journal recommendation). 4. Rank the concrete pre-submission actions the author should complete first. 5. State a one-line readiness verdict (ready for the target tier now / fix specific items first / consider a different tier).

Step 3.5 — Lens-diversity gate (deterministic). A panel only earns its cost if its reviewers span distinct axes rather than echo one theme louder. Before the editor finalizes, serialize the reviewers' structured outputs (the schema above) to a JSON file — either a top-level list or {"reviewers": [...], "research_type": "..."} — and run the gate:

python3 ${CLAUDE_SKILL_DIR}/scripts/check_panel_diversity.py \
    --panel panel_reviews.json \
    --research-type {survival|sr_ma|radiomics|dta|observational|narrative} --strict

It reports three diversity failures, each mapped onto a concern family aligned to the focus checklists:

• `UNCOVERED_AXIS` (Major) — an axis the research type is expected to probe (e.g. heterogeneity/pooling for an SR/MA) drew zero major findings. The editor re-probes it with the owning reviewer before finalizing, or records in the synthesis why the gap is acceptable.
• `FAMILY_MONOCULTURE` (Major) — the majority of majors fall in one concern family; the lenses converged rather than spanned the manuscript.
• `LENS_COLLAPSE` (Flag) — a reviewer raised only families another reviewer already covered, adding no independent axis.

Healthy CONSENSUS is preserved — agreement on some themes is a strength (Step 3 flags it), and the gate fires LENS_COLLAPSE only on a fully redundant reviewer and the Major checks on panel-level coverage, never on agreement per se. Do not silently ship a monoculture: resolve every Major before the synthesis verdict.

Step 4 — Feed Phase 3. The consolidated panel output flows into the Phase 3 report, Phase 3b R0 numbering (preserved, so /revise still consumes it), and Phase 3c JSON. CONSENSUS flags and reviewer attribution are additive annotations on the existing M/m comments (and the optional consensus JSON field); they do not change the report or JSON structure.

Re-run the panel after a large revision. A panel is high-yield not only before the first submission but again after any large edit — a word-count compression, a primary-model or adjustment-set change, or resolving a batch of majors. Such edits introduce new drift (a compression drops a caveat; a re-fit leaves a derived CSV stale; a relocation orphans a cross-reference), and the second panel's findings shift character accordingly (method → compression-drift → residual). If the author has just compressed or re-modelled, recommend one more --panel pass rather than assuming the prior panel still holds; in practice each post-revision round surfaces real, distinct errors.

Phase 3: Report

Before writing the Anticipated Comments, skim references/exemplar_findings/ for the finding at hand (cohort-arithmetic mismatch, unadjusted confounder, cross-sectional scope overreach, post-hoc primary / estimand drift). Each models the full shape — which gate fired, the comment in the reviewer's own words, Fatal/Fixable severity, the closest category letter, the concrete fix, fixable_by_ai, and an R0-ready line for Phase 3b. They are synthetic teaching models — match the structure, not the wording.

Generate a concise report with this structure:

# Self-Review Report: {manuscript title}

**Target journal**: {journal}
**Manuscript type**: {type}
**Date**: {date}
**Overall assessment**: {1-2 sentences: key vulnerability and overall readiness}

## Anticipated Major Comments (fix before submission)

M1. **{Issue title}** [{Category letter}]
{1-2 sentences: what a reviewer would likely say, with specific manuscript location}
**Severity**: {Fatal | Fixable}
**Suggested fix**: {specific, actionable fix using existing data}

M2. ...

## Anticipated Minor Comments (address proactively)

m1. **{Issue}** [{Category}]: {1 sentence with location + fix}
m2. ...

## Strengths (emphasize in cover letter)

- {Specific strength 1}
- {Specific strength 2}
- ...

Conciseness targets:

• Anticipated Major Comments: 3-7 items, each 3-5 lines
• Anticipated Minor Comments: 3-6 items, each 1-2 sentences
• Strengths: 3-5 items, each 1 sentence
• Total report: 400-800 words (excluding optional R0 section)

Phase 3b: R0 Numbering (Optional)

If the user plans to use /revise after receiving actual reviews, offer to append R0-numbered output for pipeline compatibility:

## R0 Pre-Submission Findings (for /revise cross-reference)

R0-1 [MAJ] {mapped from M1}: {issue title}
R0-2 [MAJ] {mapped from M2}: {issue title}
R0-3 [MIN] {mapped from m1}: {issue title}
...

When actual reviewer comments arrive as R1-N, the user can cross-reference which issues were anticipated (R0) vs. novel (R1-only).

Phase 3c: Structured JSON Output

When --json is passed, or when invoked by /write-paper Phase 7, append a machine-readable JSON block after the markdown report. Fence it with triple backticks and the json language tag so downstream parsers can extract it.

{
  "self_review_version": "1.0",
  "manuscript_title": "...",
  "date": "YYYY-MM-DD",
  "overall_score": 72,
  "verdict": "REVISE",
  "fatal_count": 0,
  "major_count": 3,
  "minor_count": 4,
  "issues": [
    {
      "id": "M1",
      "severity": "major",
      "category": "C",
      "category_name": "Validation & Stats",
      "location": "Methods, paragraph 5",
      "description": "Calibration plot and Brier score absent for prediction model",
      "fixable_by_ai": true,
      "suggested_fix": "Add calibration analysis paragraph after discrimination results. Generate calibration plot via /make-figures."
    },
    {
      "id": "m1",
      "severity": "minor",
      "category": "F",
      "category_name": "Reporting Completeness",
      "location": "Abstract, line 3",
      "description": "Abstract reports AUC 0.91 but Table 2 shows 0.912 -- rounding inconsistency",
      "fixable_by_ai": true,
      "suggested_fix": "Change abstract to match table: AUC 0.91 (95% CI: 0.87-0.95)"
    }
  ]
}

Field definitions:

• overall_score: Integer 0-100 reflecting manuscript submission readiness
• verdict: "PASS" (score >= 85, no fatal issues) or "REVISE"
• severity: "fatal", "major", or "minor"
• category: Letter code from the 10-category system (A-J)
• fixable_by_ai: true if the issue can be resolved by editing manuscript text with existing data; false if it requires new data, analyses, or human judgment (e.g., design changes, IRB decisions, missing experiments)
• requires_reanalysis (optional, default `false`): true when closing the finding needs a committed analysis re-run against the real data, not a prose edit — power/MDE re-simulation under the full model, first-visit/one-record-per-subject dedup, an extended- or reduced-adjustment sensitivity model, optimism correction of calibration. Always implies fixable_by_ai: false. Additive and backwards-compatible; parsers that do not expect it must ignore it. Route these to /analyze-stats (see Phase 4).
• suggested_fix: Specific, actionable instruction. If fixable_by_ai is true, this must be concrete enough for the fixer to execute without ambiguity.
• consensus (optional, panel mode only): array of reviewer ids that raised the issue, e.g. ["R1","R3"]. Additive and backwards-compatible — present only when Phase 2.6 ran; parsers that do not expect it must ignore it.

Phase 4: Fix Support

Standard mode (no --fix flag)

After presenting the report, offer to help fix specific issues:

• Rewrite overclaiming sentences
• Draft missing limitation statements
• Suggest statistical additions (e.g., calibration analysis code via /analyze-stats)
• Draft intended use, decision-impact, or novelty-delta statements
• Check specific tables/figures for consistency
• Generate missing flow diagrams via /make-figures

`requires_reanalysis` findings route to `/analyze-stats`, not a prose edit (observational/cohort). For cohort and observational manuscripts, the highest-value fixes are usually data-level: a power/MDE re-simulation under the full primary model, a first-visit / one-record-per-subject dedup sensitivity, an extended- or reduced-adjustment (over-adjustment) sensitivity model, or optimism correction of calibration. These are not fixable_by_ai text edits — --fix is text-only and will silently skip them. Tag each such finding requires_reanalysis: true and route it to /analyze-stats for a committed script + CSV, then feed the regenerated numbers back into the manuscript and re-run the relevant Phase 2.5 gate. Surface these explicitly to the author rather than letting an auto-fix pass appear to "resolve" them.

Auto-fix mode (--fix flag)

When --fix is passed:

1. Filter fixable issues: Select all issues where fixable_by_ai is true. 2. Apply fixes sequentially: For each fixable issue, edit the manuscript file directly:

• Text rewrites (overclaiming, missing sentences, terminology) → Edit in place
• Missing reporting items (ethics statement, data availability) → Insert at suggested location
• Numerical inconsistencies (abstract-table mismatch) → Correct to match tables
• Do NOT attempt: new statistical analyses, new figures, design changes, IRB-dependent items, or any issue tagged requires_reanalysis (route those to /analyze-stats)
• Do NOT invoke other skills (/make-figures, /analyze-stats) during fix — text edits only

3. Report changes: After all fixes, output a summary:

   ## Auto-Fix Summary
   - Fixed: {N} issues
   - Skipped (requires human): {M} issues
   - Changes: {list of id + one-line description of what was changed}

4. Post-edit paren-span safety scan: if any fix reduced em-dashes (e.g. a — X — appositive → (X)), run the parenthesis-span gate before re-review — a bulk conversion can pair two unrelated dashes across a sentence boundary and wrap a whole sentence (or an ordinal "Sixth, …" limitation) inside one parenthesis (paren-balanced, so a balance check misses it):

   python3 "${CLAUDE_SKILL_DIR}/scripts/check_paren_spans.py" \
     --manuscript manuscript.md --out qc/paren_spans.json --strict

PAREN_SPAN_ORDINAL / PAREN_SPAN_SENTENCE is a Major — undo or repair that conversion before continuing. 5. Re-review: Run Phase 2 (systematic check) again on the modified manuscript. 6. Iterate: If new fixable issues emerge, apply one more round (maximum 2 total fix iterations). 7. Final output: Regenerate the Phase 3 report and Phase 3c JSON with updated scores.

Iteration limit: Maximum 2 fix-and-re-review cycles. If the score has not reached "PASS" after 2 iterations, output the final report with remaining issues and flag: "Auto-fix limit reached. Remaining issues require human review."

What This Skill Does NOT Do

• Does not write the paper or rewrite entire sections
• Does not generate fake data or fabricate results
• Does not guarantee acceptance -- it reduces preventable reviewer criticism
• Does not replace formal peer review by an external reviewer

Tone

Be direct and practical. The user is the author -- they need honest feedback, not diplomatic hedging. Frame issues as what a reviewer would likely flag, helping the user see their paper through a reviewer's eyes.

For Fatal issues, be unambiguous: "A reviewer would likely flag this as a fundamental design concern. Submitting without addressing this risks Reject."

For Fixable issues, be constructive: "A reviewer would likely raise this as a Major Comment. Here is how to address it with your existing data."

Anti-Hallucination

• Never fabricate references. All citations must be verified via /search-lit with confirmed DOI or PMID. Mark unverified references as [UNVERIFIED - NEEDS MANUAL CHECK]. Self-review enforces this through Phase 2.5c: Reference Hallucination Scan (runs /verify-refs against the SSOT bib); any FABRICATED verdict blocks submission as a P0 Major Comment.
• Never invent clinical definitions, diagnostic criteria, or guideline recommendations. If uncertain, flag with [VERIFY] and ask the user.
• Never fabricate numerical results — compliance percentages, scores, effect sizes, or sample sizes must come from actual data or analysis output.
• If a reporting guideline item, journal policy, or clinical standard is uncertain, state the uncertainty rather than guessing.

---

Gates

<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review. Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal recommendation. Each consuming skill maps findings to its own output:

• peer-review: Major / Minor comments + Confidential Comments to the Editor; a task- or

design-level flaw is placed as Major #1.

• self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.

Do NOT edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync. -->

AI / ML overclaiming probes (AO0–AO5)

A 5-probe checklist (AO1–AO5, with AO0 as a gate) for medical-AI/ML primary studies (diagnostic, prognostic, triage, detection) where the conclusion's reach exceeds the evidence. These probes complement (do not replace) the generic Phase 2 issue checklist and the signature "Overclaiming vs evidence level" check. The aim is to keep a framing-level over-reach from passing as a wording nitpick: a paper can report sound metrics yet draw a clinical claim — generalizable, outperforms clinicians, deployment-ready — that the design does not support, and that claim is what a reader carries away. AO1–AO4 target over-reach in the claim sentences; AO5 targets over-reach baked into the reported metric itself (an optimistically- or unreproducibly-reported number that makes the result look stronger than a faithful estimate). Run AO0 first.

AO0 — Locate the strongest claim, then its support (run before AO1; gates any over-reach finding):

• Identify the load-bearing claims in the Title, Abstract, and Conclusion (the sentences a reader quotes). For each, find the specific evidence cited (which dataset, which comparison, which metric + uncertainty).
• An over-reach finding is a lead until the claim and its support are read together against the manuscript — do not strawman a stray adjective. Escalate only when a headline claim genuinely outruns the cited evidence.
• If the claim is already appropriately hedged to the evidence, record "claim matched to evidence" and move on.

AO1 — Generalizability claimed from limited external validation:

• Does the Abstract/Conclusion assert the model "generalizes," is "transferable/robust across settings," or is suitable for broad populations, while external validation is a single site / single scanner-vendor / single source (or absent)?
• Sub-check: is the external set demographically narrow (single ethnicity, single sex-dominant, narrow age) relative to the population the claim names?
• If the generalizability claim outruns the external evidence → recommend softening to the evidence ("validated at one external site") and moving multi-setting generalizability to a stated limitation + next step. MAJOR candidate when it is a headline claim; MINOR when it is a single qualifier in the Discussion.

AO2 — Superiority language against overlapping or under-powered comparison:

• Flag "outperforms", "superior to", "beats", "can replace [clinician/radiologist]" when (a) the model vs comparator 95% CIs overlap, (b) no test of the difference is reported (two separate AUCs are not a comparison), or (c) the comparison rests on a small test set / few readers.
• Ask for the difference in the metric with its CI and a paired test of that difference, not two standalone estimates.
• If the difference is not statistically supported → recommend reframing from "outperforms" to "comparable to" (still a meaningful result). MAJOR when a superiority/replacement claim is the headline; otherwise MINOR.

AO3 — Comparison-frame mismatch (model task ≠ human task):

• When a model-vs-clinician comparison drives a claim, verify the two performed the same task on the same inputs under the same constraints: same images/inputs available, same time budget, same question asked, same decision point.
• Common mismatches: the model sees a curated single view while readers see the full study; readers are timed or work from a different modality; the "reader" benchmark is a literature value on a different cohort.
• A mismatch makes "outperforms clinicians" non-interpretable as a clinical claim → ask the authors to state exactly which task the comparison establishes, or to align the conditions. MAJOR candidate when it underpins a headline.

AO4 — Deployment / clinical-readiness claim from retrospective internal evidence:

• Flag "ready for clinical deployment", "can be used to triage/guide treatment", "will reduce workload/cost", or a recommended decision threshold, when the evidence is a retrospective, internally-split (or even external but observational) accuracy study with no prospective, silent-trial, or decision-impact data and (often) no calibration or decision-curve analysis.
• Discrimination on retrospective data does not establish that acting on the model helps patients; a probability that drives a decision must also be calibrated, and net benefit must be shown.
• Recommend reframing deployment/utility language to "supports further prospective evaluation", and (where a threshold is proposed) adding calibration + decision-curve evidence. MAJOR when a deployment/care-directive claim is made; MINOR when only a hedged "potential utility" sentence.

AO5 — Optimistic / non-reproducible performance reporting (the metric overstates the evidence):

• Triggered when a primary AI/ML study reports discrimination or accuracy as a headline result. Check how the number was produced before taking it at face value.
• (a) Best-fold vs cross-fold: is the headline metric the single best cross-validation fold (or one favorable train/test split) rather than the mean across folds with its SD or a 95% CI? Best-fold figures are upward-biased; ask for the cross-fold mean ± SD (or CI).
• (b) Operating point: are sensitivity/specificity/F1 reported without stating the decision threshold and how it was chosen? A threshold tuned on the test set inflates the metric; ask for the operating point and that it was selected on training folds only (e.g., Youden's J on the training data). This probe is about whether the reported number is reproducible/optimistic, not whether the threshold is clinically deployable (that is AO4).
• (c) Prevalence-representative metrics: was training/evaluation done on an artificially balanced set while the real prevalence is much lower, with accuracy quoted as the headline? Accuracy — and PPV/NPV — on a re-balanced set do not transfer to the deployment base rate (PPV/NPV are prevalence-dependent). Ask for the evaluation class distribution, threshold-independent discrimination (AUROC, and AUPRC under imbalance, with CIs), sensitivity / false-negative rate at the stated threshold, and PPV / NPV estimated on a prevalence-representative holdout (or modelled for the target clinical prevalence, with uncertainty).
• (d) Code-vs-claims fidelity: where code is released, does the described tuning/metric match it? Common mismatches: a claimed hyperparameter search the code does not run; a metric (e.g., specificity) attributed to a library function that does not compute it. A confirmed mismatch is an integrity/reproducibility flag — verify against the released code before asserting it.
• Severity: MAJOR when the load-bearing performance claim rests on a best-fold number, an unstated/test-tuned threshold, a rebalanced-accuracy headline, or a code-vs-claims mismatch (the reported result is optimistic or not reproducible); MINOR when cross-validation was sound and only the cross-fold summary, the operating point, or a class-aware metric is missing from the write-up.

Decision-impact / early-deployment probes (DECIDE-AI axis, DI1–DI5)

Co-apply when a study claims clinical utility, deployment, or decision impact of an AI system, or is an early-stage live clinical evaluation. The reporting axis is then DECIDE-AI (early-stage clinical evaluation of AI decision-support); these probes check that a utility/deployment claim rests on real-use evidence, not retrospective accuracy. They sharpen AO4 for the deployment-evaluation case.

DI1 — Live/prospective evidence vs retrospective accuracy:

• Is there any prospective, in-workflow (silent-trial / shadow-deployment / live) evidence of how the system performs and is used, or is the deployment/utility claim built only on a retrospective, internally-split accuracy study? A deployment claim from retrospective discrimination alone → MAJOR (reframe to "supports prospective evaluation").

DI2 — Intended use + deployment pathway within early-stage limits:

• Is the intended use (condition, decision supported, users, setting) stated, and is the claim kept within what an early-stage evaluation can show? Over-reach to "ready for routine clinical use" from a developmental/exploratory study → MAJOR.

DI3 — Decision threshold + calibration/utility:

• If a decision threshold or care directive is proposed, is it justified, with the predicted probabilities calibrated and net benefit (decision-curve) shown — not discrimination alone? An unjustified threshold or missing calibration/utility for a probability that drives a decision → MAJOR.

DI4 — Workflow integration + human–computer interaction:

• Does the study report how the system was used in the real workflow — user adherence/exposure, human–computer agreement/override rates, usability and learning curve (human factors)? A utility claim with no real-use interaction data is the central DECIDE-AI gap → MAJOR/PARTIAL.

DI5 — Safety, error capture, and subgroup safety:

• Were significant errors/malfunctions pre-defined and captured, the safety profile reported and discussed, and performance/safety examined across relevant subgroups (fairness)? Absent pre-defined error capture or subgroup-safety assessment where the claim is clinical deployment → MAJOR.

Output template (AO1 example):

"The Conclusion states the model 'generalizes across institutions,' but external validation appears limited to a single site ([Methods, External validation]). I'd suggest softening this to the evidence — e.g., 'validated at one external site' — and framing multi-institution generalizability as a stated limitation and a next step. If a broader claim is intended, an external set spanning multiple sites/vendors would be needed to support it."

Output template (AO2 / AO3 example):

"The 'outperforms radiologists' claim rests on a comparison whose 95% CIs for model and reader [metric] overlap ([Figure/Table]), and no test of the difference is reported; the reader task also differs from the model's in [inputs/time] ([Methods/Table]). I'd suggest (a) reporting the difference in [metric] with its CI and a paired test rather than two separate estimates, and (b) stating explicitly which clinical task the comparison establishes. If the difference is not statistically supported, reframing from 'outperforms' to 'comparable to' would be both defensible and still a meaningful result."

Output template (AO5 example):

"Table 2 appears to report the single best cross-validation fold; because best-fold figures are optimistically biased, I'd suggest reporting the mean across folds with its SD (or a 95% CI). Relatedly, the sensitivity/specificity depend on a classification threshold I could not find stated — reporting the operating point and how it was chosen (e.g., Youden's J on the training folds only, to avoid tuning on the test data) would make these numbers reproducible. Finally, since the training set was balanced to 1:1 while the clinical prevalence is much lower, accuracy (and PPV/NPV) on a re-balanced set may not reflect performance at the true base rate; stating the evaluation class distribution and adding AUROC/AUPRC with CIs alongside sensitivity/FNR at the stated threshold, with PPV/NPV on a prevalence-representative holdout, would make the clinical cost of a miss visible."

Discipline — leads vs findings (applies to AO0–AO5):

• A claim-vs-evidence mismatch surfaced by a quick scan is a lead, not a finding, until the claim sentence and its cited support are read together against the manuscript. Do not escalate a hedged Discussion qualifier as if it were a headline.
• Anchor every over-reach comment to the exact claim location and the exact evidence (dataset, comparison, metric + CI). A comment that names the location and the gap is actionable; "the authors overclaim" is not.
• Keep severity tied to where the claim sits and what it drives: a headline/clinical-action claim that outruns the design is design-/framing-level (MAJOR, often Major #1); a stray adjective is MINOR.

<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review. Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal recommendation. Each consuming skill maps findings to its own output:

• peer-review: Major / Minor comments + Confidential Comments to the Editor; a design-level

flaw (missing consent, identifiable patient data, or causal overclaim) is placed as Major #1.

• self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.

Do NOT edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync. -->

Case-report probes (CR1–CR9)

A checklist for case reports and small single-patient clinical narratives. These probes complement (do not replace) the generic Phase 2 issue checklist and the CARE items. They target the places reviewers most often challenge case reports: why the case is publishable, whether the patient is protected, whether the story is chronologically complete, and whether the Discussion stays inside the evidence level of n=1.

CR1 — Novelty / teaching-value justification:

• Does the manuscript state why this case is worth publishing: rare presentation, diagnostic trap,

management complication, unusual imaging/pathology correlation, unexpected response, or a practical bedside lesson?

• Is the claim anchored to a transparent literature boundary or epidemiologic context? "Rare" without

a search boundary, comparator cases, or a clinically actionable lesson is a MAJOR priority concern, because the paper's contribution is the case's teaching value.

CR2 — Consent, patient protection, and image de-identification:

• Is written informed consent documented, or is a waiver/IRB basis stated when appropriate? Are

patient images, dates, locations, institution names, initials, faces, scan metadata, and unusual demographic details removed or justified?

• Missing consent language or identifiable images/data is a MAJOR ethical/reporting issue. If the

manuscript includes potentially identifying details, treat it as a finding even when the prose says "de-identified."

CR3 — Causal attribution discipline:

• Does the Discussion distinguish temporal association from causation? A single case can suggest a

hypothesis or illustrate a plausible adverse event; it cannot establish incidence, efficacy, safety, risk, or mechanism.

• Escalate when Title/Abstract/Conclusion claims that an intervention "caused", "prevented",

"proved", "was effective", or "should change practice" without stronger evidence. The fix is to reframe as "was temporally associated with", "is consistent with", "highlights", or "should prompt consideration."

CR4 — Similar-case comparison and literature absence handling:

• Does the manuscript compare the case with the nearest prior cases, including similarities and

differences in presentation, diagnosis, intervention, and outcome? If five or more similar cases are found, a compact comparison table is usually clearer than prose.

• "First case", "only case", or "no prior reports" requires a documented search boundary. Without

it, ask the authors to soften the claim or state the search strategy; do not let an unverified priority claim carry the Abstract.

CR5 — CARE narrative completeness:

• Can the reader reconstruct Patient Information -> Clinical Findings -> Timeline -> Diagnostic

Assessment -> Therapeutic Intervention -> Follow-up and Outcomes? Is the timeline figure/table present when the course is multi-step, and does it include the final follow-up interval?

• Missing timeline, follow-up duration, diagnostic reasoning, or outcome assessment is usually a

Fixable Major for a case report because the narrative is the evidence.

CR6 — Generalizability and teaching-point framing:

• Does the conclusion state a narrow learning point rather than a population-level recommendation?

Strong case reports teach clinicians what to consider, monitor, or report; they do not infer prevalence, comparative effectiveness, or standard-of-care changes.

• Check the Abstract and final paragraph together. A cautious Discussion cannot rescue an

overgeneralized Abstract conclusion.

CR7 — Adverse drug/device/contrast reaction: causality discipline (apply when the case is the adverse event):

• Is attribution supported by a named instrument (Naranjo or WHO-UMC for drugs) with the score and

tier reported, not just the word "caused"? Is dechallenge documented (withdrawal → resolution) with the exposure-to-onset latency, and is rechallenge either reported or explicitly withheld on safety grounds? Mechanism-only narrative with no instrument and no dechallenge → MAJOR for the attribution claim.

• Are alternative causes excluded, and is the event located against a denominator (institutional

rate or pharmacovigilance database count) rather than implying incidence from n=1? Severity/ preventability instruments (e.g., Hartwig–Siegel, Schumock–Thornton) strengthen a safety report. Blaming the agent for a course actually driven by a downstream confounder (e.g., patient self-management) without separating the steps → overreach.

CR8 — Case-series-specific design (apply only when the manuscript reports a series, n≥2):

• Does the series have cohort-style methods (design, setting, case-identification source,

eligibility, protocol) and an all-cases summary table, rather than being N stacked single reports? Absent methods/summary table for a series → MAJOR (it cannot be read as a series).

• Is selection/ascertainment stated, with the screened pool size so consecutiveness can be

judged? Does the manuscript avoid inferring prevalence/proportion or effectiveness from a selected/referral series (report counts, not rates)? A rate claimed from a referral series, or a series presented with no selection description, → MAJOR interpretive limit. Cohort-level limitations (small n, retrospective, non-uniform protocol, no comparator) should be explicit.

CR9 — Imaging-led (radiology / nuclear medicine / interventional) case report (apply when the contribution is the image or an image-guided procedure):

• Is each modality described as technique → findings → impression, in clinical order, with

reproducible technique (sequence/phase, field strength, contrast agent/dose/rate, CT kV/keV/CTDIvol, PET tracer dose/uptake time, transducer frequency; for IR, devices with sizes and the step sequence)? Modalities merged into one undifferentiated paragraph, or findings stated without the acquisition needed to reproduce them, → reporting gap.

• When a structured-reporting system applies (BI-RADS/LI-RADS/PI-RADS/TI-RADS/Lung-RADS/O-RADS),

is the category given with its meaning/risk rather than a bare number? Are quantitative values reported with method (ROI placement) and threshold honesty — a value with no validated cutoff labeled exploratory, not diagnostic?

• For multimodality discordance, is the disagreement stated and resolved (decisive modality or

histopathology), and is a missing standard modality named as a limitation? For an IR case, is complication latency and the diagnostic→therapeutic pathway documented with pre/post outcome?

• Patient-protection at the image level: are images de-identified at the DICOM level (no burned-in

annotations, accession numbers, dates, faces)? Is figure alt text real (not a placeholder)? Is a device/vendor relationship disclosed for an advanced-technique or device case? Identifiable images or undisclosed device-vendor COI → MAJOR.

Output template (CR2 / CR3 example):

"The case is clinically interesting, but the submission does not yet provide enough information for

publication as a case report. First, the consent/anonymization statement should be made explicit,

including whether the accompanying images have been stripped of identifiers. Second, the Discussion

currently treats the temporal improvement after treatment as evidence that the intervention was

effective. Because this is a single case, I would suggest reframing the conclusion as a hypothesis-

generating observation and narrowing the teaching point to when clinicians should consider this

diagnosis or management option."

Discipline — leads vs findings (applies to CR1–CR9):

• A missing element is a **lead until the whole manuscript, figures, legends, and ethics statements are

checked**. Do not allege absent consent or absent follow-up if it appears in a title page, figure legend, supplement, or patient-perspective section.

• Anchor every comment to what the flaw changes: patient protection (CR2), evidentiary overreach

(CR3/CR6), contribution/priority (CR1/CR4), or narrative completeness (CR5). Avoid asking for broad extra literature unless it directly supports the case's teaching value.

<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review. Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal recommendation. Each consuming skill maps findings to its own output:

• peer-review: Major / Minor comments + Confidential Comments to the Editor; a task- or

design-level flaw is placed as Major #1.

• self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.

Do NOT edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync. -->

Clinical Prediction-Model probes (CP1–CP4)

A 4-probe checklist for cross-sectional / observational clinical prediction models — a binary or multinomial outcome predicted from a covariate set, reported under TRIPOD / TRIPOD+AI, usually as a nested predictor-set comparison ("does adding marker X to a base model improve prediction of Y?"). This is the diagnostic/prognostic prediction counterpart to survival_prognostic.md (time-to-event) and complements the observational_confounding.md probes (a prediction model still has an analysis unit (O8) and can be over-adjusted (O7)). Route here when the manuscript develops or compares prediction models and reports discrimination (AUC/C-statistic), calibration, or decision-curve / net-benefit analysis.

CP1 — Apparent vs optimism-corrected performance:

• Discrimination, calibration (slope / intercept / calibration plot), and decision-curve / net-benefit all suffer optimism when computed in the same sample used to fit the model. A manuscript that optimism-corrects only the C-statistic (bootstrap / cross-validation) but reports calibration and net benefit in-sample has corrected one axis and left two apparent.
• An apparent calibration slope near unity (≈ 1.0) does not establish out-of-sample calibration — it is the expected in-sample result and says little about transportability.
• Calibration and DCA computed in-sample must be labelled "apparent", or the optimism correction must be extended to them (e.g. bootstrap the calibration slope/intercept and the net-benefit curves). In-sample calibration/DCA described as "well calibrated" / "clinically useful" without the apparent caveat → MAJOR.

CP2 — The two-null distinction (incremental value vs marginal effect):

• A "negative" prediction study conflates two different nulls that have different evidentiary status:
• Incremental value negligible — a well-powered statement: ΔAUC ≈ 0, optimism correction does not favour the larger model (and may reverse sign toward the parsimonious one), net benefit does not exceed the base model. This is a genuine "adding X does not improve prediction."
• Marginal predictor effect null — an underpowered statement: the predictor's adjusted OR/coefficient has a confidence interval that still admits a clinically relevant effect (e.g. OR up to ~1.7). This is "we could not exclude an effect," not "there is no effect."
• A manuscript that collapses both into a blanket "X did not predict Y" overstates the marginal-effect arm. Flag when a negative conclusion rests on a non-significant marginal OR whose CI admits a relevant effect, without separating it from the (better-powered) incremental-value finding → MAJOR. The honest reframe reports both explicitly.

CP3 — Events-per-variable (EPV) per nested model:

• Report EPV for each nested model, not just the full cohort. A model with 6 covariates and 60 events (EPV 10) is adequately supported; the same 6 covariates in an age≥60 subgroup with 13 events (EPV ≈ 2) is separation-prone and unstable.
• Penalization (Firth / ridge / LASSO) is justified and should be stated when EPV < 10; subgroup or interaction models with EPV in the low single digits are separation-affected and must be descriptive-only, not presented as estimated effects.
• Subgroup/interaction effect estimates from a model with EPV ≲ 5, presented as if reliably estimated, or no EPV disclosure for the nested models → MAJOR (subgroup) / MINOR (EPV simply not reported but adequate).

CP4 — Net benefit is a model comparison, not a policy endorsement:

• A decision curve showing the model exceeds treat-all / treat-none across a threshold range is a model-vs-default statement about ranking, not evidence that the underlying imaging / test / screening strategy should be adopted in practice. Net benefit is computed on the same outcomes and says nothing about cost, downstream harm, or whether the test should be ordered.
• A conclusion that reads decision-curve superiority as an endorsement of the clinical strategy ("supports using X for screening / triage") → MAJOR (scope; cross-link scope-coherence-gate.md). Reframe to the model-comparison claim the DCA actually supports.

CP5 — Intended-use horizon leakage (claim-timepoint vs feature availability):

• Read the intended-use horizon off the title / abstract / aim — the adjectives preoperative, pretreatment, baseline, screening, triage, pre-procedure assert a decision made before some event X. Then check each predictor's availability timepoint: any feature that exists only after X (post-procedure pathology, treatment-response variables, post-contrast-only measures in a "pre-contrast screening" claim, follow-up-derived labels) is leakage relative to the claimed use, even if it is not train/test leakage.
• This is a claim × feature-availability mismatch: the model may be statistically clean yet unusable at the moment its claim targets. Distinct from ordinary data leakage (which is about train/test contamination) — this keys on the decision horizon.
• Flag the specific offending predictor(s) and either narrow the intended-use claim to the horizon at which all inputs exist, or drop the post-horizon features and re-fit. → MAJOR when a post-horizon variable supports a pre-horizon decision claim.

CP6 — Validation-nomenclature conflation (development step vs test step):

• Distinguish the development step (including any internal cross-validation or bootstrap optimism correction) from a held-out / external test step. Flag phrasing that merges them: "developed with external validation" where a single external set is used as development data; "trained and validated on the external cohort"; an "external validation set" that was used for tuning/feature selection.
• Cross-validation and bootstrapping are development-time optimism corrections, not external validation; calling them "validation" overclaims out-of-sample evidence. Require an explicit statement of which data trained, which tuned, and which was touched only once for the final estimate.
• → MAJOR when the overclaim props up a generalisability/deployment-readiness conclusion; a clarify-request when the data flow is actually clean but the wording is loose.

Output template (CP1 / CP2 example):

"Discrimination is bootstrap optimism-corrected, but the calibration slope (0.99) and the net-benefit curves are computed in the development sample and described as 'well calibrated' and 'clinically useful' without the apparent caveat — an in-sample slope near 1.0 is expected and does not establish out-of-sample calibration. I'd suggest labelling the calibration and decision-curve results 'apparent' or extending the optimism correction to them. Relatedly, the Conclusion states that the marker 'did not predict' the outcome, but two distinct results are merged: the incremental-value analysis is well powered (ΔAUC ≈ 0, optimism correction favours the parsimonious model), whereas the marginal adjusted OR has a confidence interval that still admits an effect up to ~1.7 (underpowered). I'd report these separately — 'adding the marker did not improve prediction' is supported; 'the marker has no association' is not."

<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review. Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal recommendation. Each consuming skill maps findings to its own output:

• peer-review: Major / Minor comments + Confidential Comments to the Editor; a design-level

flaw (verification bias, two-gate sampling, unblinded reference) is placed as Major #1.

• self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.

Do NOT edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync. -->

Diagnostic-accuracy / reader-study probes (D1–D6)

A checklist for diagnostic test accuracy (DTA) primary studies — an index test against a reference standard, including multi-reader multi-case (MRMC) reader studies (e.g., AI-vs-reader or modality-comparison). These probes complement (do not replace) the generic Phase 2 issue checklist and the STARD / QUADAS-2 items; they target the biases QUADAS-2 names and the MRMC design/variance issues a reader study adds. Pairs the analyze-stats table-standards/table-types/reader_study.md table and the make-figures exemplar_plots/mrmc_roc.md figure. (For a DTA meta-analysis, use sr_ma.md.)

D1 — Reference standard validity + verification bias:

• Is the reference standard appropriate and applied to (essentially) all participants? Partial verification (only test-positives get the gold standard), differential verification (different reference standards by index result), and incorporation bias (the index test is part of the reference) each inflate accuracy. QUADAS-2 Domain 3/4.
• Was the reference standard interpreted without knowledge of the index result, and was the time interval between index and reference short enough that the target condition did not change? A long or undefined interval, or a reference that incorporates the index, → MAJOR.

D2 — Spectrum + sampling design:

• How were participants sampled — a single-gate consecutive/random series of those with the target condition suspected (preserves spectrum and prevalence), or a two-gate case-control design (known cases vs healthy controls)? Two-gate sampling overestimates accuracy and distorts spectrum; it is a MAJOR interpretive limit when the headline is clinical accuracy, not just proof-of-concept.
• Is the study prospective with a pre-specified index threshold, or retrospective with a threshold chosen on the same data (D-linked to optimism)? Spectrum (disease severity, comorbid mimics) should match the intended-use population.

D3 — Blinding of index and reference interpretation:

• Was the index test interpreted blind to the reference standard and to clinical information that would not be available at the point of use, and vice versa? Unblinded interpretation (review bias) inflates agreement.
• For an AI index test, was the operating threshold pre-specified (not tuned on the test set)? A test-set-tuned threshold reported as the result is an optimistic-validation finding (pair with exemplar_reviews/optimistic_validation_reporting.md).

D4 — Indeterminate / uninterpretable results:

• How were indeterminate, uninterpretable, or intermediate index results handled — excluded, or counted (intention-to-diagnose)? Silently dropping them inflates accuracy. State the rule and report an intention-to-diagnose sensitivity analysis (non-diagnostic counted as wrong). Undisclosed exclusion of indeterminates → MAJOR.

D5 — MRMC reader-study design:

• Is the design fully crossed (every reader reads every case in every modality) or a nested/split-plot variant — and is that stated? Was reading order randomized and a washout interval used between modalities so a case is not recognised from a prior read? Absent washout/order control in a within-reader modality comparison → MAJOR (memory/recognition bias).
• Are the readers sampled to generalise to the intended reader population (number, expertise mix), and were training/instructions and the information available (priors, clinical data) standardised? A 2–3 expert-reader convenience panel cannot support a population-level "AI matches radiologists" claim — flag the generalisation gap.

D6 — MRMC analysis + estimand:

• Does the analysis account for both reader and case variability (Obuchowski–Rockette / Dorfman–Berbaum–Metz or an equivalent multi-reader model), or does it treat readers as fixed / pool reads as if independent? Ignoring reader variance understates uncertainty and is a MAJOR statistical flaw for a generalising claim.
• Is the estimand clear: reader-averaged vs a fixed specific reader; per-patient vs per-lesion (clustered) unit; superiority vs non-inferiority with a pre-specified margin? Are per-reader results shown alongside the reader-averaged estimate (a single averaged AUC can hide one weak reader)? A non-inferiority claim with no pre-specified margin, or a clustered design analysed as independent, → MAJOR.

D7 — Index-test-as-enrollment-criterion circularity:

• Cross-check the inclusion criteria against the index / proposed test. When a study proposes or validates a diagnostic or classification instrument and uses a threshold of that same instrument (or a component of it) as an enrollment criterion, the validation is circular: the spectrum is built into the design, and sensitivity/specificity (or the instrument's apparent discrimination) are inflated by construction.
• Typical signatures: "patients were included if [index score] ≥ T" in a paper whose aim is to evaluate that index; enrolling on a positive screening test to then "validate" the screening test; defining the diseased group by the same reader/algorithm output under study.
• This is design-level, not a reporting fix — escalate past an ordinary Major (a co-reviewer / editor reads it as a fatal selection/spectrum artifact). The fix is a reference standard and an enrollment criterion that are independent of the index test (a consecutive suspected-disease series), not a re-analysis.

Output template (D2 / D6 example):

"The study uses a case-control (two-gate) design — confirmed cases versus healthy controls — rather than a consecutive series of patients in whom the diagnosis was suspected. This typically overestimates accuracy and does not reflect the intended-use spectrum, so I'd read the reported sensitivity/specificity as proof-of-concept rather than clinical accuracy, and suggest tempering the Abstract accordingly. Separately, the reader study reports a single reader-averaged AUC; because readers are a sample, I'd suggest an MRMC analysis (e.g., Obuchowski–Rockette) that accounts for both reader and case variance, with per-reader estimates shown and the unit of analysis (per-patient vs per-lesion) stated."

Discipline — leads vs findings (applies to D1–D6):

• A verification/blinding/spectrum concern from a quick scan is a lead until Methods and the participant flow are read together — distinguish under-reporting (ask to clarify) from a true design bias (MAJOR).
• Anchor each comment to the exact bias (partial vs differential verification; single- vs two-gate; reader-averaged vs fixed-reader; per-patient vs per-lesion) and the location. Keep severity tied to what the flaw does: two-gate sampling, incorporation bias, or ignoring reader variance is design/analysis-level (MAJOR, often Major #1); an unreported reading-order detail is a clarify-request.

<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review. Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal recommendation. Each consuming skill maps findings to its own output:

• peer-review: Major / Minor comments + Confidential Comments to the Editor; a task- or

design-level flaw is placed as Major #1.

• self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.

Do NOT edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync. -->

Fairness / Equity / Subgroup-performance probes (EQ0–EQ6)

A 7-probe checklist for AI/ML, prediction, or diagnostic studies that make (or imply) a claim about performance across a heterogeneous population — generalizable / deployment-ready / "works for patients" claims, or studies that present subgroup analyses as evidence of fairness or equity. These probes complement (do not replace) the generic Phase 2 issue checklist, the AI-overclaiming probes (ai_overclaiming.md, with which EQ co-applies), the prediction-model probes (clinical_prediction_model.md, whose EPV logic EQ5 reuses), and the reporting items in TRIPOD+AI, DECIDE-AI, and CONSORT-AI. They target one recurring failure: an aggregate performance number, or an eyeballed "similar across groups" statement, presented as evidence that a model is equitable or broadly deployable.

EQ0 — Applicability gate (apply only when a cross-population claim is made):

• Apply EQ1–EQ6 when the manuscript claims (or its framing implies) that the model/score/test

performs adequately across a heterogeneous population, or reports subgroup analyses as a fairness/equity argument.

• Do not fire these probes on a study that explicitly **scopes its claim to a single,

well-defined population** and does not generalise beyond it — there, the right probe is scope coherence (the conclusion must stay within the evaluated population), not a fairness audit.

EQ1 — Disaggregated subgroup performance, not aggregate-only:

• An overall AUC / sensitivity / specificity / calibration is not evidence of per-subgroup

performance — it can hide large, offsetting subgroup gaps. A broad-applicability or deployment claim requires disaggregated metrics, with confidence intervals, for pre-specified subgroups defined by clinically and (where collected and appropriate) socially relevant attributes (e.g. age band, sex, acquisition site / scanner / vendor, and protected attributes when available).

• Reporting only the pooled metric while claiming the model works "across patients" / "in practice"

→ MAJOR. Reporting per-subgroup point estimates without CIs, so the reader cannot tell a real gap from noise → MINOR.

EQ2 — Error-rate parity ≠ discrimination parity; base-rate dependence:

• Equal AUC across groups does not imply equal sensitivity/specificity at the operating

threshold, nor equal PPV/NPV — and a single threshold applied across subgroups with different outcome prevalence yields different error trade-offs by construction. "Discrimination is similar, therefore the model is fair" conflates two different fairness criteria.

• The probe asks for the metric that drives the deployment harm (often the subgroup

false-negative rate, or PPV at the decision threshold), not just AUC parity. A fairness claim resting on AUC parity alone, with the threshold-dependent error rates unreported or unequal → MAJOR.

EQ3 — A fairness claim needs a named estimand + a gap test, not eyeballed point estimates:

• "Performance was similar across groups" is an estimand-free assertion. If fairness is claimed, the

manuscript must (a) name the fairness criterion — error-rate parity / calibration-within-groups / equalized odds — recognising that these can mathematically conflict and cannot all hold at once when base rates differ; (b) report the between-group gap with a confidence interval and a test, not overlapping point estimates; and (c) ideally pre-specify it.

• A parity conclusion read off non-overlapping-looking point estimates with no gap CI / test →

MAJOR (eyeballed/data-mined parity). A reported gap that was not pre-specified and is selected post hoc among several metrics → MINOR (disclose as exploratory).

EQ4 — Development-cohort representativeness vs the deployment population:

• Report the demographic / site composition of the derivation (training) data. A subgroup that

is small or absent in development cannot be claimed to generalise to it, regardless of the pooled test metric; and including an attribute as a covariate is not the same as validating performance within that stratum.

• A deployment / "generalisable to [population]" claim for a subgroup that is unrepresented or

trivially small in the development data → MAJOR. Composition simply not reported → MINOR (it is a required disclosure under TRIPOD+AI).

EQ5 — Subgroup power / events-per-variable for the fairness comparison:

• A "no disparity" conclusion drawn from subgroups with few events is underpowered — the same

separation/instability problem the prediction-model EPV probe (CP3) raises. A fairness null needs the between-group gap CI to exclude a clinically relevant difference; a wide CI that still admits a meaningful gap is "we could not detect a disparity," not "the model is equitable."

• Subgroup comparisons with EPV in the low single digits, presented as evidence of fairness → MAJOR

(overclaim); the comparison should be descriptive-only. EPV / event counts per subgroup not reported → MINOR.

EQ6 — Equity-aware framing + guideline alignment; a limitation is not an endorsement:

• Scope the conclusion to the populations actually evaluated, and align reporting to TRIPOD+AI

(fairness / subgroup-performance and training-data composition items), DECIDE-AI, and CONSORT-AI (pre-specified subgroup analyses). A known fairness limitation (untested subgroup, unequal error rate, unrepresentative training data) must not be converted into a deployment endorsement or a recommendation-grade equity claim.

• Recommendation-grade equity language ("equitable", "fair across groups", "ready for diverse

populations") that the evidence does not support, or a conclusion that silently generalises past the evaluated subgroups → MAJOR. Cross-link ~/.claude/rules/scope-coherence-gate.md: an aggregate or single-population claim cannot stand on subgroup-silent data.

---

Output mapping. In peer-review, map each finding to a Major / Minor comment; EQ1 (aggregate-only performance behind a deployment claim), EQ2 (AUC-parity-only fairness claim), and EQ4 (deployment claim for an unrepresented subgroup) are design-level — surface them in the Confidential Comments to the Editor and place the strongest as the Major #1 candidate. EQ4 and an EQ5 underpowered-null overclaim are frequently unfixable in the current data (the missing subgroup or events cannot be added in revision) and govern the recommendation per Phase 2F. In self-review, a design-level EQ finding becomes a Fatal Anticipated Major Comment and a reporting-level one a Fixable Anticipated Minor Comment, tagged with the closest category letter. EQ co-applies with ai_overclaiming.md (a fairness gap is one route to an over-broad clinical claim) and reuses the EPV logic of clinical_prediction_model.md (CP3) at the subgroup level.

Synthetic manuscript fixture (claim-vs-artifact test)

Methods

The primary analysis was the association between emphysema and all-cause mortality estimated by a multiple-imputation Cox model. This primary endpoint was re-designated at the manuscript stage from the pre-specified complete-case model after the imputed estimate was found to be more favourable.

Results

In the primary model, emphysema was not associated with all-cause mortality. The E-value for the primary association (HR 1.34) was 2.79, which we interpret as robust to unmeasured confounding.

In an exploratory analysis, the E-value for the cancer-specific subdistribution hazard (sHR 2.25) was 3.93, suggesting the cancer signal is hard to explain away.

Synthetic pre-registration fixture (claim-vs-artifact test)

Primary outcome: the association between emphysema and all-cause mortality in the complete-case multivariable Cox model is the primary analysis of this study.

Secondary outcomes: cause-specific mortality (cancer, cardiovascular, respiratory).