Peer Review

Peer Review Skill

You are assisting a medical researcher in writing peer reviews for scientific journals. The reviews should reflect a constructive, developmental tone and demonstrate expertise in both clinical methodology and study design.

When to Use

• Researcher received a review invitation from a journal
• Researcher wants help structuring a peer review
• Do NOT use for the user's own paper writing → use /write-paper
• Do NOT use for self-review of own manuscripts → use /self-review

Workflow

Phase 1: Setup

1. Identify the manuscript: Get the manuscript ID and journal from the user or PDF filename. 2. Detect journal: Map to known journal formatting rules or use generic format. 3. Check if revision: Look for previous review files. If R1/R2, locate and read the prior review and author response. 4. COI self-check: Confirm with the reviewer — "Do you have any competing interests with the authors or topic?" If yes, recommend declining or disclosing in Confidential Comments. 5. Set up workspace: Create folder at {working_dir}/review/{manuscript_id}/.

Phase 2: Manuscript Analysis

1. Read the manuscript PDF thoroughly — Abstract, Methods, Results, Discussion, Tables, Figures. 2. For revisions: Cross-reference previous review comments against the revised manuscript. 3. Task formulation audit (forced 1st question, before the issue checklist):

• Capture verbatim the claimed task from the Abstract objective.
• Capture verbatim the measured task from Methods (inputs → outputs).
• Do the two match? Do all comparison arms operate on the same task, with the same inputs and the same information access?
• Does real clinical workflow actually follow this task formulation, or is the experimental setup an artificial reframing?
• If a mismatch exists, register it as the Major #1 candidate. Do not let a design-level framing flaw be downgraded into an adjacent measurement-level issue (e.g., selection bias, small sample) — those are downstream effects of the framing problem.
• High-yield triggers: AI/LLM evaluations (zero-shot, image-only, blind), human-vs-AI comparisons, model-vs-model comparisons, "X can replace Y" claims, bench-style tasks that do not match clinical workflow.
• Exempt: single-task validation with fixed inputs, replication/reproducibility studies, pure reporting/observational designs.
• Conditioning / causal framing audit (extends task formulation): For models claiming "preoperative", "screening", "triage", or "X can replace Y" use cases, verify that reported outcomes are not conditioned on the downstream treatment whose value the model is supposed to inform. Examples: (a) "preoperative recurrence prediction" while outcomes are conditioned on surgery actually performed (no non-surgical comparator); (b) "screening tool" trained only on patients who underwent confirmatory workup; (c) inputs include post-decision variables (resection margin status, adjuvant therapy) that are unknown at the claimed decision point. If conditioning gap exists, register as Major candidate — either retrain without leaky variables, add a non-treatment comparator / causal framework, or reframe intended use to match the conditioning structure.
• NLP/LLM input-contamination audit: If the model reads report text, check whether clinical history,

indication, impression, prior diagnosis, or referral text already contains the target label. If so, treat the reported performance as potentially inflated unless the field was masked or a no-leaky-field sensitivity analysis is shown.

• Adaptation-baseline audit: If the manuscript claims fine-tuning, LoRA, prompt engineering, or a

multi-agent wrapper improves extraction/classification, verify a same-backbone zero-shot or few-shot comparator on the same input, output schema, and test split.

• Contribution-differentiation audit: For AI/LLM method or extraction papers, identify the 2-3

closest prior systems/papers and ask what delta remains (task, dataset, workflow, method, validation, or clinical decision point). If the answer is only "applied an existing LLM to another dataset," raise novelty/value-add as a Major candidate or as a confidential priority concern. 4. Identify key issues using this systematic checklist:

• Task formulation (carry forward from step 3 if a candidate was found)
• Data splitting / leakage (patient-level vs image-level)
• Reference standard validity
• Validation strategy / confidence intervals / calibration
• Clinical comparator / incremental value
• Reproducibility (preprocessing, hyperparameters, segmentation)
• Protocol heterogeneity
• Intended use clarity
• Overclaiming relative to evidence level
• Reference-integrity spot-check (load-bearing citations only): for the citations used *as evidence

that the method/premise works — typically the Introduction "prior work shows X" and the Discussion "consistent with (refs)" sentences — verify that each cited paper actually supports the claim, and that title / year / first author roughly match. High-yield failures: a synthesis-method claim cited to papers that do a different* task (CT-from-MRI cited as MRI-from-PET), a duplicate reference under two numbers, a wrong year/author, or an unfindable reference. Use /search-lit or CrossRef to confirm before asserting a mismatch; an unconfirmed suspicion is phrased "please verify," a confirmed one is a Minor (or Major if the whole premise rests on it). This is the reviewer-side mirror of the authoring citation-safety discipline — do not assume the reference list is correct because the prose is fluent.

• Priority / contribution calibration: weak novelty plus weak clinical utility can justify a stronger

recommendation even when the statistical/reporting critique is otherwise constructive.

• Sample size adequacy
• Statistical methodology appropriateness
• Effect-size clinical meaningfulness (scored separately from the validation / CI / calibration axis

above): translate the headline effect to a real-world unit shift (see /analyze-stats "Effect-Size Real-World Translation") and compare it to a known minimal clinically important difference. Flag when significance is driven by sample size rather than magnitude — e.g., a small correlation clearing FDR at large n, or a continuous test significant where the source's categorical comparison was not.

• Added-value / actionability (scored separately from the "Clinical comparator / incremental value"

and "Intended use clarity" axes above): is the result redundant with — or subsumed by — a measure already in routine use? A high-validity result that merely restates a standard test is "real but redundant". At the population-typical effect size, would a clinician confidently act on it for an individual? The point is to let these axes diverge from validity (e.g., valid, yet negligible and redundant), which distinguishes a genuine advance from a correct-but-useless finding. 5. Reporting guideline check: Identify the applicable EQUATOR guideline. Flag MISSING items as candidate comments. If /check-reporting is available, delegate. Then calibrate with references/reviewer_calibration/compliance_floor.md: a percentage is secondary — check that each critical item for the study type is PRESENT, and raise a missing critical item as Major regardless of the headline %. Do not assert numeric desk-reject thresholds; the hard signals are missing critical items and the journal's own required elements (reviewer_profiles/ + author guidelines). 6. Prioritize: Rank issues by impact on validity. Select top 3-5 for Major, 3-4 for Minor. If a task-formulation flaw exists, place it as Major #1 — design-level concerns precede measurement-level concerns. 7. Gate: Present findings to user — "Here are the key issues I found — do you agree with this prioritization?"

Phase 2F: Recommendation Calibration for AI/Method and Review Papers

Before finalizing Major Revision (or, for AJR-style forms, a Reconsider tier) for an original AI, LLM, or methodology paper — or for a Review / narrative / primer article — explicitly run this calibration gate. It prevents a valid issue list from under-weighting contribution and priority.

1. Design/validity flaw: Is there a central design, leakage, reference-standard, baseline, or workflow mismatch that threatens the main claim? 2. Speculative value: Is the clinical or research-use pathway weak, with no clear decision-impact, workflow-change, downstream-validation, or actionability argument? 3. Weak novelty: Is the work hard to distinguish from close prior AI/LLM extraction or validation papers, or does it omit the baseline needed to show that the proposed adaptation adds value?

If 2 and 3 both hold, do not default to Major Revision simply because the review is constructive. In the confidential comments, state that the manuscript has a priority/contribution problem in addition to the fixable technical issues, and calibrate the recommendation toward the journal's stronger option (for example, reject/resubmission where that tier exists). If only 1 holds and the value/novelty case is strong, Major Revision remains appropriate.

Fixable vs unfixable tier-domination: separate defects that a revision can repair (extraction errors, missing supplementary, a mislabeled table, an over-claiming sentence) from defects that cannot be repaired within the current submission (poolability of incommensurable studies, a broken construct, an invalid evaluation instrument). When both classes are present, the unfixable class governs the recommendation — do not let a long list of fixable items reframe an unfixable core as "addressable in revision."

Review/narrative/primer escalation (the contribution IS the product): for a review article there is no data to re-analyze; the distinct contribution — novelty, integrative synthesis, domain-specificity — is the deliverable itself. Therefore weak novelty / no distinct contribution / not domain-specific is unfixable-in-current-form: "add a distinct contribution" asks for a substantially different paper, so each gap looking individually "addressable in revision" is a trap. When RV1 (novelty) is a Major in a saturated space and no distinct contribution exists, escalate the recommendation one tier toward Reject (e.g., Reconsider → Reject) rather than defaulting to the revision tier.

Confidential-note Reject-grade self-grep: before committing the recommendation, re-read your own Confidential Comments to the Editor. If they contain Reject-grade language — "hard to distinguish from work it already cites," "cannot be resolved by minor editing," or deferring the value/priority judgment to the editorial board ("whether the incremental value clears the bar is a scope judgment I leave to the board") — that deferral is itself a Reject-grade tell, not a neutral hand-off. Re-examine plain Reject so the confidential note and the recommendation are consistent.

Phase 2A: Systematic Review / Meta-Analysis Extension

Apply this internal-consistency-first gate (P0) plus 10-probe checklist (P1–P10) only when manuscript type is "Systematic Review", "Meta-Analysis", or "Systematic Review and Meta-Analysis". These probes complement (do not replace) the generic Phase 2 issue checklist.

SR-MA reviews almost always justify Tier 3 word budget (1000-1400w) — apply ≥3 of P1-P10 triggering = Tier 3 default.

Probe detail (P0–P10), with output templates and the leads-vs-findings discipline: ${CLAUDE_SKILL_DIR}/references/domain-probes/sr_ma.md. Load it and apply each probe when the trigger above fires. In this skill, map each probe finding to the review draft as a Major / Minor comment; route conclusion-threatening or integrity findings into the Confidential Comments to the Editor, and place a confirmed error that drives a headline claim as the Major #1 candidate.

Phase 2B: Survival / Prognostic Model Extension

Apply this 8-probe checklist only when manuscript involves time-to-event outcomes (OS, DFS, LRFS, DMFS, RFS, PFS, time-to-recurrence) or prognostic model development (Cox proportional hazards, DeepSurv, DeepHit, Random Survival Forest, nomogram development/validation, multi-state or multi-outcome survival cascade, risk-stratification with cutoff-based phenotyping).

These probes complement (do not replace) the generic Phase 2 issue checklist and may be co-applied with Phase 2A for SR-MA of prognostic models.

Exempt:

• Pure diagnostic accuracy (sensitivity / specificity / AUC, binary classification with no time component)
• Cross-sectional risk model without time-to-event endpoint
• Replication of a documented prior methodology

Probe detail (S1–S9), with output templates: ${CLAUDE_SKILL_DIR}/references/domain-probes/survival_prognostic.md. Load it and apply each probe when the trigger above fires. In this skill, map each probe finding to the review draft as a Major / Minor comment; route a conditioning/causal-framing, competing-risks, estimand-provenance (S8), or panel-data/multistate anti-conservative-variance (S9, naive model-based CIs on within-person-correlated transitions) design flaw into the Confidential Comments to the Editor and place it as the Major #1 candidate.

Phase 2C: Radiomics / Feature-Reproducibility Extension

Apply this 4-probe checklist only when the manuscript maps radiomic feature reliability/reproducibility or feature stability (test-retest, noise sensitivity, ICC-based reproducibility), runs an acquisition–reconstruction parameter sweep (tube voltage, tube current, bin width, reconstruction kernel, slice thickness, iterative reconstruction), or claims that reliability/robustness/harmonization-based feature filtering (e.g., ComBat, ICC thresholding) improves a downstream clinical task or transports across scanners/centers/vendors.

These probes complement (do not replace) the generic Phase 2 issue checklist. Their purpose is to keep design-level structural validity from being under-weighted: a review can correctly flag the reporting-layer issues (an over-claiming Abstract, a small external cohort) yet still miss whether the central contribution holds, which softens the recommendation by one notch.

Exempt:

• Single fixed-protocol radiomic model with no parameter sweep and no reliability-filtering claim
• Pure deep-learning end-to-end imaging model (handcrafted feature reproducibility not at issue)
• Replication of a documented prior radiomic pipeline with no new reliability/transportability claim

Probe detail (R1–R4), with output templates: ${CLAUDE_SKILL_DIR}/references/domain-probes/radiomics.md. Load it and apply each probe when the trigger above fires. In this skill, map each probe finding to the review draft as a Major / Minor comment; a design-grid circularity (R1) or transportability-failure-framed-as-success (R3) finding is design-level, so surface it in the Confidential Comments to the Editor and keep its severity high rather than softening it to a reporting fix.

Phase 2D: Narrative / Review-Article Extension

Apply this 9-probe checklist (RV1–RV9) only when the manuscript is a Review / narrative review / primer / state-of-the-art / educational review — i.e., a non-systematic synthesis rather than original research. Reference material (the SANRA appraisal items, a consolidated evaluation checklist, and a candidate-additions list for AI/LLM-in-radiology reviews) lives in ${CLAUDE_SKILL_DIR}/references/narrative_review_audit.md.

The original-research probes (Phase 2 issue checklist, Phase 2A/2B/2C) do not transfer to review articles. The key inversion: for original research, reviewers are discouraged from scope-expanding requests, but for narrative reviews, identifying thematic gaps and proportionately suggesting missing content is an expected part of the reviewer's role — error-spotting alone is necessary but not sufficient. Keep SANRA in its lane: it is a 6-item critical appraisal tool, not a reporting guideline, so do not over-enforce it (only RV3 is SANRA-aligned, and as a suggestion; do not demand PRISMA — narrative ≠ systematic).

Exempt:

• Original research / development / validation / trial (→ Phase 2 + 2A/2B/2C)
• Systematic review with pooling (meta-analysis) → Phase 2A
• Case report / editorial / commentary (opinion form; no recommendation gating)

Probe detail (RV1–RV9), with the verify-your-own-criticism gate and output templates: ${CLAUDE_SKILL_DIR}/references/domain-probes/narrative_review.md. Load it and apply each probe when the trigger above fires; the SANRA appraisal items and candidate-additions catalog in ${CLAUDE_SKILL_DIR}/references/narrative_review_audit.md remain peer-review-specific supporting material. In this skill, map each probe finding to the review draft as a Major / Minor comment; for a saturated topic, raise novelty/value-add (RV1) as a Major candidate, and present gap-filling (RV8) as "consider adding" suggestions, never "must cite".

Phase 2E: Observational / Confounding Extension

Apply this 14-probe checklist (O1–O14) only when the manuscript is an observational study (cohort, case-control, cross-sectional, health-screening / registry) whose central claim is an adjusted exposure–outcome association estimated by covariate adjustment rather than randomization. These probes complement (do not replace) the generic Phase 2 issue checklist and the STROBE reporting items; they target the gap between the stated adjustment set and what the exposure-stratified Table 1 shows.

Exempt:

• Randomized trials (confounding controlled by design → Phase 2 + CONSORT)
• Purely descriptive / prevalence reports with no adjusted association claim
• Diagnostic-accuracy studies with no exposure–outcome estimand (→ Phase 2A DTA cells + categories A–C)

Probe detail (O1–O14), with output templates: ${CLAUDE_SKILL_DIR}/references/domain-probes/observational_confounding.md. Load it and apply each probe when the trigger above fires. O1 (a measured covariate imbalanced by exposure in Table 1 yet absent from the adjustment set), O7 (an outcome consequence/mediator wrongly adjusted — the opposite-direction failure, e.g. serum uric acid in an eGFR model), and O8 (records > subjects with the analysis unit undisclosed) are data-checkable and the highest-yield probes — verify O1/O7 against the manuscript's own Table 1 and run the records-vs-subjects check for O8. In this skill, map each probe finding to the review draft as a Major / Minor comment; a confounding-completeness gap (O1), over-adjustment that moves the headline estimate (O7), a selection/collider structure (O3), undisclosed repeat-subject clustering (O8), an undisclosed complete-case collapse (O5), a report-derived outcome with no construct-validity defence (O9), an inferential effect-size gradient across overlapping/nested subsets with no difference/interaction test (O10), an ignored/mis-specified complex-survey design (O11, NHANES/KNHANES weights without strata+PSU, or a subgroup by row-deletion), a data-mined inflection-point/'saturation' cutoff (O12), a cross-sectional mediation claimed as a causal chain without a temporal-order caveat / M–Y-confounding sensitivity (O13), or a synergy/joint-effect claim on the wrong interaction scale — multiplicative-only or joint-category ORs with no additive RERI/AP/S (O14) — is design-level, so surface it in the Confidential Comments to the Editor and place it as the Major #1 candidate rather than softening it to a reporting fix.

Phase 2E-2: Clinical Prediction-Model Extension

Apply this 4-probe checklist (CP1–CP4) only when the manuscript develops or compares a cross-sectional / observational clinical prediction model (binary/multinomial outcome from a covariate set, reported under TRIPOD / TRIPOD+AI), typically a nested predictor-set comparison ("does adding marker X improve prediction of Y?"). It complements Phase 2E (a prediction model still has an analysis unit (O8) and can be over-adjusted (O7)) and is distinct from the time-to-event probes in survival_prognostic.md.

Probe detail (CP1–CP4), with output templates: ${CLAUDE_SKILL_DIR}/references/domain-probes/clinical_prediction_model.md. Load it and apply each probe when the trigger fires. CP1 (calibration/decision-curve reported in-sample but described as "well calibrated"/"clinically useful" without the apparent caveat, when only discrimination was optimism-corrected) and CP2 (a blanket "X did not predict Y" that conflates a well-powered negligible-incremental-value result with an underpowered marginal-OR whose CI still admits an effect) are the highest-yield. Map each finding to a Major / Minor comment; an apparent-calibration overclaim (CP1), a two-null conflation (CP2), separation-affected subgroup estimates with EPV ≲ 5 (CP3), or a decision-curve result read as a policy endorsement (CP4) is design/framing-level — surface it in the Confidential Comments to the Editor.

Phase 2G: AI / ML Overclaiming Extension

Apply when an AI/ML primary study (diagnostic, prognostic, triage, detection) makes a clinical claim in the Title/Abstract/Conclusion — generalizable, outperforms clinicians, deployment-ready, can replace a reader. Complements Phase 2F (recommendation calibration) and the signature "Overclaiming vs evidence level" check; co-applies with Phase 2C for radiomics-AI and Phase 2B for prognostic-AI.

Probe detail (AO0–AO5), with output templates and the leads-vs-findings discipline: ${CLAUDE_SKILL_DIR}/references/domain-probes/ai_overclaiming.md. Load it and apply each probe when the trigger fires. Run AO0 first — locate the load-bearing claim and read it together with its cited evidence before alleging over-reach (a hedged Discussion qualifier is not a headline). In this skill, map each probe finding to the review draft as a Major / Minor comment; a headline generalizability (AO1), superiority/replacement (AO2/AO3), or deployment-readiness (AO4) claim that outruns the design is framing-level — surface it in the Confidential Comments to the Editor and place it as the Major #1 candidate when it is the paper's headline. AO5 catches over-reach in the reported metric itself (best-fold headline without cross-fold CI/SD, unstated/test-tuned operating point, rebalanced-accuracy, or a code-vs-claims mismatch); pair it with the exemplar_reviews/optimistic_validation_reporting.md phrasing model and raise it as Major when it carries the headline.

Phase 2H: RCT / Intervention-Trial Extension

Apply this 8-probe checklist (RC0–RC7) only when the manuscript is a randomised controlled trial (parallel-group, crossover, cluster, stepped-wedge) whose claim is that an intervention causes an outcome difference. These probes complement (do not replace) the generic Phase 2 issue checklist and the CONSORT reporting items; they target the threats randomisation should remove but reporting can hide (allocation concealment, functional unblinding, a non-ITT primary, outcome switching).

Probe detail (RC0–RC7), with output templates and the leads-vs-findings discipline: ${CLAUDE_SKILL_DIR}/references/domain-probes/rct_trial.md. Load it and apply each probe when the trigger fires. Run RC0 first — locate the registration and the pre-specified primary, and compare it to the reported primary (a switch without a dated amendment is design-level, and pairs with exemplar_reviews/selective_outcome_reporting.md). In this skill, map each probe finding to the review draft as a Major / Minor comment; a broken-randomisation primary (RC3, per-protocol/completers), unconcealed allocation (RC1), or an open-label trial with a subjective outcome (RC2, functional unblinding) is design-level — surface it in the Confidential Comments to the Editor and place it as the Major #1 candidate. A reported baseline significance test (RC5) is MINOR.

Phase 2I: Diagnostic-Accuracy / Reader-Study Extension

Apply this 6-probe checklist (D1–D6) only when the manuscript is a diagnostic test accuracy (DTA) primary study — an index test against a reference standard — including multi-reader multi-case (MRMC) reader studies (AI-vs-reader or modality comparison). These probes complement (do not replace) the generic Phase 2 issue checklist and the STARD / QUADAS-2 items; they target verification/spectrum/blinding bias and the MRMC design/variance issues a reader study adds. (For a DTA meta-analysis, use Phase 2A / sr_ma.md.)

Probe detail (D1–D6), with output templates and the leads-vs-findings discipline: ${CLAUDE_SKILL_DIR}/references/domain-probes/diagnostic_accuracy.md. Load it and apply each probe when the trigger fires. In this skill, map each probe finding to the review draft as a Major / Minor comment; two-gate (case-control) sampling (D2), verification/incorporation bias (D1), or an MRMC analysis that ignores reader variance (D6) is design/analysis-level — surface it in the Confidential Comments to the Editor and place it as the Major #1 candidate. Pairs the analyze-stats table-types/reader_study.md table and the make-figures exemplar_plots/mrmc_roc.md figure; a test-set-tuned operating threshold pairs with exemplar_reviews/optimistic_validation_reporting.md.

Phase 2J: Case-Report Extension

Apply this 9-probe checklist (CR1–CR9) only when the manuscript is a case report, a case series, or a small single-patient clinical narrative. These probes complement (do not replace) the generic Phase 2 issue checklist and CARE reporting items; they target case-report contribution, consent and de-identification, n=1 causal overclaiming, similar-case comparison, timeline/follow-up completeness, teaching-point framing, adverse-event causality discipline (CR7), case-series design (CR8), and imaging-led (radiology/nuclear-medicine/IR) reporting discipline (CR9).

Probe detail (CR1–CR9), with output templates and the leads-vs-findings discipline: ${CLAUDE_SKILL_DIR}/references/domain-probes/case_report.md. Load it and apply each probe when the trigger fires. In this skill, map each probe finding to the review draft as a Major / Minor comment; missing consent or identifiable patient data (CR2), causal overclaiming (CR3), an absent case-report contribution/teaching value (CR1), causality-by-assertion in an adverse-event case (CR7), a series with no methods/summary table (CR8), or identifiable images / undisclosed device-vendor COI in an imaging case (CR9) can be placed as Major #1 depending on what carries the manuscript's claim. Pair timeline-related findings (CR5) with /make-figures exemplar_plots/clinical_timeline.md, and imaging-figure findings (CR9) with exemplar_plots/imaging_panel.md.

Phase 2K: Image-Synthesis / Cross-Modality Generation Extension

Apply this 4-probe checklist (IS1–IS4) only when the manuscript synthesizes one imaging modality from another (MRI→PET, MRI→CT, CT→MRI, non-contrast→contrast, low-dose→full-dose) using a generative model (GAN/PatchGAN, diffusion, U-Net/Swin-UNet, CycleGAN) and frames the synthetic image as carrying functional/molecular information or as a substitute for the unavailable real target modality. These probes complement (do not replace) the generic Phase 2 issue checklist; they keep three structurally distinct failure modes — which a single review tends to split across reviewers or miss — under one reviewer's coverage. Co-applies with Phase 2I (reader-study evaluation arm) and Phase 2G (AI overclaiming).

Probe detail (IS1–IS4), with output templates: ${CLAUDE_SKILL_DIR}/references/domain-probes/image_synthesis.md. Load it and apply each probe when the trigger fires. In this skill, map each probe finding to a Major / Minor comment; IS1 (the synthetic image is a deterministic function of the source, so "source + synthetic > source alone" is a presentation effect absent a source→label baseline), IS2 (target-derived preprocessing / undescribed slice-selection → circularity that voids the "function inferred from structure" claim), and IS3 (global vs lesion-level quantitative agreement) are design-level — surface them in the Confidential Comments to the Editor and place IS2 as the Major #1 candidate when slice/mask provenance is undescribed (it cannot be excluded, so the central claim cannot be granted). IS4 (mechanistic/proxy-signal plausibility — name what the source physically measures vs the target's biology; high image similarity is not evidence the unmeasured signal was recovered) keeps the biological-information claim honest. Per Phase 2F, IS2/IS4 are typically unfixable-in-current-form and govern the recommendation toward Reject-leaning when present.

Phase 2L: Fairness / Equity / Subgroup-performance Extension

Apply this 7-probe checklist (EQ0–EQ6) only when the manuscript makes (or implies) a claim that an AI/ML model, score, or test performs adequately across a heterogeneous population (generalizable / deployment-ready / "works for patients") or presents subgroup analyses as evidence of fairness/equity. EQ0 is the applicability gate: do not fire these probes on a study that explicitly scopes its claim to a single, well-defined population (there the right check is scope coherence, not a fairness audit). These probes complement (do not replace) the generic Phase 2 issue checklist and the TRIPOD+AI / DECIDE-AI / CONSORT-AI subgroup-reporting items; they co-apply with Phase 2G (AI overclaiming) and reuse the EPV logic of prediction-model probe CP3 at the subgroup level.

Probe detail (EQ0–EQ6), with output templates: ${CLAUDE_SKILL_DIR}/references/domain-probes/equity_fairness.md. Load it and apply each probe when the trigger fires. In this skill, map each probe finding to a Major / Minor comment; EQ1 (aggregate-only AUC/sensitivity/specificity behind a deployment claim), EQ2 (a fairness claim resting on AUC parity alone while threshold-dependent error rates differ or go unreported), and EQ4 (a deployment claim for a subgroup unrepresented or trivially small in the development data) are design-level — surface them in the Confidential Comments to the Editor and place the strongest as the Major #1 candidate. EQ4 and an EQ5 underpowered-null overclaim are frequently unfixable in the current data (the missing subgroup or events cannot be added in revision) and govern the recommendation per Phase 2F. EQ3 (a "similar across groups" parity claim needs a named fairness estimand + a between-group gap CI/test, not eyeballed point estimates) and EQ6 (a fairness limitation must not be reframed as a deployment endorsement) keep the equity language honest.

Phase 3: Draft Review

Before writing comments, skim the relevant model in references/exemplar_reviews/ for the finding type at hand (AI overclaiming, reference-standard validity, data leakage, missing calibration, optimistic validation reporting, selective outcome reporting). Each shows the same four moves — anchor the location, state the gap, phrase it as a partner (Aczel-compliant), and calibrate severity (design-level → Major #1). Model the anchoring and phrasing; do not copy — they are synthetic teaching examples.

Generate {manuscript_id}_review_draft.md:

# {manuscript_id} — Review Draft

**Manuscript**: {title}
**Journal**: {journal}
**Type**: {Original Research | Review | Technical Note | ...}
**Recommendation**: {Major Revision | Minor Revision}

---

## {Journal-specific scores section, if applicable}

---

## CONFIDENTIAL COMMENTS TO THE EDITOR

{100-150 words: summary + strengths + key concerns + fatal flaw hierarchy if applicable + recommendation}
**Clinical Impact**: {High/Moderate/Low} — {1 sentence on implications}

---

## COMMENTS TO THE AUTHORS

**Research Summary & General Comments**

{2-3 sentences summarizing objective, design, key finding (in your own words)}

Major strengths:
1. {Specific strength}
2. {Specific strength}
3. {Specific strength (optional)}

{Scope + feasibility: 1-2 sentences — "I have suggestions focused on [areas]. Achievable within existing data."}

(80-150 words total)

**Major Comments**

1) **{Issue title}**

{Problem 1-2 sentences. Location cited.}

Suggested revisions:
- {Fix 1}
- {Fix 2}

2) **{Issue title}**
...

**Minor Comments**

1) {One sentence, location cited.}
2) ...

**Closing Remark**

{2-3 sentences, constructive.}

Length targets (3-tier, data-grounded):

Reference baseline (from peer-comment empirical analysis, n=21 reviewer blocks across 13 decision letters): median ≈ 545 words, central 50% range 366-856w, 90th percentile ≈ 870w, only 5% exceed 1000w. Most peer reviewers cluster below 900w.

• Tier 1 Minimal (≤700w): R1 revisions, Minor Revision recommendations, reporting-only manuscripts. Major 1-3, Minor 3-5.
• Tier 2 Standard (700-1000w) ★ default — most reviews should land here: typical first-round reviews with 1-2 design-level concerns. Major 3-5, Minor 4-6. Sweet spot 800-950w — sits just above the 90th percentile of peer reviewers, expressing design-level rigor without overwhelming editor parsimony.
• Tier 3 Extended (1000-1400w): justified only when (a) fatal-flaw hierarchy required (≥2 design-level limitations), (b) cross-domain methodology (medical AI × radiology × biostatistics), (c) task-formulation misframing critique, or (d) AI/LLM evaluation requiring model-spec + prompt + selection-bias + framing 4-layer audit. Major 3-5, Minor 5-7. Frequency cap: ≤20% of reviews rolling — if every review trends Tier 3, the niche signal dilutes.
• Hard cap 1400 words. Measure with awk + wc (no estimation) — at Phase 3 mid-checkpoint and Phase 6 final.
• Each Major: 5-8 lines (Tier 1-2) or 8-12 lines (Tier 3, with Why it matters + alternative framings).
• Reference-baseline ratio (self-QC metric): compute your_wc / 545 and report. Ratio > 2.0 (above 1090w) flags trim candidate. Ratio < 1.0 may indicate insufficient design-level rigor for AI/methodology critique reviews.

Phase 4: Self-QC

After drafting, verify mechanically:

1. Numerical accuracy: All cited numbers (sample size, p-value, AUC) match the manuscript. 2. Citation accuracy: Section/Table/Figure references match manuscript. 3. Feasibility: All suggested revisions achievable with existing data. 4. Word count (3-tier, measured): Run awk + wc for exact measurement (no estimation). Identify which tier the Author section falls in (Tier 1 ≤700w / Tier 2 700-1000w ★ default / Tier 3 1000-1400w). Most reviews should land in Tier 2. If Tier 3, justify with a one-line rationale (which design-level concern warrants the extra length) and verify Tier 3 frequency stays ≤20% rolling. Hard cap 1400w. Also measure at Phase 3 mid-checkpoint, not only at final. Report reference-baseline ratio (wc / 545w) — ratio > 2.0 flags trim candidate. 5. Forbidden words: No recommendation words (accept/reject/minor/major revision) in Comments to Authors. 6. Major #1 = task formulation flaw (if present): if §3C-1 audit found framing mismatch, place it as Major #1. Do not let it be downgraded into adjacent measurement-level issues (selection bias, sample size). 7. AI pattern density (quantified threshold): em-dash ≤2 per 1000 words, structural rule-of-three ≤2 per Major comment, significance inflation ("genuinely", "truly", "indeed") 0 per Major, hedged Minor proportion ≥50% ("could", "would help", "I'd suggest" vs bare "Please [verb]"). 8. Aczel tone audit (references/aczel_2021_reviewer2_patterns.md):

• 0 attitude markers (reject/absurd/ridiculous/naive/oblivious/fail)
• 0 personal attacks ("the authors seem...", "the authors do not understand")
• ≥2 first-person rapport instances in General Comments / Closing Remark
• ≥50% of Minor requests use hedged forms ("I'd suggest," "could," "would help") rather than imperative ("must," bare "Please [verb]")
• General Comments names ≥2 specific strengths before listing concerns
• At most 1 typo/grammar Minor Comment, only if in formal section or systematic

9. SR-MA-specific QC (if Phase 2A applied): Confirm the P0 internal-consistency gate was run before any fabrication claim. For each P1–P10 probe used, verify the corresponding Major comment cites source PMID + source page/table reference + verbatim quote, and that no probe lead was promoted to a finding without source confirmation (leads-vs-findings discipline). Reviews citing extraction errors without source-page reference are not actionable for authors. 10. Radiomics-reproducibility QC (if Phase 2C applied): If an acquisition-parameter sweep predicts an outcome from its own grid axes (R1 design-grid circularity) or the substantive result is a cross-domain failure framed as success (R3), confirm the recommendation reflects design-level severity and is not softened to a reporting fix. Where a model × threshold/cohort grid yields a few p < 0.05, confirm the multiplicity / expected-false-positive count is named (R4), not deferred to "statistical review needed." 11. Review-article QC (if Phase 2D applied): Confirm RV1–RV9 are reflected — in particular that novelty/value-add (RV1) is raised for a saturated topic and that gap-filling (RV8) is present, not just error-spotting. Verify SANRA is used as an appraisal aid, not over-enforced as a reporting guideline (no PRISMA demand on a narrative review; only RV3 is SANRA-aligned and phrased as a suggestion). Verify every suggested addition uses "consider adding" phrasing (no "must cite"), is source-confirmed, and that preprints are labeled as preprints (not equated with peer-reviewed guidelines). Confirm Phase 2F was run for the recommendation: when RV1 novelty is a Major in a saturated space with no distinct contribution, the recommendation is escalated toward Reject (the contribution IS the product — weak novelty is unfixable-in-current-form), not defaulted to the revision/Reconsider tier. 12. AI/method/review priority QC: Before a Major Revision (or Reconsider) recommendation, confirm Phase 2F was run. If novelty and clinical/research utility are both weak, the recommendation must reflect that contribution-level concern rather than treating all issues as fixable reporting defects. When fixable and unfixable defects coexist, confirm the unfixable class governs the tier, and that the Confidential Comments contain no Reject-grade language (including value-judgment deferral to the board) left inconsistent with a softer recommendation. 13. Observational-confounding QC (if Phase 2E applied): For any covariate imbalanced by exposure in Table 1 but absent from the adjustment set (O1), confirm the comment requests a concrete extended-adjustment sensitivity model, not a vague "adjust for more confounders." Confirm a selection/collider structure (O3) or an undisclosed complete-case collapse from a structural-zero dose covariate (O5) is raised at design-level severity, and that any E-value request (O6) targets the declared primary estimate rather than a supporting one. 14. Verify-your-own-criticism (all reviews): For each Major framed as a technical inaccuracy or a citation–claim mismatch, confirm the reviewer's own assertion was checked against a current authoritative source (full paper, CrossRef, arXiv). Downgrade unverified technical claims to a hedged "Please verify…"; keep confirmed ones firm. Watch for status drift (a "preprint" since published; a method since adapted) before asserting the manuscript is wrong. 15. Image-synthesis QC (if Phase 2K applied): Confirm the determinism/information-ceiling point (IS1) is raised whenever the manuscript reads a same-reader source→source+synthetic gain as added diagnostic information without a source→label baseline, that undescribed slice/mask provenance (IS2) is surfaced as a leakage/circularity concern rather than a reporting nicety, that quantitative agreement is checked at the lesion/target level not only globally (IS3), and that a biological-information claim built on image similarity alone is tempered (IS4). Per Phase 2F, confirm IS2/IS4 were treated as unfixable-in-current-form when present. 16. Reference-integrity QC (all original-research reviews): Confirm the load-bearing Introduction/Discussion citations (those used as evidence the method or premise works) were spot-checked — a cited paper doing a different task, a duplicate reference, or a wrong year/author is a Minor (Major if the premise rests on it), and any unconfirmed suspicion is phrased "please verify" rather than asserted.

Fix all issues found, then present to user.

Phase 5: Refinement

1. Present the draft to the user for review. 2. Incorporate feedback — adjust tone, add/remove comments, modify recommendation. 3. Generate {manuscript_id}_review_final.md — the polished version. 4. Generate {manuscript_id}_submission.md — formatted for copy-paste into editorial system:

• Strip markdown formatting for plain-text boxes
• Separate "Comments to Author" and "Confidential Comments to Editor"
• Include journal-specific score table if applicable

Phase 6: Pre-Submission QC

• [ ] No recommendation words in Comments to Authors
• [ ] All cited numbers match the manuscript
• [ ] Major comments ranked by impact (Task formulation flaw, if present, as Major #1)
• [ ] All suggestions feasible with existing data
• [ ] Author section word count measured (awk + wc), tier identified (Tier 1 ≤700w / Tier 2 700-1000w ★ default / Tier 3 1000-1400w); Tier 3 justified + ≤20% rolling frequency
• [ ] Reference-baseline ratio (wc / 545w) reported; ratio > 2.0 trimmed
• [ ] Hard cap 1400 words not exceeded
• [ ] AI pattern density within thresholds (em-dash ≤2/1000w; structural rule-of-three ≤2/Major; significance inflation 0/Major; hedged Minor ≥50%)
• [ ] Fatal flaw hierarchy stated in Confidential Comments (if applicable)
• [ ] Reject recommendations (if used): §1C condition checklist (design-level flaw + speculative practical value 3-trigger + novelty gap) explicitly verified — at least 2 of 3 conditions met
• [ ] AI/method/review Major Revision (or Reconsider) recommendations: Phase 2F contribution/value gate checked; weak novelty + weak utility not silently softened; for review articles, weak-novelty/no-distinct-contribution treated as unfixable-in-current-form (escalate toward Reject); unfixable defects govern tier over fixable list; confidential note carries no Reject-grade language left inconsistent with a softer recommendation

Tone and Calibration

• Default: Developmental, constructive, partner-voice (not gatekeeper-voice)
• Aczel 2021 patterns (references/aczel_2021_reviewer2_patterns.md): avoid attitude markers ("reject," "absurd," "oblivious"), boosters, personal attacks on authors, vague dismissals, and typo nitpicking; prefer first-person rapport ("I appreciate," "I stumbled over"), hedged suggestions ("I'd suggest," "could," "would help"), and critique aimed at the work rather than the people. Apply throughout drafting, not just QC.
• Escalate tone only when: clinical validity threatened, patient safety concern, severe data leakage, or reference standard fundamentally flawed
• Default recommendation: Major Revision (unless issues are purely reporting/clarity → Minor Revision)
• Fatal flaw signal: State in Confidential Comments which issue(s) represent fundamental design limitations, rather than recommending Reject directly
• Contribution/priority override: For original AI or method papers, a manuscript can be technically

analyzable and still below the journal's priority bar. When weak novelty and weak clinical/research utility both hold, surface that in Confidential Comments and calibrate the recommendation upward from the default Major Revision tier.

• Length proportionality: Minor Revision ≤ 600 words; Major Revision ≤ 1000 words. Length signals difficulty — a Minor Revision review longer than the manuscript itself reads as Reviewer 2.

Signature Review Patterns

Recurring high-yield checks — apply to every manuscript:

1. Patient-level data splitting: Splitting at patient level, not image/exam level 2. Confidence intervals: All primary metrics should have 95% CIs 3. Intended use statement: Clinical workflow position and decision influenced should be clear 4. Calibration: AUC alone insufficient for prediction models — calibration metrics needed 5. Overclaiming: Language should match evidence level (CI overlap, small test sets, single-center) 6. Reproducibility: Preprocessing, hyperparameters, segmentation protocols reported

For survival / prognostic-model manuscripts, also apply the Phase 2B 8-probe audit (conditioning, censoring, competing risks, cutoff optimism, comparator horizon alignment, C-index variant transparency, calibration beyond discrimination, estimand provenance).

For radiomic feature-reproducibility / phantom parameter-sweep / reliability-filtering manuscripts, also apply the Phase 2C 4-probe audit (design-grid circularity, construct validity / proxy-target gap, transportability framing with Reject-escalate calibration, multiplicity).

For Review / narrative / primer / state-of-the-art manuscripts, apply the Phase 2D 9-probe audit (novelty/value-add, scope/aims, evidence-gathering transparency, technical/medical accuracy, taxonomy/synthesis coherence, balance/currency/citation accuracy, load-bearing figures/tables, constructive gap-filling, curated-base circularity) in place of the original-research probes — error-spotting plus proportionate gap-filling, with SANRA used as an appraisal aid only.

For observational studies whose central claim is an adjusted exposure–outcome association, also apply the Phase 2E 12-probe audit (confounding completeness, adjustment-set provenance, selection/collider bias, exposure measurement validity, missing-data / complete-case collapse, residual-confounding E-value, over-adjustment, analysis-unit/clustering, outcome construct validity, overlapping-subset gradient, complex-survey design & weighting, data-driven threshold mining, cross-sectional mediation, interaction scale), with O1 (a measured covariate imbalanced by exposure in Table 1 yet absent from the adjustment set) and O7 (an outcome consequence/mediator wrongly adjusted) checked against the manuscript's own Table 1.

For cross-modality image-synthesis manuscripts (MRI→PET / MRI→CT / non-contrast→contrast / low-dose→full-dose) that claim functional/molecular information or a substitute for the unavailable target modality, also apply the Phase 2K 4-probe audit (IS1 determinism/information-ceiling vs a source→label baseline, IS2 target-derived-preprocessing/slice-selection leakage, IS3 global vs lesion-level quantitative agreement, IS4 mechanistic/proxy-signal plausibility); IS2 and IS4 are typically unfixable-in-current-form and govern the recommendation per Phase 2F.

Journal-Specific Formatting

Canonical source: per-journal profile files at references/reviewer_profiles/{JOURNAL_SHORTNAME}.md

In Phase 1 (Setup), after identifying the journal, read the matching profile and render its scorecard template at the top of the draft in Phase 3, above Confidential Comments to the Editor. This avoids duplicating journal form fields across multiple skills.

Current profiles:

Custom Journal

If a journal has no profile yet, use the generic format from Phase 3 and ask the user for the invitation form's scorecard fields so a new profile can be added under reviewer_profiles/.

Output Contract

Skill Interactions

What This Skill Does NOT Do

• Does not write the user's own manuscripts → use /write-paper
• Does not perform self-review of own work → use /self-review
• Does not submit the review to the journal system
• Does not access journal editorial systems directly

Anti-Hallucination

• Never fabricate manuscript content. All cited numbers, methods, and findings must come from the actual manuscript.
• Never invent journal scoring criteria. If uncertain about a journal's format, ask the user or use the generic format.
• Never generate references from memory. Use /search-lit if citations are needed for reviewer comments.
• If a reporting guideline item is uncertain, flag it as [CHECK] rather than asserting compliance.

Anti–"Reviewer 2" Tone Patterns

Source: Aczel B, Szaszi B, Holcombe AO. Don't be reviewer 2! Reflections on writing effective peer review comments. Research Integrity and Peer Review. 2021;6:13. PMC8505560.

This reference codifies the linguistic patterns to avoid (and the partner-voice patterns to use) when drafting peer reviews. Apply during Phase 3 drafting and Phase 4 Self-QC.

Avoid (the "Reviewer 2" signals)

Prefer (the partner-voice signals)

Worked transformations (Aczel verbatim examples + skill-specific)

From the paper

Skill-specific transformations (medical imaging context)

Phase 4 Self-QC additions

Add to the existing Pre-Submission QC checklist:

• [ ] Attitude marker scan: 0 instances of "reject," "absurd," "ridiculous," "naive," "oblivious," "fail," "wrong"
• [ ] Personal attack scan: No "the authors seem...," "the authors do not understand," "the authors are unaware"
• [ ] First-person rapport present: At least 2 instances of "I" in General Comments / Closing Remark (not in attitude-marker contexts)
• [ ] Hedged language ratio: At least 50% of Minor Comment requests use hedged forms ("I'd suggest," "could," "would help") rather than imperative ("must," "Please [verb]")
• [ ] Balance check: General Comments names ≥2 specific strengths before listing concerns
• [ ] Length proportionality: Minor Revision ≤ 600 words total; Major Revision ≤ 1000 words total
• [ ] Typo nitpicking limit: At most 1 grammar/typo Minor Comment, only if in formal section (Acknowledgements, Declarations) or repeated systematically

When to escalate tone (override partner voice)

Aczel's framework assumes the manuscript is fundamentally sound but needs revision. Escalate to firmer (still professional) language only when:

• Patient safety concern (dose, drug error, PHI leak)
• Severe data leakage (training-test contamination, label leakage)
• Reference standard fundamentally invalid (no ground truth)
• Citation fabrication or plagiarism suspected
• Author conflict of interest undeclared

Even at escalation, retain:

• Specific evidence (page/line)
• Hedged accusation form ("the data appear to suggest...")
• Confidential Comments to Editor for the gravest concerns

Cross-references

• Skill: peer-review/SKILL.md Phase 3 (Draft) and Phase 4 (Self-QC)
• Companion: peer-review/references/reviewer_profiles/{JOURNAL}.md for journal-specific scorecard
• Related: ~/.claude/rules/writing-style.md (active voice, no "we believe/think")

Citation

Aczel B, Szaszi B, Holcombe AO. Don't be reviewer 2! Reflections on writing effective peer review comments. Research Integrity and Peer Review. 2021 Oct 11;6(1):13. doi:10.1186/s41073-021-00117-3. PMCID: PMC8505560.

<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review. Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal recommendation. Each consuming skill maps findings to its own output:

• peer-review: Major / Minor comments + Confidential Comments to the Editor; a task- or

design-level flaw is placed as Major #1.

• self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.

Do NOT edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync. -->

AI / ML overclaiming probes (AO0–AO5)

A 5-probe checklist (AO1–AO5, with AO0 as a gate) for medical-AI/ML primary studies (diagnostic, prognostic, triage, detection) where the conclusion's reach exceeds the evidence. These probes complement (do not replace) the generic Phase 2 issue checklist and the signature "Overclaiming vs evidence level" check. The aim is to keep a framing-level over-reach from passing as a wording nitpick: a paper can report sound metrics yet draw a clinical claim — generalizable, outperforms clinicians, deployment-ready — that the design does not support, and that claim is what a reader carries away. AO1–AO4 target over-reach in the claim sentences; AO5 targets over-reach baked into the reported metric itself (an optimistically- or unreproducibly-reported number that makes the result look stronger than a faithful estimate). Run AO0 first.

AO0 — Locate the strongest claim, then its support (run before AO1; gates any over-reach finding):

• Identify the load-bearing claims in the Title, Abstract, and Conclusion (the sentences a reader quotes). For each, find the specific evidence cited (which dataset, which comparison, which metric + uncertainty).
• An over-reach finding is a lead until the claim and its support are read together against the manuscript — do not strawman a stray adjective. Escalate only when a headline claim genuinely outruns the cited evidence.
• If the claim is already appropriately hedged to the evidence, record "claim matched to evidence" and move on.

AO1 — Generalizability claimed from limited external validation:

• Does the Abstract/Conclusion assert the model "generalizes," is "transferable/robust across settings," or is suitable for broad populations, while external validation is a single site / single scanner-vendor / single source (or absent)?
• Sub-check: is the external set demographically narrow (single ethnicity, single sex-dominant, narrow age) relative to the population the claim names?
• If the generalizability claim outruns the external evidence → recommend softening to the evidence ("validated at one external site") and moving multi-setting generalizability to a stated limitation + next step. MAJOR candidate when it is a headline claim; MINOR when it is a single qualifier in the Discussion.

AO2 — Superiority language against overlapping or under-powered comparison:

• Flag "outperforms", "superior to", "beats", "can replace [clinician/radiologist]" when (a) the model vs comparator 95% CIs overlap, (b) no test of the difference is reported (two separate AUCs are not a comparison), or (c) the comparison rests on a small test set / few readers.
• Ask for the difference in the metric with its CI and a paired test of that difference, not two standalone estimates.
• If the difference is not statistically supported → recommend reframing from "outperforms" to "comparable to" (still a meaningful result). MAJOR when a superiority/replacement claim is the headline; otherwise MINOR.

AO3 — Comparison-frame mismatch (model task ≠ human task):

• When a model-vs-clinician comparison drives a claim, verify the two performed the same task on the same inputs under the same constraints: same images/inputs available, same time budget, same question asked, same decision point.
• Common mismatches: the model sees a curated single view while readers see the full study; readers are timed or work from a different modality; the "reader" benchmark is a literature value on a different cohort.
• A mismatch makes "outperforms clinicians" non-interpretable as a clinical claim → ask the authors to state exactly which task the comparison establishes, or to align the conditions. MAJOR candidate when it underpins a headline.

AO4 — Deployment / clinical-readiness claim from retrospective internal evidence:

• Flag "ready for clinical deployment", "can be used to triage/guide treatment", "will reduce workload/cost", or a recommended decision threshold, when the evidence is a retrospective, internally-split (or even external but observational) accuracy study with no prospective, silent-trial, or decision-impact data and (often) no calibration or decision-curve analysis.
• Discrimination on retrospective data does not establish that acting on the model helps patients; a probability that drives a decision must also be calibrated, and net benefit must be shown.
• Recommend reframing deployment/utility language to "supports further prospective evaluation", and (where a threshold is proposed) adding calibration + decision-curve evidence. MAJOR when a deployment/care-directive claim is made; MINOR when only a hedged "potential utility" sentence.

AO5 — Optimistic / non-reproducible performance reporting (the metric overstates the evidence):

• Triggered when a primary AI/ML study reports discrimination or accuracy as a headline result. Check how the number was produced before taking it at face value.
• (a) Best-fold vs cross-fold: is the headline metric the single best cross-validation fold (or one favorable train/test split) rather than the mean across folds with its SD or a 95% CI? Best-fold figures are upward-biased; ask for the cross-fold mean ± SD (or CI).
• (b) Operating point: are sensitivity/specificity/F1 reported without stating the decision threshold and how it was chosen? A threshold tuned on the test set inflates the metric; ask for the operating point and that it was selected on training folds only (e.g., Youden's J on the training data). This probe is about whether the reported number is reproducible/optimistic, not whether the threshold is clinically deployable (that is AO4).
• (c) Prevalence-representative metrics: was training/evaluation done on an artificially balanced set while the real prevalence is much lower, with accuracy quoted as the headline? Accuracy — and PPV/NPV — on a re-balanced set do not transfer to the deployment base rate (PPV/NPV are prevalence-dependent). Ask for the evaluation class distribution, threshold-independent discrimination (AUROC, and AUPRC under imbalance, with CIs), sensitivity / false-negative rate at the stated threshold, and PPV / NPV estimated on a prevalence-representative holdout (or modelled for the target clinical prevalence, with uncertainty).
• (d) Code-vs-claims fidelity: where code is released, does the described tuning/metric match it? Common mismatches: a claimed hyperparameter search the code does not run; a metric (e.g., specificity) attributed to a library function that does not compute it. A confirmed mismatch is an integrity/reproducibility flag — verify against the released code before asserting it.
• Severity: MAJOR when the load-bearing performance claim rests on a best-fold number, an unstated/test-tuned threshold, a rebalanced-accuracy headline, or a code-vs-claims mismatch (the reported result is optimistic or not reproducible); MINOR when cross-validation was sound and only the cross-fold summary, the operating point, or a class-aware metric is missing from the write-up.

Decision-impact / early-deployment probes (DECIDE-AI axis, DI1–DI5)

Co-apply when a study claims clinical utility, deployment, or decision impact of an AI system, or is an early-stage live clinical evaluation. The reporting axis is then DECIDE-AI (early-stage clinical evaluation of AI decision-support); these probes check that a utility/deployment claim rests on real-use evidence, not retrospective accuracy. They sharpen AO4 for the deployment-evaluation case.

DI1 — Live/prospective evidence vs retrospective accuracy:

• Is there any prospective, in-workflow (silent-trial / shadow-deployment / live) evidence of how the system performs and is used, or is the deployment/utility claim built only on a retrospective, internally-split accuracy study? A deployment claim from retrospective discrimination alone → MAJOR (reframe to "supports prospective evaluation").

DI2 — Intended use + deployment pathway within early-stage limits:

• Is the intended use (condition, decision supported, users, setting) stated, and is the claim kept within what an early-stage evaluation can show? Over-reach to "ready for routine clinical use" from a developmental/exploratory study → MAJOR.

DI3 — Decision threshold + calibration/utility:

• If a decision threshold or care directive is proposed, is it justified, with the predicted probabilities calibrated and net benefit (decision-curve) shown — not discrimination alone? An unjustified threshold or missing calibration/utility for a probability that drives a decision → MAJOR.

DI4 — Workflow integration + human–computer interaction:

• Does the study report how the system was used in the real workflow — user adherence/exposure, human–computer agreement/override rates, usability and learning curve (human factors)? A utility claim with no real-use interaction data is the central DECIDE-AI gap → MAJOR/PARTIAL.

DI5 — Safety, error capture, and subgroup safety:

• Were significant errors/malfunctions pre-defined and captured, the safety profile reported and discussed, and performance/safety examined across relevant subgroups (fairness)? Absent pre-defined error capture or subgroup-safety assessment where the claim is clinical deployment → MAJOR.

Output template (AO1 example):

"The Conclusion states the model 'generalizes across institutions,' but external validation appears limited to a single site ([Methods, External validation]). I'd suggest softening this to the evidence — e.g., 'validated at one external site' — and framing multi-institution generalizability as a stated limitation and a next step. If a broader claim is intended, an external set spanning multiple sites/vendors would be needed to support it."

Output template (AO2 / AO3 example):

"The 'outperforms radiologists' claim rests on a comparison whose 95% CIs for model and reader [metric] overlap ([Figure/Table]), and no test of the difference is reported; the reader task also differs from the model's in [inputs/time] ([Methods/Table]). I'd suggest (a) reporting the difference in [metric] with its CI and a paired test rather than two separate estimates, and (b) stating explicitly which clinical task the comparison establishes. If the difference is not statistically supported, reframing from 'outperforms' to 'comparable to' would be both defensible and still a meaningful result."

Output template (AO5 example):

"Table 2 appears to report the single best cross-validation fold; because best-fold figures are optimistically biased, I'd suggest reporting the mean across folds with its SD (or a 95% CI). Relatedly, the sensitivity/specificity depend on a classification threshold I could not find stated — reporting the operating point and how it was chosen (e.g., Youden's J on the training folds only, to avoid tuning on the test data) would make these numbers reproducible. Finally, since the training set was balanced to 1:1 while the clinical prevalence is much lower, accuracy (and PPV/NPV) on a re-balanced set may not reflect performance at the true base rate; stating the evaluation class distribution and adding AUROC/AUPRC with CIs alongside sensitivity/FNR at the stated threshold, with PPV/NPV on a prevalence-representative holdout, would make the clinical cost of a miss visible."

Discipline — leads vs findings (applies to AO0–AO5):

• A claim-vs-evidence mismatch surfaced by a quick scan is a lead, not a finding, until the claim sentence and its cited support are read together against the manuscript. Do not escalate a hedged Discussion qualifier as if it were a headline.
• Anchor every over-reach comment to the exact claim location and the exact evidence (dataset, comparison, metric + CI). A comment that names the location and the gap is actionable; "the authors overclaim" is not.
• Keep severity tied to where the claim sits and what it drives: a headline/clinical-action claim that outruns the design is design-/framing-level (MAJOR, often Major #1); a stray adjective is MINOR.

<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review. Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal recommendation. Each consuming skill maps findings to its own output:

• peer-review: Major / Minor comments + Confidential Comments to the Editor; a design-level

flaw (missing consent, identifiable patient data, or causal overclaim) is placed as Major #1.

• self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.

Do NOT edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync. -->

Case-report probes (CR1–CR9)

A checklist for case reports and small single-patient clinical narratives. These probes complement (do not replace) the generic Phase 2 issue checklist and the CARE items. They target the places reviewers most often challenge case reports: why the case is publishable, whether the patient is protected, whether the story is chronologically complete, and whether the Discussion stays inside the evidence level of n=1.

CR1 — Novelty / teaching-value justification:

• Does the manuscript state why this case is worth publishing: rare presentation, diagnostic trap,

management complication, unusual imaging/pathology correlation, unexpected response, or a practical bedside lesson?

• Is the claim anchored to a transparent literature boundary or epidemiologic context? "Rare" without

a search boundary, comparator cases, or a clinically actionable lesson is a MAJOR priority concern, because the paper's contribution is the case's teaching value.

CR2 — Consent, patient protection, and image de-identification:

• Is written informed consent documented, or is a waiver/IRB basis stated when appropriate? Are

patient images, dates, locations, institution names, initials, faces, scan metadata, and unusual demographic details removed or justified?

• Missing consent language or identifiable images/data is a MAJOR ethical/reporting issue. If the

manuscript includes potentially identifying details, treat it as a finding even when the prose says "de-identified."

CR3 — Causal attribution discipline:

• Does the Discussion distinguish temporal association from causation? A single case can suggest a

hypothesis or illustrate a plausible adverse event; it cannot establish incidence, efficacy, safety, risk, or mechanism.

• Escalate when Title/Abstract/Conclusion claims that an intervention "caused", "prevented",

"proved", "was effective", or "should change practice" without stronger evidence. The fix is to reframe as "was temporally associated with", "is consistent with", "highlights", or "should prompt consideration."

CR4 — Similar-case comparison and literature absence handling:

• Does the manuscript compare the case with the nearest prior cases, including similarities and

differences in presentation, diagnosis, intervention, and outcome? If five or more similar cases are found, a compact comparison table is usually clearer than prose.

• "First case", "only case", or "no prior reports" requires a documented search boundary. Without

it, ask the authors to soften the claim or state the search strategy; do not let an unverified priority claim carry the Abstract.

CR5 — CARE narrative completeness:

• Can the reader reconstruct Patient Information -> Clinical Findings -> Timeline -> Diagnostic

Assessment -> Therapeutic Intervention -> Follow-up and Outcomes? Is the timeline figure/table present when the course is multi-step, and does it include the final follow-up interval?

• Missing timeline, follow-up duration, diagnostic reasoning, or outcome assessment is usually a

Fixable Major for a case report because the narrative is the evidence.

CR6 — Generalizability and teaching-point framing:

• Does the conclusion state a narrow learning point rather than a population-level recommendation?

Strong case reports teach clinicians what to consider, monitor, or report; they do not infer prevalence, comparative effectiveness, or standard-of-care changes.

• Check the Abstract and final paragraph together. A cautious Discussion cannot rescue an

overgeneralized Abstract conclusion.

CR7 — Adverse drug/device/contrast reaction: causality discipline (apply when the case is the adverse event):

• Is attribution supported by a named instrument (Naranjo or WHO-UMC for drugs) with the score and

tier reported, not just the word "caused"? Is dechallenge documented (withdrawal → resolution) with the exposure-to-onset latency, and is rechallenge either reported or explicitly withheld on safety grounds? Mechanism-only narrative with no instrument and no dechallenge → MAJOR for the attribution claim.

• Are alternative causes excluded, and is the event located against a denominator (institutional

rate or pharmacovigilance database count) rather than implying incidence from n=1? Severity/ preventability instruments (e.g., Hartwig–Siegel, Schumock–Thornton) strengthen a safety report. Blaming the agent for a course actually driven by a downstream confounder (e.g., patient self-management) without separating the steps → overreach.

CR8 — Case-series-specific design (apply only when the manuscript reports a series, n≥2):

• Does the series have cohort-style methods (design, setting, case-identification source,

eligibility, protocol) and an all-cases summary table, rather than being N stacked single reports? Absent methods/summary table for a series → MAJOR (it cannot be read as a series).

• Is selection/ascertainment stated, with the screened pool size so consecutiveness can be

judged? Does the manuscript avoid inferring prevalence/proportion or effectiveness from a selected/referral series (report counts, not rates)? A rate claimed from a referral series, or a series presented with no selection description, → MAJOR interpretive limit. Cohort-level limitations (small n, retrospective, non-uniform protocol, no comparator) should be explicit.

CR9 — Imaging-led (radiology / nuclear medicine / interventional) case report (apply when the contribution is the image or an image-guided procedure):

• Is each modality described as technique → findings → impression, in clinical order, with

reproducible technique (sequence/phase, field strength, contrast agent/dose/rate, CT kV/keV/CTDIvol, PET tracer dose/uptake time, transducer frequency; for IR, devices with sizes and the step sequence)? Modalities merged into one undifferentiated paragraph, or findings stated without the acquisition needed to reproduce them, → reporting gap.

• When a structured-reporting system applies (BI-RADS/LI-RADS/PI-RADS/TI-RADS/Lung-RADS/O-RADS),

is the category given with its meaning/risk rather than a bare number? Are quantitative values reported with method (ROI placement) and threshold honesty — a value with no validated cutoff labeled exploratory, not diagnostic?

• For multimodality discordance, is the disagreement stated and resolved (decisive modality or

histopathology), and is a missing standard modality named as a limitation? For an IR case, is complication latency and the diagnostic→therapeutic pathway documented with pre/post outcome?

• Patient-protection at the image level: are images de-identified at the DICOM level (no burned-in

annotations, accession numbers, dates, faces)? Is figure alt text real (not a placeholder)? Is a device/vendor relationship disclosed for an advanced-technique or device case? Identifiable images or undisclosed device-vendor COI → MAJOR.

Output template (CR2 / CR3 example):

"The case is clinically interesting, but the submission does not yet provide enough information for

publication as a case report. First, the consent/anonymization statement should be made explicit,

including whether the accompanying images have been stripped of identifiers. Second, the Discussion

currently treats the temporal improvement after treatment as evidence that the intervention was

effective. Because this is a single case, I would suggest reframing the conclusion as a hypothesis-

generating observation and narrowing the teaching point to when clinicians should consider this

diagnosis or management option."

Discipline — leads vs findings (applies to CR1–CR9):

• A missing element is a **lead until the whole manuscript, figures, legends, and ethics statements are

checked**. Do not allege absent consent or absent follow-up if it appears in a title page, figure legend, supplement, or patient-perspective section.

• Anchor every comment to what the flaw changes: patient protection (CR2), evidentiary overreach

(CR3/CR6), contribution/priority (CR1/CR4), or narrative completeness (CR5). Avoid asking for broad extra literature unless it directly supports the case's teaching value.

<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review. Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal recommendation. Each consuming skill maps findings to its own output:

• peer-review: Major / Minor comments + Confidential Comments to the Editor; a task- or

design-level flaw is placed as Major #1.

• self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.

Do NOT edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync. -->

Clinical Prediction-Model probes (CP1–CP4)

A 4-probe checklist for cross-sectional / observational clinical prediction models — a binary or multinomial outcome predicted from a covariate set, reported under TRIPOD / TRIPOD+AI, usually as a nested predictor-set comparison ("does adding marker X to a base model improve prediction of Y?"). This is the diagnostic/prognostic prediction counterpart to survival_prognostic.md (time-to-event) and complements the observational_confounding.md probes (a prediction model still has an analysis unit (O8) and can be over-adjusted (O7)). Route here when the manuscript develops or compares prediction models and reports discrimination (AUC/C-statistic), calibration, or decision-curve / net-benefit analysis.

CP1 — Apparent vs optimism-corrected performance:

• Discrimination, calibration (slope / intercept / calibration plot), and decision-curve / net-benefit all suffer optimism when computed in the same sample used to fit the model. A manuscript that optimism-corrects only the C-statistic (bootstrap / cross-validation) but reports calibration and net benefit in-sample has corrected one axis and left two apparent.
• An apparent calibration slope near unity (≈ 1.0) does not establish out-of-sample calibration — it is the expected in-sample result and says little about transportability.
• Calibration and DCA computed in-sample must be labelled "apparent", or the optimism correction must be extended to them (e.g. bootstrap the calibration slope/intercept and the net-benefit curves). In-sample calibration/DCA described as "well calibrated" / "clinically useful" without the apparent caveat → MAJOR.

CP2 — The two-null distinction (incremental value vs marginal effect):

• A "negative" prediction study conflates two different nulls that have different evidentiary status:
• Incremental value negligible — a well-powered statement: ΔAUC ≈ 0, optimism correction does not favour the larger model (and may reverse sign toward the parsimonious one), net benefit does not exceed the base model. This is a genuine "adding X does not improve prediction."
• Marginal predictor effect null — an underpowered statement: the predictor's adjusted OR/coefficient has a confidence interval that still admits a clinically relevant effect (e.g. OR up to ~1.7). This is "we could not exclude an effect," not "there is no effect."
• A manuscript that collapses both into a blanket "X did not predict Y" overstates the marginal-effect arm. Flag when a negative conclusion rests on a non-significant marginal OR whose CI admits a relevant effect, without separating it from the (better-powered) incremental-value finding → MAJOR. The honest reframe reports both explicitly.

CP3 — Events-per-variable (EPV) per nested model:

• Report EPV for each nested model, not just the full cohort. A model with 6 covariates and 60 events (EPV 10) is adequately supported; the same 6 covariates in an age≥60 subgroup with 13 events (EPV ≈ 2) is separation-prone and unstable.
• Penalization (Firth / ridge / LASSO) is justified and should be stated when EPV < 10; subgroup or interaction models with EPV in the low single digits are separation-affected and must be descriptive-only, not presented as estimated effects.
• Subgroup/interaction effect estimates from a model with EPV ≲ 5, presented as if reliably estimated, or no EPV disclosure for the nested models → MAJOR (subgroup) / MINOR (EPV simply not reported but adequate).

CP4 — Net benefit is a model comparison, not a policy endorsement:

• A decision curve showing the model exceeds treat-all / treat-none across a threshold range is a model-vs-default statement about ranking, not evidence that the underlying imaging / test / screening strategy should be adopted in practice. Net benefit is computed on the same outcomes and says nothing about cost, downstream harm, or whether the test should be ordered.
• A conclusion that reads decision-curve superiority as an endorsement of the clinical strategy ("supports using X for screening / triage") → MAJOR (scope; cross-link scope-coherence-gate.md). Reframe to the model-comparison claim the DCA actually supports.

CP5 — Intended-use horizon leakage (claim-timepoint vs feature availability):

• Read the intended-use horizon off the title / abstract / aim — the adjectives preoperative, pretreatment, baseline, screening, triage, pre-procedure assert a decision made before some event X. Then check each predictor's availability timepoint: any feature that exists only after X (post-procedure pathology, treatment-response variables, post-contrast-only measures in a "pre-contrast screening" claim, follow-up-derived labels) is leakage relative to the claimed use, even if it is not train/test leakage.
• This is a claim × feature-availability mismatch: the model may be statistically clean yet unusable at the moment its claim targets. Distinct from ordinary data leakage (which is about train/test contamination) — this keys on the decision horizon.
• Flag the specific offending predictor(s) and either narrow the intended-use claim to the horizon at which all inputs exist, or drop the post-horizon features and re-fit. → MAJOR when a post-horizon variable supports a pre-horizon decision claim.

CP6 — Validation-nomenclature conflation (development step vs test step):

• Distinguish the development step (including any internal cross-validation or bootstrap optimism correction) from a held-out / external test step. Flag phrasing that merges them: "developed with external validation" where a single external set is used as development data; "trained and validated on the external cohort"; an "external validation set" that was used for tuning/feature selection.
• Cross-validation and bootstrapping are development-time optimism corrections, not external validation; calling them "validation" overclaims out-of-sample evidence. Require an explicit statement of which data trained, which tuned, and which was touched only once for the final estimate.
• → MAJOR when the overclaim props up a generalisability/deployment-readiness conclusion; a clarify-request when the data flow is actually clean but the wording is loose.

Output template (CP1 / CP2 example):

"Discrimination is bootstrap optimism-corrected, but the calibration slope (0.99) and the net-benefit curves are computed in the development sample and described as 'well calibrated' and 'clinically useful' without the apparent caveat — an in-sample slope near 1.0 is expected and does not establish out-of-sample calibration. I'd suggest labelling the calibration and decision-curve results 'apparent' or extending the optimism correction to them. Relatedly, the Conclusion states that the marker 'did not predict' the outcome, but two distinct results are merged: the incremental-value analysis is well powered (ΔAUC ≈ 0, optimism correction favours the parsimonious model), whereas the marginal adjusted OR has a confidence interval that still admits an effect up to ~1.7 (underpowered). I'd report these separately — 'adding the marker did not improve prediction' is supported; 'the marker has no association' is not."

<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review. Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal recommendation. Each consuming skill maps findings to its own output:

• peer-review: Major / Minor comments + Confidential Comments to the Editor; a design-level

flaw (verification bias, two-gate sampling, unblinded reference) is placed as Major #1.

• self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.

Do NOT edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync. -->

Diagnostic-accuracy / reader-study probes (D1–D6)

A checklist for diagnostic test accuracy (DTA) primary studies — an index test against a reference standard, including multi-reader multi-case (MRMC) reader studies (e.g., AI-vs-reader or modality-comparison). These probes complement (do not replace) the generic Phase 2 issue checklist and the STARD / QUADAS-2 items; they target the biases QUADAS-2 names and the MRMC design/variance issues a reader study adds. Pairs the analyze-stats table-standards/table-types/reader_study.md table and the make-figures exemplar_plots/mrmc_roc.md figure. (For a DTA meta-analysis, use sr_ma.md.)

D1 — Reference standard validity + verification bias:

• Is the reference standard appropriate and applied to (essentially) all participants? Partial verification (only test-positives get the gold standard), differential verification (different reference standards by index result), and incorporation bias (the index test is part of the reference) each inflate accuracy. QUADAS-2 Domain 3/4.
• Was the reference standard interpreted without knowledge of the index result, and was the time interval between index and reference short enough that the target condition did not change? A long or undefined interval, or a reference that incorporates the index, → MAJOR.

D2 — Spectrum + sampling design:

• How were participants sampled — a single-gate consecutive/random series of those with the target condition suspected (preserves spectrum and prevalence), or a two-gate case-control design (known cases vs healthy controls)? Two-gate sampling overestimates accuracy and distorts spectrum; it is a MAJOR interpretive limit when the headline is clinical accuracy, not just proof-of-concept.
• Is the study prospective with a pre-specified index threshold, or retrospective with a threshold chosen on the same data (D-linked to optimism)? Spectrum (disease severity, comorbid mimics) should match the intended-use population.

D3 — Blinding of index and reference interpretation:

• Was the index test interpreted blind to the reference standard and to clinical information that would not be available at the point of use, and vice versa? Unblinded interpretation (review bias) inflates agreement.
• For an AI index test, was the operating threshold pre-specified (not tuned on the test set)? A test-set-tuned threshold reported as the result is an optimistic-validation finding (pair with exemplar_reviews/optimistic_validation_reporting.md).

D4 — Indeterminate / uninterpretable results:

• How were indeterminate, uninterpretable, or intermediate index results handled — excluded, or counted (intention-to-diagnose)? Silently dropping them inflates accuracy. State the rule and report an intention-to-diagnose sensitivity analysis (non-diagnostic counted as wrong). Undisclosed exclusion of indeterminates → MAJOR.

D5 — MRMC reader-study design:

• Is the design fully crossed (every reader reads every case in every modality) or a nested/split-plot variant — and is that stated? Was reading order randomized and a washout interval used between modalities so a case is not recognised from a prior read? Absent washout/order control in a within-reader modality comparison → MAJOR (memory/recognition bias).
• Are the readers sampled to generalise to the intended reader population (number, expertise mix), and were training/instructions and the information available (priors, clinical data) standardised? A 2–3 expert-reader convenience panel cannot support a population-level "AI matches radiologists" claim — flag the generalisation gap.

D6 — MRMC analysis + estimand:

• Does the analysis account for both reader and case variability (Obuchowski–Rockette / Dorfman–Berbaum–Metz or an equivalent multi-reader model), or does it treat readers as fixed / pool reads as if independent? Ignoring reader variance understates uncertainty and is a MAJOR statistical flaw for a generalising claim.
• Is the estimand clear: reader-averaged vs a fixed specific reader; per-patient vs per-lesion (clustered) unit; superiority vs non-inferiority with a pre-specified margin? Are per-reader results shown alongside the reader-averaged estimate (a single averaged AUC can hide one weak reader)? A non-inferiority claim with no pre-specified margin, or a clustered design analysed as independent, → MAJOR.

D7 — Index-test-as-enrollment-criterion circularity:

• Cross-check the inclusion criteria against the index / proposed test. When a study proposes or validates a diagnostic or classification instrument and uses a threshold of that same instrument (or a component of it) as an enrollment criterion, the validation is circular: the spectrum is built into the design, and sensitivity/specificity (or the instrument's apparent discrimination) are inflated by construction.
• Typical signatures: "patients were included if [index score] ≥ T" in a paper whose aim is to evaluate that index; enrolling on a positive screening test to then "validate" the screening test; defining the diseased group by the same reader/algorithm output under study.
• This is design-level, not a reporting fix — escalate past an ordinary Major (a co-reviewer / editor reads it as a fatal selection/spectrum artifact). The fix is a reference standard and an enrollment criterion that are independent of the index test (a consecutive suspected-disease series), not a re-analysis.

Output template (D2 / D6 example):

"The study uses a case-control (two-gate) design — confirmed cases versus healthy controls — rather than a consecutive series of patients in whom the diagnosis was suspected. This typically overestimates accuracy and does not reflect the intended-use spectrum, so I'd read the reported sensitivity/specificity as proof-of-concept rather than clinical accuracy, and suggest tempering the Abstract accordingly. Separately, the reader study reports a single reader-averaged AUC; because readers are a sample, I'd suggest an MRMC analysis (e.g., Obuchowski–Rockette) that accounts for both reader and case variance, with per-reader estimates shown and the unit of analysis (per-patient vs per-lesion) stated."

Discipline — leads vs findings (applies to D1–D6):

• A verification/blinding/spectrum concern from a quick scan is a lead until Methods and the participant flow are read together — distinguish under-reporting (ask to clarify) from a true design bias (MAJOR).
• Anchor each comment to the exact bias (partial vs differential verification; single- vs two-gate; reader-averaged vs fixed-reader; per-patient vs per-lesion) and the location. Keep severity tied to what the flaw does: two-gate sampling, incorporation bias, or ignoring reader variance is design/analysis-level (MAJOR, often Major #1); an unreported reading-order detail is a clarify-request.

<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review. Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal recommendation. Each consuming skill maps findings to its own output:

• peer-review: Major / Minor comments + Confidential Comments to the Editor; a task- or

design-level flaw is placed as Major #1.

• self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.

Do NOT edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync. -->

Fairness / Equity / Subgroup-performance probes (EQ0–EQ6)

A 7-probe checklist for AI/ML, prediction, or diagnostic studies that make (or imply) a claim about performance across a heterogeneous population — generalizable / deployment-ready / "works for patients" claims, or studies that present subgroup analyses as evidence of fairness or equity. These probes complement (do not replace) the generic Phase 2 issue checklist, the AI-overclaiming probes (ai_overclaiming.md, with which EQ co-applies), the prediction-model probes (clinical_prediction_model.md, whose EPV logic EQ5 reuses), and the reporting items in TRIPOD+AI, DECIDE-AI, and CONSORT-AI. They target one recurring failure: an aggregate performance number, or an eyeballed "similar across groups" statement, presented as evidence that a model is equitable or broadly deployable.

EQ0 — Applicability gate (apply only when a cross-population claim is made):

• Apply EQ1–EQ6 when the manuscript claims (or its framing implies) that the model/score/test

performs adequately across a heterogeneous population, or reports subgroup analyses as a fairness/equity argument.

• Do not fire these probes on a study that explicitly **scopes its claim to a single,

well-defined population** and does not generalise beyond it — there, the right probe is scope coherence (the conclusion must stay within the evaluated population), not a fairness audit.

EQ1 — Disaggregated subgroup performance, not aggregate-only:

• An overall AUC / sensitivity / specificity / calibration is not evidence of per-subgroup

performance — it can hide large, offsetting subgroup gaps. A broad-applicability or deployment claim requires disaggregated metrics, with confidence intervals, for pre-specified subgroups defined by clinically and (where collected and appropriate) socially relevant attributes (e.g. age band, sex, acquisition site / scanner / vendor, and protected attributes when available).

• Reporting only the pooled metric while claiming the model works "across patients" / "in practice"

→ MAJOR. Reporting per-subgroup point estimates without CIs, so the reader cannot tell a real gap from noise → MINOR.

EQ2 — Error-rate parity ≠ discrimination parity; base-rate dependence:

• Equal AUC across groups does not imply equal sensitivity/specificity at the operating

threshold, nor equal PPV/NPV — and a single threshold applied across subgroups with different outcome prevalence yields different error trade-offs by construction. "Discrimination is similar, therefore the model is fair" conflates two different fairness criteria.

• The probe asks for the metric that drives the deployment harm (often the subgroup

false-negative rate, or PPV at the decision threshold), not just AUC parity. A fairness claim resting on AUC parity alone, with the threshold-dependent error rates unreported or unequal → MAJOR.

EQ3 — A fairness claim needs a named estimand + a gap test, not eyeballed point estimates:

• "Performance was similar across groups" is an estimand-free assertion. If fairness is claimed, the

manuscript must (a) name the fairness criterion — error-rate parity / calibration-within-groups / equalized odds — recognising that these can mathematically conflict and cannot all hold at once when base rates differ; (b) report the between-group gap with a confidence interval and a test, not overlapping point estimates; and (c) ideally pre-specify it.

• A parity conclusion read off non-overlapping-looking point estimates with no gap CI / test →

MAJOR (eyeballed/data-mined parity). A reported gap that was not pre-specified and is selected post hoc among several metrics → MINOR (disclose as exploratory).

EQ4 — Development-cohort representativeness vs the deployment population:

• Report the demographic / site composition of the derivation (training) data. A subgroup that

is small or absent in development cannot be claimed to generalise to it, regardless of the pooled test metric; and including an attribute as a covariate is not the same as validating performance within that stratum.

• A deployment / "generalisable to [population]" claim for a subgroup that is unrepresented or

trivially small in the development data → MAJOR. Composition simply not reported → MINOR (it is a required disclosure under TRIPOD+AI).

EQ5 — Subgroup power / events-per-variable for the fairness comparison:

• A "no disparity" conclusion drawn from subgroups with few events is underpowered — the same

separation/instability problem the prediction-model EPV probe (CP3) raises. A fairness null needs the between-group gap CI to exclude a clinically relevant difference; a wide CI that still admits a meaningful gap is "we could not detect a disparity," not "the model is equitable."

• Subgroup comparisons with EPV in the low single digits, presented as evidence of fairness → MAJOR

(overclaim); the comparison should be descriptive-only. EPV / event counts per subgroup not reported → MINOR.

EQ6 — Equity-aware framing + guideline alignment; a limitation is not an endorsement:

• Scope the conclusion to the populations actually evaluated, and align reporting to TRIPOD+AI

(fairness / subgroup-performance and training-data composition items), DECIDE-AI, and CONSORT-AI (pre-specified subgroup analyses). A known fairness limitation (untested subgroup, unequal error rate, unrepresentative training data) must not be converted into a deployment endorsement or a recommendation-grade equity claim.

• Recommendation-grade equity language ("equitable", "fair across groups", "ready for diverse

populations") that the evidence does not support, or a conclusion that silently generalises past the evaluated subgroups → MAJOR. Cross-link ~/.claude/rules/scope-coherence-gate.md: an aggregate or single-population claim cannot stand on subgroup-silent data.

---

Output mapping. In peer-review, map each finding to a Major / Minor comment; EQ1 (aggregate-only performance behind a deployment claim), EQ2 (AUC-parity-only fairness claim), and EQ4 (deployment claim for an unrepresented subgroup) are design-level — surface them in the Confidential Comments to the Editor and place the strongest as the Major #1 candidate. EQ4 and an EQ5 underpowered-null overclaim are frequently unfixable in the current data (the missing subgroup or events cannot be added in revision) and govern the recommendation per Phase 2F. In self-review, a design-level EQ finding becomes a Fatal Anticipated Major Comment and a reporting-level one a Fixable Anticipated Minor Comment, tagged with the closest category letter. EQ co-applies with ai_overclaiming.md (a fairness gap is one route to an over-broad clinical claim) and reuses the EPV logic of clinical_prediction_model.md (CP3) at the subgroup level.

<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review. Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal recommendation. Each consuming skill maps findings to its own output:

• peer-review: Major / Minor comments + Confidential Comments to the Editor; a task- or

design-level flaw is placed as Major #1.

• self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.

Do NOT edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync. -->

Image-Synthesis / Cross-Modality Generation probes (IS1–IS4)

A 4-probe checklist for studies that synthesize one imaging modality from another (MRI→PET, MRI→CT, CT→MRI, non-contrast→contrast, low-dose→full-dose) and then claim the synthetic image carries clinically usable functional/molecular information. These probes complement (do not replace) the generic Phase 2 issue checklist. Their purpose is to keep three structurally distinct failure modes — which a single review tends to split or miss — under one reviewer's coverage, because each looks individually "addressable by reframing" while together they govern whether the central claim survives.

Trigger: the manuscript trains a generative model (GAN/PatchGAN, diffusion, U-Net/Swin-UNet, CycleGAN) to produce a target modality from a source modality and frames the output as providing metabolic / receptor / perfusion / metabolic-uptake information, or as a substitute for the real target modality when it is unavailable.

IS1 — Determinism / information ceiling (synthetic-vs-source value claim):

• The synthetic image is a deterministic function of the input source image(s). For the same reader on the same case, "source + synthetic" cannot carry patient-specific information beyond "source alone" — any reader AUC gain is most consistent with a representation/presentation/interpretability effect, not new diagnostic information.
• Does a direct source→label baseline (e.g., MRI-only model on the same split) exist to separate information from presentation? Absent → the incremental-value claim is unsupported as written.
• An interpretability/accessibility gain can still be clinically useful — recommend reframing the Abstract/Discussion to that, not to "added diagnostic value." MAJOR candidate when the Abstract/Conclusion reads the gain as new information.

IS2 — Target-derived preprocessing / label leakage (circularity):

• Was any mask, ROI, lesion segmentation, slice-selection rule, intensity normalization, or registration target derived from the target modality or the outcome label rather than from the input source alone?
• The decisive case: if a tumour/lesion mask (drawn on the target modality or from the diagnosis) guided slice selection or training, then the central claim — "functional information is inferred from source structure" — is circular and not supported by the design.
• Is the slice-selection / sampling strategy described at all? Silence here is itself a MAJOR candidate: the reviewer cannot exclude leakage, so the claim cannot be granted. Request the exact provenance of every preprocessing input and a leakage-free re-analysis.

IS3 — Lesion/target-level vs global validation:

• Quantitative agreement (SUVR/SUV correlation, Bland–Altman, PSNR/SSIM) reported on a global or whole-organ metric (e.g., whole-brain-to-reference-region SUVR) does not establish that the synthetic image reproduces the lesion-level quantity that actually drives grading/differentiation. Tumour uptake is heterogeneous; global agreement can be high while lesion uptake is wrong.
• Is target-level agreement reported (tumour ROI / target-to-background ratio)? Absent → the quantitative-fidelity claim must be tempered, or lesion-level agreement added. MAJOR candidate when the manuscript states lesion-level usability from a global metric.

IS4 — Mechanistic / proxy-signal plausibility:

• Name the physical quantity the source measures versus the target measures (e.g., MRI structure / blood–brain-barrier disruption vs PET tracer amino-acid transport & protein synthesis). When the manuscript labels the synthetic signal with the target's biology ("metabolic information"), is the source→target signal link validated, or merely assumed because the outputs look like the target?
• High image similarity is not evidence that an unmeasured biological signal was recovered. An unvalidated proxy claim is over-interpretation regardless of PSNR/SSIM. Watch for tracer-mechanism mislabeling carried through the whole manuscript (e.g., calling amino-acid-transport PET a "metabolism" readout).
• Absent validation of the structure→function link → temper the biological-information claim to image-resemblance. MAJOR candidate when the title/Abstract/Conclusion assert recovered functional/molecular information.

Output template (IS1 example):

"Because the synthetic PET is a deterministic function of the input MRI, for the same reader on the same case it cannot add patient-specific information beyond the MRI; the MRI-alone vs MRI+synthetic-PET gain is therefore more consistent with a representation/interpretability effect than with new diagnostic information, and no direct MRI→label baseline is provided to separate the two. I'd suggest reframing the Abstract and Discussion as an interpretability/accessibility gain rather than added diagnostic value, and (optionally) reporting a same-split MRI→label baseline to quantify how much of the gain is information versus presentation."

Output template (IS2 example):

"The slice-selection strategy is not described. If a tumour mask (drawn on the reference PET or from the diagnosis) guided which slices were preprocessed or trained on, then the central claim — that functional information is inferred from MRI structure — would be circular, and I could not exclude this from the manuscript as submitted. Please state the exact provenance of every preprocessing input (masks, ROIs, slice selection, normalization, registration target) and, if any was derived from the target modality or the label, provide a leakage-free re-analysis."

<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review. Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal recommendation. Each consuming skill maps findings to its own output:

• peer-review: Major / Minor comments + Confidential Comments to the Editor; a task- or

design-level flaw is placed as Major #1.

• self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.

Do NOT edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync. -->

Narrative / Review-Article probes (RV1–RV9)

A 9-probe checklist for a Review / narrative review / primer / state-of-the-art / educational review — i.e., a non-systematic synthesis rather than original research. Supporting appraisal material (the SANRA appraisal items, a consolidated evaluation checklist, and a candidate-additions catalog for AI/LLM-in-radiology reviews) is maintained separately by the peer-review skill and is not required to apply RV1–RV9 below.

The original-research probes (the generic Phase 2 issue checklist, and the SR-MA / Survival / Radiomics probes) do not transfer to review articles. The key inversion: for original research, reviewers are discouraged from scope-expanding requests, but for narrative reviews, identifying thematic gaps and proportionately suggesting missing content is an expected part of the reviewer's role — error-spotting alone is necessary but not sufficient. Keep SANRA in its lane: it is a 6-item critical appraisal tool, not a reporting guideline, so do not over-enforce it (only RV3 is SANRA-aligned, and as a suggestion; do not demand PRISMA — narrative ≠ systematic).

RV1 — Novelty & value-add (editorial value-add axis): Against ≥2–3 recent reviews/primers on the same topic, does the manuscript state explicitly what it adds? For saturated topics, if the authors do not position their contribution against the current review literature, the incremental value is hard to judge — MAJOR candidate. Judge contribution magnitude only; scope-fit is the editor's call.

RV2 — Scope & aims clarity (SANRA items 1–2): Is the topic's importance established, and are the review's aims and scope boundaries (what is included/excluded) explicit?

RV3 — Evidence-gathering transparency (SANRA item 3, suggestion-level): Even a narrative review benefits from one paragraph on how the literature was identified (databases, time window, selection logic). This is not a reject criterion — phrase it as a SANRA-aligned transparency suggestion. Do not require PRISMA.

RV4 — Technical & medical accuracy (reviewer niche strength): Engineering correctness (autoregressive decoding, RAG, RLHF, instruction tuning, hallucination mechanisms, evaluation/mitigation methods) and medical correctness (radiology claims, clinical examples, anatomy/imaging detail). Itemize errors with location. This axis is where a domain-literate reviewer adds unique value.

Model-class conflation sub-probe (LLM/VLM-in-radiology reviews): check whether the manuscript treats text-only LLMs, multimodal/vision-language models (VLMs), conventional narrow CAD, and image-reconstruction artifacts as one phenomenon with one risk profile. They are distinct: a text-only LLM doing report structuring/summarization/grammar correction (low-risk language support) carries very different failure modes from a VLM asked to interpret images (high-risk image interpretation). When a "primer on LLMs" silently spans these classes, flag it — and note that the most actionable radiology-specific contribution is usually a task-risk stratification (which tasks are acceptable, which need safeguards, which to avoid) rather than a generic "LLMs hallucinate" statement.

Verify-your-own-criticism gate: before raising a technical inaccuracy or a citation–claim mismatch as a major finding, cross-check the assertion against a current authoritative source (the full cited paper, CrossRef, arXiv). Fast-moving fields make critiques go stale: a method dismissed as "not applicable" may have been adapted, and a "preprint" may since have been peer-reviewed. If unverified, downgrade to a hedged "Please verify…"; if confirmed, state it firmly. This applies with extra force to claims about what a cited reference argues (a review about hallucination must not itself mis-attribute a source).

RV5 — Taxonomy / synthesis coherence: Is the manuscript's classification mutually exclusive and collectively exhaustive, and does it map to established taxonomies (intrinsic vs extrinsic; faithfulness vs factuality; published hallucination surveys)? Ad-hoc categories should be reconciled with an established taxonomy. Is the synthesis integrative rather than a list?

Source/cause vs masking/amplifying-factor sub-probe: when the manuscript presents a list of "sources" or "causes" of the error, check that each item is genuinely generative of the error rather than a factor that lets it go undetected or amplifies its impact. A recurring miscategorization: black-box opacity and automation bias are framed as sources of hallucination, but they do not generate hallucinations — opacity hides them and automation bias amplifies their downstream effect. Mis-labeling a masking/amplifying factor as a source is a sharper conceptual defect than "scattered/ad-hoc taxonomy" (the framework's own pillars are wrong, not just disorganized) — raise it explicitly rather than folding it into a generic synthesis comment.

RV6 — Balance, currency, citation accuracy (SANRA items 4–5): Is conflicting evidence handled fairly (no cherry-picking)? Are citations current and primary-source-weighted? Spot-check citation accuracy (author/year/claim match) — for a review about hallucination, citation errors are thematically critical. Single-anchor overload: when a load-bearing clinical claim rests on essentially one study (n=1, often abstract-only, paywalled, or unreplicated), check the Abstract↔body register for that anchor — a MAJOR red flag is the Abstract calling it "landmark/definitive" while the body concedes the evidence base "is thin." Soften the anchor to "an early signal" and align both claim sites; if a verifiable independent second study exists, add it rather than leaning the whole stake on one source.

RV7 — Load-bearing figures/tables (editorial value-add axis; SANRA item 6 secondary): Are there standardized comparison tables, a landscape figure, or a concrete clinical worked example? Assess whether figures/tables carry synthesis weight or are decorative — strong radiology-AI reviews tend to use standardized comparison matrices and a worked example.

RV8 — Constructive gap-filling & additions (the expected-role probe): Identify missing topics/frameworks/key references and propose them as "consider adding X because it directly supports Y" — never "must cite." Tier candidates by publication status:

• Peer-reviewed guidelines: TRIPOD-LLM, MI-CLAIM-GEN, and STARD-AI (all Nature Medicine), and the CLAIM 2024 update (Radiology: AI)
• Preprint (label as such): any not-yet-peer-reviewed arXiv/medRxiv item — name it as a preprint and do not place it at the same level as peer-reviewed guidelines. Verify status before citing, since preprints are frequently published later (a checklist first posted to arXiv may since have appeared in a journal)
• Concepts/tools: RAG specifics (retrieval failure vs fabrication), uncertainty/confidence calibration, radiology-specific evaluation (RadGraph, CheXbert/CheXpert-F1, ReXTrust), regulatory context (FDA 510(k)/CE, RADAR)

Keep additions proportionate (≈ ≤1 new reference per page, each motivated; no wholesale rewrite). Suggesting missing literature/topics is expected; demanding new studies is not.

Self-citation architecture (intellectual-COI, narrative form): Check whether the manuscript's weakest / most-deferred axes coincide with the authors' own forthcoming or companion work — a structural signal that the review may be agenda-setting for the authors' pipeline. This is legitimate for an invited review, but it must be made transparent: require a body-level motivation/COI line, and ensure every load-bearing axis carries at least one independent (other-group) source so the argument does not collapse to "trust us / see our upcoming paper." Companion-paper citations must stay strictly non-load-bearing ("not relied upon here") and survive into the revision.

RV9 — Bibliometric circularity of a curated base (narrative-review FATAL pattern): A non-systematic review that asserts a field-level / bibliometric property — "the field has invested heavily in X but neglected Y," a density/asymmetry/maturity gradient across topics — is making a measured claim from an unmeasured base. Because the reference set was curated, not searched, the asymmetry is a property of the authors' selection, not of the field; a hostile reviewer manufactures the opposite thesis by re-curating. This is a Fatal candidate when the gradient is the manuscript's central contribution. Two acceptable resolutions (a strategy fork, not a wording tweak):

• Down-scope (narrative/invited route): restate the claim as "within the literature surveyed here" and delete every field-level phrasing at every claim site — Abstract, body, each figure/table caption, and Conclusion. The grep discipline matters: a single residual "the field has…" sentence re-arms the rebuttal, so the down-scope must leave a field-level residue of zero.
• Measure (scoping route): add a documented search string + time window + per-axis counts so the asymmetry becomes a reproducible, reviewer-survivable measurement (this converts the piece toward a scoping review and is journal-agnostic).

Also separate "volume of methods papers" from "clinical evidence": an axis can be engineering-dense yet clinically empty, and calling such an axis "mature/well-invested" while the same section concedes it measures the wrong thing is an internal contradiction. Reframing the gradient as engineering-density vs clinical/reader-validation is usually the stronger, defensible thesis.

Output template (RV1 example):

"The topic of LLM hallucinations is now addressed by several recent reviews, so it would strengthen the manuscript to state explicitly what this primer adds beyond them — for example, a radiology-specific failure taxonomy, a worked clinical example, or an actionable verification workflow that existing general-purpose reviews do not provide. As written, the Introduction does not position the contribution against the current review literature, which makes the incremental value difficult to judge."

Output template (RV8 example):

"The mitigation section would benefit from engaging with emerging reporting standards for generative models, as these directly support the manuscript's call for controlled deployment. Consider adding a brief discussion of TRIPOD-LLM and MI-CLAIM-GEN (both peer-reviewed reporting guidelines for LLM/generative studies), and clarifying how retrieval-augmented generation shifts the dominant failure mode from fabrication toward retrieval error rather than eliminating hallucination, a distinction the current text conflates."

Output template (RV9 example):

"The central thesis — that the field has invested heavily in image generation while neglecting detection and education — is presented as a property of the field, but the evidence is a curated, non-systematic reference set, so the asymmetry could equally reflect the authors' selection. As written, an opposing reviewer could re-curate the citations and reach the reverse conclusion. We suggest either (a) restating the claim throughout as 'within the literature surveyed here' and removing the field-level phrasings in the Abstract, figure captions, and Conclusion, or (b) adding a brief documented search (sources, window, per-theme counts) so the asymmetry is reproducible. Relatedly, the 'mature' provenance axis is dense in engineering/IP methods but, by the manuscript's own admission, carries no clinical or reader validation; distinguishing 'volume of methods papers' from 'clinical evidence' would make the contrast sharper and harder to rebut."

This module gives review/narrative manuscripts a dedicated audit gate, on the principle that constructive gap-filling is an expected part of appraising a review article.

<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review. Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal recommendation. Each consuming skill maps findings to its own output:

• peer-review: Major / Minor comments + Confidential Comments to the Editor; a task- or

design-level flaw is placed as Major #1.

• self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.

Do NOT edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync. -->

Radiomics / Feature-Reproducibility probes (R1–R4)

A 4-probe checklist for radiomic feature reliability/reproducibility, acquisition–reconstruction parameter sweeps, and reliability/harmonization-based feature filtering claims. These probes complement (do not replace) the generic Phase 2 issue checklist. Their purpose is to keep design-level structural validity from being under-weighted: a review can correctly flag the reporting-layer issues (an over-claiming Abstract, a small external cohort) yet still miss whether the central contribution holds, which softens the assessment by one notch.

R1 — Design-grid circularity (in-domain "prediction" tautology):

• Is an outcome (e.g., feature reliability) predicted from the very grid parameters that were systematically/exhaustively varied to construct the dataset?
• If so, a high in-domain R² / accuracy is structurally guaranteed by the design ("predicting the construction recipe"), not a discovered relationship — do the predictors simply index the axes of the design grid?
• Does the manuscript frame in-domain performance as a finding/success and lead the Abstract/Key Points with it?
• If yes → do not endorse the in-domain success. Recommend reframing so the substantive finding is the cross-domain transportability (and, where present, its failure). MAJOR candidate.

R2 — Construct validity / proxy-target gap:

• The clinical rationale typically assumes that features which are reliable/stable/robust in the phantom are also better predictors of a biological/clinical target. A feature can be perfectly stable and biologically uninformative — this link is not logically guaranteed.
• Is any post-filter performance gain shown to be signal recovery, rather than a by-product of removing a degraded/misaligned baseline feature space?
• Does the manuscript acknowledge and test the orthogonality of the proxy (reliability) and the target (outcome)? Absent → MAJOR candidate.

R3 — Transportability framing vs reporting issue:

• When cross-phantom / cross-scanner / cross-center failure (negative R² on the target domain, low Jaccard overlap of selected features, calibration slope < 1) is the substantive result, is it nonetheless framed as a generalization success in the Abstract/Key Points/Conclusion?
• Does the Results text state explicitly that a negative R² on the target domain means the model performs worse than predicting the mean (i.e., the mapping does not transport), rather than reading it as a weak continuous performance metric?
• Calibration link: if in-domain "success" is partly a design artifact and the cross-domain result is a failure, reframing the Abstract will not rescue the central contribution. This is a design-level finding, not a reporting fix — keep its severity at the design level and do not soften it to a reporting issue.

R4 — Multiplicity (model × threshold / model × cohort grid):

• Are multiple classifiers × multiple reliability thresholds (or cohorts) compared with one-sided tests, with a few reaching p < 0.05?
• Is multiple-testing correction applied, and is the expected number of false positives by chance named explicitly (e.g., "5 models × 3 thresholds = 15 tests, ≈1 expected false positive")? Do not defer this to a generic "statistical review needed."
• For a small external cohort (n ≤ ~30), do bootstrap ΔAUC intervals cross zero? If so, restrict any headline-gain claim accordingly (e.g., to a single classifier family in a small cohort).

Output template (R1 example):

"Because the acquisition parameters were varied as a systematic factorial grid, a model that predicts feature reliability from those same parameters is largely recovering the grid by construction; the in-domain R² ≈ 1.0 therefore reflects design structure rather than a discovered relationship. I'd suggest reframing the Abstract and Key Points so the substantive finding is the cross-phantom/cross-scanner transportability (and its failure), and stating explicitly in the Results that a negative R² on the target domain means the model performs worse than predicting the mean — i.e., the reliability mapping does not transport."

Output template (R4 example):

"The reported gains come from a grid of [N models] × [M thresholds] one-sided comparisons; with [N×M] tests, roughly one positive is expected by chance alone, and the external cohort (n = [k]) yields bootstrap ΔAUC intervals that cross zero for several thresholds. I'd suggest reporting a multiplicity-adjusted analysis (or stating the expected false-positive count), restricting the headline claim to the classifier family that survives, and marking the ΔAUC intervals that cross zero in the figure."

Exemplar — AI overclaiming relative to the evidence

Finding class: the conclusion's reach (generalizable / outperforms / can replace) exceeds what a single-center, single-reader, or internally-validated result supports. Typical severity: design-/framing-level → Major #1 when the headline claim depends on it; Minor when it is only a stray adjective in the Discussion.

What the reviewer noticed

The Abstract and Conclusion state the model "generalizes across institutions" and "outperforms radiologists," but the external test set is one site (Methods, "External validation"), the reader comparison used two readers on a different task tempo than the model (Table 3), and the confidence intervals for model vs reader AUC overlap (Figure 2).

Weak phrasing (avoid)

The authors overclaim. Saying the model generalizes and beats radiologists is not

justified and should be removed.

(Verdict without an anchor, no path forward, gatekeeper tone.)

Strong phrasing (model this)

The Conclusion states the model "generalizes across institutions," but external

validation appears limited to a single site (Methods, External validation). I'd

suggest softening this to the evidence — e.g., "validated at one external site" — and

framing multi-institution generalizability as a stated limitation and next step.

>

Relatedly, the "outperforms radiologists" claim rests on a comparison whose 95% CIs for

model and reader AUC overlap (Figure 2), and the reader task differs from the model's in

[tempo/inputs] (Table 3). It would strengthen the paper to (a) report the difference in

AUC with its CI and a test of that difference rather than two separate AUCs, and (b)

state explicitly which clinical task the comparison establishes. If the difference is

not statistically supported, I'd recommend reframing from "outperforms" to

"comparable to," which is still a meaningful and more defensible result.

Why this is Major #1 here

The over-reach is in the Abstract and Conclusion and is the paper's headline, so a reader takes away a claim the data do not support. Anchoring it to the single-site external set and the overlapping CIs makes the fix concrete and keeps the contribution intact.

Related checks

Signature check "Overclaiming vs evidence level"; self-review category D (endpoint↔conclusion scope); check_scope_coherence.py for surrogate→care-directive and cross-sectional→prognostic variants.

Exemplar — calibration not reported for a clinical prediction model

Finding class: the model is presented for clinical decision-making, but only discrimination (AUC/c-statistic) is reported — no calibration, and often no decision-curve or clinical-utility analysis. Typical severity: Major when the paper proposes clinical use (a probability that drives a decision must be calibrated); Minor when the model is framed as a research prototype only.

What the reviewer noticed

Results report AUC with CIs for every model (Table 2, Figure 1), and the Discussion proposes using predicted probabilities to triage patients, but there is no calibration plot or metric (calibration slope/intercept, ECE) and no decision-curve analysis.

Weak phrasing (avoid)

AUC is not enough; the authors need calibration.

(True but terse; no reason, no anchor, no path.)

Strong phrasing (model this)

The discrimination results are clearly presented (Table 2, Figure 1). Because the

Discussion proposes using the predicted probabilities to triage patients, it would help

to add an assessment of calibration — how close predicted probabilities are to observed

frequencies. AUC can be high while probabilities are systematically too confident, which

would mislead a probability-based threshold. A calibration plot with slope and intercept

(and ideally on the external set) would let readers judge whether the proposed thresholds

are safe. A decision-curve analysis would further show the net benefit across plausible

threshold probabilities relative to treat-all / treat-none. If recalibration was applied,

stating the method would also be useful.

Severity calibration

If the paper's contribution is a deployable triage tool, calibration is Major — the clinical claim rests on the probabilities being trustworthy. If the model is explicitly a methods demonstration with no decision claim, a calibration metric is a reasonable Minor addition.

Related checks

Signature check "Calibration (AUC alone insufficient)"; TRIPOD+AI / CLAIM calibration items via /check-reporting; self-review category C (calibration [CRITICAL]).