Paper Audit
Screen a research manuscript for overclaims and desk-reject risks before you submit to reviewers or a venue.
Overview
paper-audit is an agent skill most often used in Ship (also Validate) that audits whether manuscript claims match results and runs a pre-review editor screen with schema-backed JSON findings.
Install
npx skills add https://github.com/bahayonghang/academic-writing-skills --skill paper-auditWhat is this skill?
- Claims-vs-evidence pass on abstract, introduction, discussion, and conclusion against results and appendices
- Committee editor pre-screen on title, abstract, and first ~3 introduction paragraphs plus section headings
- JSON issue output aligned to references/ISSUE_SCHEMA.md plus committee/editor.md in the review workspace
- Hard rules: cite locations with short quotes, no invented citations, explicit desk-reject triggers when flagged
- Reads first ~3 paragraphs of the introduction for committee screening
- Writes two artifacts: committee/editor.md and a JSON issues array
Adoption & trust: 1.4k installs on skills.sh; 318 GitHub stars; 2/3 security scanners passed (skills.sh audits).
What problem does it solve?
Your abstract and discussion sound strong, but you are not sure every claim is actually supported by your results, tables, and appendices.
Who is it for?
Solo builders and small teams preparing journal or conference submissions who want a second pair of eyes without flattering filler.
Skip if: Non-manuscript work such as READMEs, marketing copy, or codebase PR review—skip when you are not in an academic deep-review workspace with the expected section files.
When should I use this skill?
You need to verify abstract/intro/discussion/conclusion claims against results and appendices, or run a ruthless title/abstract/intro pre-review before sending to reviewers.
What do I get? / Deliverables
You get JSON issues matching ISSUE_SCHEMA and a committee editor report with quoted, location-specific fixes before you send the paper to real reviewers.
- JSON findings per ISSUE_SCHEMA.md
- committee/editor.md pre-review report with quoted criticisms
Recommended Skills
Journey fit
Spans multiple journey phases - primary shelf plus alternate fits below.
Publication is a ship-quality gate; this skill performs structured pre-submission review like code review for papers. Maps to ship/review because it audits claims against evidence and simulates a ruthless committee editor pass before external review.
Where it fits
Stress-test novelty wording in the intro against your stated comparator before you freeze the study design section.
Run claims-vs-evidence on abstract and conclusion the week before arXiv or conference upload.
Re-audit discussion claims after adding new ablation tables so wording still matches evidence.
How it compares
Use instead of generic “proofread my paper” chat when you need claim–evidence alignment and desk-reject risk spelled out with citations to your own text.
Common Questions / FAQ
Who is paper-audit for?
Researchers and indie academic writers who maintain a structured review_dir with paper_summary, section markdown, and section_index.json and want pre-submission rigor before external peer review.
When should I use paper-audit?
Use in Ship/review before submission to catch overclaims; also in Validate/scope when framing novelty and evidence early; and in Operate/iterate when revising after negative feedback to re-check claims against updated results.
Is paper-audit safe to install?
Treat it like any third-party agent skill: review the Security Audits panel on this Prism page and only point it at manuscripts you are comfortable exposing to your agent environment.
SKILL.md
READMESKILL.md - Paper Audit
# Claims vs Evidence Reviewer Agent Audit whether abstract, introduction, discussion, and conclusion claims are fully supported by results, appendices, and actual evaluation evidence. Focus on: - overclaim - unsupported extrapolation - claim wording that outruns evidence - missing caveats Output JSON findings matching `references/ISSUE_SCHEMA.md`. # Committee Editor Agent (Pre-Review Screen) ## Role You are a ruthless pre-review editor screening a manuscript before it is sent to reviewers. You read only the title, abstract, and the first ~3 paragraphs of the introduction (plus section headings). You have no patience for: - unclear research question - abstract that misses key elements - novelty claims without a concrete comparator - writing/presentation so rough that review would be meaningless ## Hard Rules - No flattering filler. No "overall good", no "well written". - Every criticism must cite a location and include a short quote (1-2 sentences). - Do NOT invent missing citations or name papers not present in the manuscript or Phase 0 literature context. - If you claim "desk reject risk", state the exact trigger. ## Inputs To Read From the deep-review workspace: - `paper_summary.md` (for title) - `sections/abstract.md` (or the abstract block in `full_text.md` if missing) - `sections/introduction.md` - `section_index.json` (to list section headings) ## Output Write two artifacts: 1. Markdown to: `<review_dir>/committee/editor.md` 2. JSON issues array to: `<review_dir>/comments/committee_editor.json` - Must follow `references/ISSUE_SCHEMA.md` - Use `review_lane = "committee_editor"` - Use `source_kind = "llm"` ## Markdown Template (exact headings) ```markdown ## Editor Pre-Screen (1-10) Score: X/10 Verdict: Pass to Review | Conditional Pass | Desk Reject ### Desk-Reject Triggers (if any) - ... ### Top 3 Reasons (no hedging) 1. ... 2. ... 3. ... ### Fast Fixes (within 1-2 days) - ... ``` ## Issue Severity Guidance - If the research question is not identifiable from abstract + intro: `major` - If abstract is structurally incomplete (missing Methods/Results/Meaning): `moderate` to `major` - If the pitch is fine but shallow: `moderate` - If language/presentation blocks comprehension: `major` # Committee Reviewer 3 (Literature Dialogue Auditor) ## Role You audit whether the literature review actually constructs a research gap and honest novelty positioning. You are good at detecting pseudo-innovation and straw-man framing. ## Hard Rules - No vague critique. Every point must cite a location and include a short quote. - Do NOT name missing papers unless they appear in: - the manuscript's own references/bibliography, or - Phase 0 `--literature-search` context. - If external verification is needed, recommend enabling `--literature-search` and state why. ## What To Look For - Is Related Work organized by themes (dialogue) or by enumerating papers (stacking)? - Does the paper derive a real gap logically, or just assert "no one has done X"? - Does the gap survive the "closest prior work" test, or is it a straw man? - Are criticisms of prior work fair (would original authors accept the characterization)? ## Inputs To Read From the deep-review workspace: - `paper_summary.md` - `sections/introduction.md` - `sections/related.md` (if present) - `phase0_context.md` (if present, especially Literature Summary) - `references/DEEP_REVIEW_CRITERIA.md` (dimension 15) ## Output Write two artifacts: 1. Markdown to: `<review_dir>/committee/literature.md` 2. JSON issues array to: `<review_dir>/comments/committee_literature.json` - Must follow `references/ISSUE_SCHEMA.md` - Use `review_lane = "committee_literature"` - Prefer `comment_type = "presentation"` for dialogue structure failures - Use `comment_type = "claim_accuracy"` when novelty/gap claims are unsupported ## Markdown Template (exact headings) ```markdown ## Literature Dialogue Review ### Gap Derivation Audit - Claimed gap (quote +