
Outline Agent
- 44 installs
- 628 repo stars
- Updated July 9, 2026
- ar9av/paperorchestra
outline-agent is a Claude skill that turns raw research inputs into a validated JSON outline driving the rest of the PaperOrchestra pipeline.
About
Step 1 of the PaperOrchestra pipeline that converts raw materials (idea, experimental log, LaTeX template, conference guidelines) into a strict JSON outline. The outline contains a plotting plan, a literature-search plan for the Introduction and Related Work, and a section-level writing plan with citation hints. A developer runs it first because every downstream agent parses this JSON.
- Step 1 of PaperOrchestra: one LLM call to build the outline
- Emits strict JSON: plotting plan, lit-search plan, section plan
- Validated against a machine-readable JSON schema
Outline Agent by the numbers
- 44 all-time installs (skills.sh)
- Ranked #7,851 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
outline-agent capabilities & compatibility
- Capabilities
- research · planning · documentation
- Use cases
- research · planning · documentation
What outline-agent says it does
Convert (idea.md, experimental_log.md, template.tex, conference_guidelines.md) into a strict JSON outline containing a plotting plan, literature search plan (Intro + Related Work), and section-level w
**Cost: 1 LLM call.**
npx skills add https://github.com/ar9av/paperorchestra --skill outline-agentAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 44 |
|---|---|
| repo stars | ★ 628 |
| Last updated | July 9, 2026 |
| Repository | ar9av/paperorchestra ↗ |
What it does
Convert raw research materials into a strict JSON outline with plotting, literature, and section plans.
Who is it for?
Producing the structured plan that seeds a multi-agent paper-writing pipeline
Skip if: Drafting prose, generating figures, or verifying citations
When should I use this skill?
The orchestrator delegates Step 1, or the user asks to outline a paper from raw materials or generate the paper structure
What you get
Produces a validated workspace/outline.json with plotting, literature, and section plans
- workspace/outline.json
By the numbers
- 1 LLM call per run
- 3 top-level JSON keys (plotting_plan, intro_related_work_plan, section_plan)
- Related Work divided into 2-4 methodology clusters
Files
Outline Agent (Step 1)
Faithful implementation of the Outline Agent from PaperOrchestra (Song et al., 2026, arXiv:2604.05018, App. F.1, pp. 40–44).
Cost: 1 LLM call.
Your task
Read four input files from the workspace and produce a single JSON object at workspace/outline.json with three top-level keys:
plotting_plan— array of figure objectsintro_related_work_plan— object withintroduction_strategyandrelated_work_strategysection_plan— array of section objects, each withsection_titleandsubsections[]
How to do it
1. Read the verbatim prompt at `references/prompt.md`. This is the exact Outline Agent system prompt from the paper. Use it as your system message. 2. Prepend the Anti-Leakage Prompt from ../paper-orchestra/references/anti-leakage-prompt.md. 3. Read the four input files:
workspace/inputs/idea.mdworkspace/inputs/experimental_log.mdworkspace/inputs/template.texworkspace/inputs/conference_guidelines.md
4. Synthesize across all four — the global instruction in the prompt is "Do not analyze inputs in isolation. You must synthesize information across all provided documents for every step." 5. Emit a single JSON object following the schema in references/outline-schema.md. Cross-check against references/outline_schema.json (machine-readable). 6. Save to workspace/outline.json. 7. Validate:
python skills/outline-agent/scripts/validate_outline.py workspace/outline.jsonIf validation fails, fix the JSON and re-validate. Do not proceed to Step 2 or Step 3 with an invalid outline — every downstream agent depends on this schema.
8. Append §1 to research_brief.md (see skills/shared/research_brief_template.md):
After outline.json passes validation, append the §1 section to workspace/research_brief.md (create the file if absent). Template:
## §1 · Core Claim and Narrative
_Written by: outline-agent, Step 1_
**Core claim:** <one-sentence contribution>
**Narrative tension:** <gap this paper resolves>
**Key novelty framing:** <how the contribution is framed relative to prior work>
**Outline decisions:**
- Plotting plan: <N> figures
- Related Work clusters: <names>
- Section structure: <section titles>
**Potential weaknesses flagged at outline stage:**
- <any claim in idea.md that may be hard to support>This is a free-form prose append; no machine-readable schema required.
Hard rules from the prompt (do not violate)
These are excerpted from references/prompt.md. The validator enforces them.
Plotting plan (Directive 1)
plot_typeMUST be exactly one of"plot"or"diagram".data_sourceMUST be exactly one of"idea.md","experimental_log.md",
or "both".
aspect_ratioMUST be exactly one of:
"1:1", "1:4", "2:3", "3:2", "3:4", "4:1", "4:3", "4:5", "5:4", "9:16", "16:9", "21:9".
figure_idMUST be a semantically meaningful snake_case identifier
(e.g., fig_framework_overview, fig_ablation_study_parameter_sensitivity).
figure_idMUST NOT contain the word"Figure".
Intro / Related Work strategy (Directive 2)
- Strictly separate Introduction (macro-level context, 10-20 papers,
foundational + survey + impact) from Related Work (micro-level technical baselines, 30-50 papers, divided into 2-4 methodology clusters that directly compete with or precede the proposed approach).
- For each Related Work cluster: provide
methodology_cluster,
sota_investigation_mission, limitation_hypothesis, limitation_search_queries, bridge_to_our_method.
- CRITICAL TIMELINE RULE: Do not instruct searches for any papers
published after {cutoff_date}. Derive cutoff_date from conference_guidelines.md (e.g., "ICLR 2025 → cutoff October 2024", "CVPR 2025 → cutoff November 2024"). If unspecified, default to one month before today's date.
Section plan (Directive 3)
- Structural hierarchy: if Subsection X.1 is created, X.2 is mandatory.
No orphaned subsections. Omit subsections entirely if a section does not require division.
- Content specificity: each
content_bulletsentry must reference source
materials concretely. AVOID "Describe the model". REQUIRE "Formalize the Temporal-Aware Attention mechanism using Eq. 3 from idea.md."
- Mandatory citations: every dataset, optimizer, metric, and
foundational architecture/model mentioned in idea.md or experimental_log.md MUST have a citation hint, no matter how ubiquitous (e.g., AdamW, ResNet, ImageNet, CLIP, Transformer, LLaMA, GPT, LLaVA).
- Citation hint format:
- If you know the exact author and title:
"Author (Exact Paper Title)"
- Otherwise:
"research paper or technical report introducing '[Exact Model/Dataset/Metric Name]'" - Do NOT guess or hallucinate authors.
Output
Exactly one file: workspace/outline.json. No prose, no code blocks, no markdown. The Section Writing Agent and Literature Review Agent will parse this JSON directly.
See references/example-output.json for a complete worked example from the paper (App. F.1, pp. 43–44).
Resources
references/prompt.md— verbatim Outline Agent prompt from App. F.1references/outline-schema.md— prose explanation of the schemareferences/outline_schema.json— machine-readable JSON Schemareferences/example-output.json— example output from the paperreferences/allowed-values.md— enumerated allowed values for each enum fieldscripts/validate_outline.py— JSON Schema validatorskills/shared/research_brief_template.md— NEW §1 schema; append after outline.json passes validation
Allowed enumerated values
Source: arXiv:2604.05018, App. F.1 Outline Agent prompt.
These are the only values the validator will accept for each enum field. Anything else will fail validate_outline.py.
plot_type
| Value | Use for |
|---|---|
"plot" | Statistical/quantitative charts (line, bar, radar, scatter, box, histogram, heatmap, etc.). The objective field MUST also name the specific chart type. |
"diagram" | Conceptual / architectural diagrams (block diagram, flowchart, system overview). Rendered with graphviz or matplotlib patches. |
data_source
| Value | Meaning |
|---|---|
"idea.md" | The figure visualizes a concept or architecture from the idea (typically diagrams). |
"experimental_log.md" | The figure plots numeric values from the experimental log. |
"both" | The figure combines conceptual context with empirical data. |
aspect_ratio
Exactly one of these 12 strings:
1:1 1:4 2:3 3:2 3:4 4:1 4:3 4:5 5:4 9:16 16:9 21:9Common conventions:
| Use | Ratio |
|---|---|
| Square ablation grid | 1:1 |
| Single column figure (2-col layout) | 4:3 or 5:4 |
| Wide cross-column figure (2-col layout) | 16:9 or 21:9 |
| Vertical mobile / tall comparison | 9:16 or 2:3 |
| Teaser banner | 21:9 |
| Tall stacked subplots | 3:4 or 4:5 |
The Plotting Agent's render_matplotlib.py translates the ratio into pixel dimensions at 300 DPI.
figure_id rules
- Snake_case:
^[a-z0-9_]+$ - Convention: prefix with
fig_(e.g.,fig_main_results,
fig_ablation_temperature_sensitivity).
- MUST be semantically meaningful (no
fig_1,fig_a,chart). - MUST NOT contain the word
figureas a substring after thefig_prefix.
Citation hint format
Two acceptable forms (per the prompt):
| You know | Use |
|---|---|
| Exact author and title | "Author (Exact Paper Title)" — e.g., "Vaswani et al. (Attention Is All You Need)" |
| You don't know either | "research paper or technical report introducing '[Exact Model/Dataset/Metric Name]'" — e.g., "research paper or technical report introducing 'AdamW optimizer'" |
Never guess authors. The Literature Review Agent will use these hints as search queries — wrong authors lead to wrong citations.
{
"_comment": "Verbatim example from arXiv:2604.05018, App. F.1, pp. 43-44. Use as a reference for shape and style. Do not copy literally.",
"plotting_plan": [
{
"figure_id": "fig_teaser_fig_cross_modal_alignment_performance",
"title": "Teaser: Cross-Modal Alignment Performance",
"plot_type": "plot",
"data_source": "experimental_log.md",
"objective": "Visual summary (Radar Chart) demonstrating that our method achieves SOTA balance across 5 metrics.",
"aspect_ratio": "16:9"
}
],
"intro_related_work_plan": {
"introduction_strategy": {
"hook_hypothesis": "Video-LLMs are currently the dominant paradigm for short clips.",
"problem_gap_hypothesis": "Context window limits prevent scaling to >5s videos efficiently.",
"search_directions": [
"Find highly cited papers establishing the real-world impact of context limits in video generation",
"Search for published 'long-context video generation' surveys",
"Identify foundational papers establishing causal video generation"
]
},
"related_work_strategy": {
"overview": "Investigate three specific paradigms to build a graph proving the necessity of our Sliding-Window approach.",
"subsections": [
{
"subsection_title": "2.1 Autoregressive Video Generation",
"methodology_cluster": "Discrete Tokenization & Transformers",
"sota_investigation_mission": "Identify the current SOTA autoregressive models from 2024-2025. Determine their maximum stable generation length.",
"limitation_hypothesis": "These models suffer from 'drift' or 'error propagation' because they lack bidirectional context.",
"limitation_search_queries": [
"Autoregressive video generation error propagation metrics",
"Causal masking limitations in temporal video transformers"
],
"bridge_to_our_method": "Our method introduces bidirectional blocks to fix the hypothesized drift issue."
},
{
"subsection_title": "2.2 Diffusion-Based Editing Frameworks",
"methodology_cluster": "DDIM Inversion & Cross-Attention",
"sota_investigation_mission": "Find recent papers using DDIM inversion for editing. Identify the standard benchmarks they use.",
"limitation_hypothesis": "They fail at large structural changes because cross-attention maps are too rigid.",
"limitation_search_queries": [
"DDIM inversion failure cases large motion",
"Cross-attention control rigidity video editing"
],
"bridge_to_our_method": "Our Flow-Guided Attention allows for spatial deformation, addressing rigidity."
}
]
}
},
"section_plan": [
{
"section_title": "Abstract",
"subsections": [
{
"subsection_title": "Abstract Content",
"content_bullets": [
"Briefly state the problem of temporal inconsistency.",
"Introduce the proposed method.",
"Highlight key results."
],
"citation_hints": []
}
]
},
{
"section_title": "3. Methodology",
"subsections": [
{
"subsection_title": "3.1 Temporal-Aware Attention Mechanism",
"content_bullets": [
"Define the query-key matching logic",
"Explain the masking strategy"
],
"citation_hints": [
"Vaswani et al. (Attention Is All You Need)",
"research paper or technical report introducing 'FlashAttention-2'"
]
},
{
"subsection_title": "3.2 Optimization Objective",
"content_bullets": [
"Detail the loss function",
"Discuss regularization terms"
],
"citation_hints": []
}
]
},
{
"section_title": "4. Experiments",
"subsections": [
{
"subsection_title": "4.1 Experimental Setup",
"content_bullets": [
"Implementation details",
"Hyperparameters and datasets used"
],
"citation_hints": [
"research paper or technical report introducing 'WebVid-10M'",
"Paszke et al. (PyTorch: An Imperative Style, High-Performance Deep Learning Library)",
"research paper or technical report introducing 'AdamW optimizer'",
"research paper or technical report introducing 'Jaccard Index'"
]
},
{
"subsection_title": "4.2 Main Results",
"content_bullets": [
"Comparison with Baselines",
"Quantitative Analysis"
],
"citation_hints": [
"Ho et al. (Denoising Diffusion Probabilistic Models)",
"research paper or technical report introducing 'AVSegFormer baseline'"
]
}
]
}
]
}
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "PaperOrchestra Outline",
"description": "Schema for outline.json — output of the Outline Agent (Step 1 of arXiv:2604.05018, App. F.1).",
"type": "object",
"required": ["plotting_plan", "intro_related_work_plan", "section_plan"],
"additionalProperties": true,
"properties": {
"plotting_plan": {
"type": "array",
"minItems": 1,
"items": { "$ref": "#/$defs/FigureSpec" }
},
"intro_related_work_plan": {
"type": "object",
"required": ["introduction_strategy", "related_work_strategy"],
"properties": {
"introduction_strategy": {
"type": "object",
"required": ["hook_hypothesis", "problem_gap_hypothesis", "search_directions"],
"properties": {
"hook_hypothesis": { "type": "string", "minLength": 1 },
"problem_gap_hypothesis": { "type": "string", "minLength": 1 },
"search_directions": {
"type": "array",
"minItems": 3,
"maxItems": 8,
"items": { "type": "string", "minLength": 1 }
}
}
},
"related_work_strategy": {
"type": "object",
"required": ["overview", "subsections"],
"properties": {
"overview": { "type": "string", "minLength": 1 },
"subsections": {
"type": "array",
"minItems": 2,
"maxItems": 4,
"items": { "$ref": "#/$defs/RelWorkSubsection" }
}
}
}
}
},
"section_plan": {
"type": "array",
"minItems": 1,
"items": { "$ref": "#/$defs/SectionSpec" }
}
},
"$defs": {
"FigureSpec": {
"type": "object",
"required": ["figure_id", "title", "plot_type", "data_source", "objective", "aspect_ratio"],
"properties": {
"figure_id": {
"type": "string",
"pattern": "^[a-z0-9_]+$",
"not": { "pattern": "figure(?!_)" }
},
"title": { "type": "string", "minLength": 1 },
"plot_type": { "enum": ["plot", "diagram"] },
"data_source": { "enum": ["idea.md", "experimental_log.md", "both"] },
"objective": { "type": "string", "minLength": 1 },
"aspect_ratio": {
"enum": ["1:1", "1:4", "2:3", "3:2", "3:4", "4:1", "4:3", "4:5", "5:4", "9:16", "16:9", "21:9"]
}
}
},
"RelWorkSubsection": {
"type": "object",
"required": [
"subsection_title", "methodology_cluster", "sota_investigation_mission",
"limitation_hypothesis", "limitation_search_queries", "bridge_to_our_method"
],
"properties": {
"subsection_title": { "type": "string", "minLength": 1 },
"methodology_cluster": { "type": "string", "minLength": 1 },
"sota_investigation_mission": { "type": "string", "minLength": 1 },
"limitation_hypothesis": { "type": "string", "minLength": 1 },
"limitation_search_queries": {
"type": "array",
"minItems": 1,
"items": { "type": "string", "minLength": 1 }
},
"bridge_to_our_method": { "type": "string", "minLength": 1 }
}
},
"SectionSpec": {
"type": "object",
"required": ["section_title", "subsections"],
"properties": {
"section_title": { "type": "string", "minLength": 1 },
"subsections": {
"type": "array",
"items": {
"type": "object",
"required": ["subsection_title", "content_bullets", "citation_hints"],
"properties": {
"subsection_title": { "type": "string", "minLength": 1 },
"content_bullets": {
"type": "array",
"minItems": 1,
"items": { "type": "string", "minLength": 1 }
},
"citation_hints": {
"type": "array",
"items": { "type": "string", "minLength": 1 }
}
}
}
}
}
}
}
}
Outline JSON Schema (prose)
The Outline Agent emits a single JSON object with three top-level keys: plotting_plan, intro_related_work_plan, section_plan. Source: arXiv:2604.05018, App. F.1, pp. 43–44.
The machine-readable JSON Schema is in outline_schema.json and is enforced by scripts/validate_outline.py. This document is the human-readable view.
Top-level structure
{
"plotting_plan": [ <FigureSpec>, ... ],
"intro_related_work_plan": <IntroRelWorkPlan>,
"section_plan": [ <SectionSpec>, ... ]
}FigureSpec
{
"figure_id": "fig_framework_overview", // snake_case, no "Figure"
"title": "Framework Overview",
"plot_type": "diagram", // "plot" or "diagram"
"data_source": "idea.md", // "idea.md" | "experimental_log.md" | "both"
"objective": "Visual summary (Block Diagram) of the proposed pipeline.",
"aspect_ratio": "16:9" // see allowed-values.md
}If plot_type is "plot", the objective MUST name the specific chart type (e.g., "Visual summary (Radar Chart) demonstrating that our method achieves SOTA balance across 5 metrics.").
IntroRelWorkPlan
{
"introduction_strategy": {
"hook_hypothesis": "Video-LLMs are currently the dominant paradigm for short clips.",
"problem_gap_hypothesis": "Context window limits prevent scaling to >5s videos efficiently.",
"search_directions": [
"Find highly cited papers establishing the real-world impact of context limits in video generation",
"Search for published 'long-context video generation' surveys",
"Identify foundational papers establishing causal video generation"
]
},
"related_work_strategy": {
"overview": "Investigate three specific paradigms to build a graph proving the necessity of our Sliding-Window approach.",
"subsections": [
{
"subsection_title": "2.1 Autoregressive Video Generation",
"methodology_cluster": "Discrete Tokenization & Transformers",
"sota_investigation_mission": "Identify the current SOTA autoregressive models from 2024-2025. Determine their maximum stable generation length.",
"limitation_hypothesis": "These models suffer from 'drift' or 'error propagation' because they lack bidirectional context.",
"limitation_search_queries": [
"Autoregressive video generation error propagation metrics",
"Causal masking limitations in temporal video transformers"
],
"bridge_to_our_method": "Our method introduces bidirectional blocks to fix the hypothesized drift issue."
}
]
}
}SectionSpec
{
"section_title": "3. Methodology",
"subsections": [
{
"subsection_title": "3.1 Temporal-Aware Attention Mechanism",
"content_bullets": [
"Define the query-key matching logic",
"Explain the masking strategy"
],
"citation_hints": [
"Vaswani et al. (Attention Is All You Need)",
"research paper or technical report introducing 'FlashAttention-2'"
]
},
{
"subsection_title": "3.2 Optimization Objective",
"content_bullets": [
"Detail the loss function",
"Discuss regularization terms"
],
"citation_hints": []
}
]
}Validation rules (enforced by validate_outline.py)
1. The top-level object MUST contain exactly the keys plotting_plan, intro_related_work_plan, section_plan (extras allowed but reported). 2. plotting_plan is a non-empty array. 3. Each FigureSpec has all six fields with correct enum values. 4. figure_id matches ^[a-z0-9_]+$ and does not contain figure as a substring after fig_. 5. intro_related_work_plan.introduction_strategy and .related_work_strategy both present. 6. related_work_strategy.subsections has 2–4 entries (the paper's "2-4 distinct methodology clusters" rule). 7. section_plan is a non-empty array. Each SectionSpec has section_title and subsections. 8. Hierarchy rule: if a section has any subsection, it must have ≥2 (no orphans). The validator emits a WARNING for sections with exactly 1 subsection. 9. citation_hints is always an array (may be empty).
Outline Agent — verbatim prompt
Source: arXiv:2604.05018, Appendix F.1, pages 40–44 (verbatim).
This is the exact system prompt used by the Outline Agent in the paper. Reproduce it as your system message. Substitute {cutoff_date} with the research cutoff derived from conference_guidelines.md.
---
You are a senior AI researcher drafting a paper for a top-tier conference
(e.g., NeurIPS, ICML, CVPR, ICLR). Your task is to convert the provided
methodology and experimental logs into a detailed, venue-compliant paper
outline. You must output a single JSON object.
Your inputs are:
1. idea.md: A detailed summary of the methodology, core contributions, and
theoretical framework.
2. experimental_log.md: A summary of experimental results, including raw
data points, ablation studies, and performance metrics.
3. template.tex: The template structure. You must use the section commands
(e.g., \section{...}) found here as your primary skeleton.
4. conference_guidelines.md: Formatting rules, specific page limits (for
word count calculation), and mandatory sections.
Processing Directives
Global Instruction: Do not analyze inputs in isolation. You must synthesize
information across all provided documents for every step.
Directive 1: Plotting & Visualization Plan
Synthesize experimental_log.md and idea.md to identify the most compelling
evidence.
- Determine which figures are essential to visually prove the hypothesis
(e.g., convergence rates, qualitative visual comparisons).
- The plot_type MUST be exactly "plot" or "diagram". If it is a plot,
specify the specific chart type (e.g., Radar Chart) inside the objective.
- The data_source MUST be exactly "idea.md", "experimental_log.md", or
"both".
- Determine the ideal aspect_ratio for each figure. The aspect_ratio MUST
be exactly one of: "1:1", "1:4", "2:3", "3:2", "3:4", "4:1", "4:3",
"4:5", "5:4", "9:16", "16:9", "21:9".
- The figure_id MUST be a semantically meaningful string identifier
summarizing the plot contents, like "fig_framework_overview" or
"fig_ablation_study_parameter_sensitivity". It MUST NOT contain the word
"Figure".
- Output Focus: Create an array of objects for the plotting_plan key.
Directive 2: Research Graph & Investigation Strategy (Intro & Related Work)
Provide search instructions for a downstream literature review agent to build
a Research Graph. Do not write the actual paper content.
Prevent Citation Overlap: Strictly separate the scope of the Introduction
from Related Work to ensure the agent searches for different tiers of
literature.
- Introduction: Focuses on macro-level context (foundational papers,
surveys).
- Related Work: Focuses on micro-level technical comparisons (recent SOTA
baselines, benchmarks).
Introduction Strategy (Macro-Level Context, 10-20 papers):
- Hypotheses: Define the "Hook" (broad context) and "Problem Gap" to be
verified. CRITICAL: Strictly scope the problem gap and claims to match
the specific datasets and evaluations present in experimental_log.md.
Do not over-claim generalization.
- Search Directions: Provide 3-5 specific queries to find:
1. Papers establishing the real-world impact or urgency of the problem
gap.
2. Good survey or review papers on the topic.
3. 3-5 Foundational papers that established the sub-field.
Related Work Strategy (Micro-Level Technical Baselines, 30-50 papers):
- Divide the field into 2-4 distinct methodology clusters that directly
compete with or precede our approach.
- For each cluster, define:
1. Methodology Cluster Name: The technical category.
2. SOTA Investigation: Instructions to find recent papers for conceptual
context. CRITICAL TIMELINE RULE: Do not instruct searches for any
papers published after {cutoff_date}. Furthermore, do NOT instruct
the search for new "competitors" to beat if they are not exclusively
in experimental_log.md.
3. Limitation Hypothesis: The suspected failure point of these
competing methods, based on idea.md.
4. Limitation Search Queries: Highly specific, narrow queries to find
papers documenting these exact limitations.
5. The Bridge: How our proposed method resolves this specific limitation.
Output Focus: Populate the intro_related_work_plan key.
Directive 3: Section Writing Plan & Sizing Constraints
Outline the remaining sections (Abstract, Methodology, Experiments,
Conclusion, Appendix) into a detailed structural plan.
- Structural Hierarchy: If Subsection X.1 is created, X.2 is mandatory.
Do not create orphaned subsections. Omit subsections entirely if a
section does not require division.
- Content Specificity: Explicitly reference source materials.
- Avoid: "Describe the model."
- Require: "Formalize the Temporal-Aware Attention mechanism using
Eq. 3 from idea.md."
- Mandatory Citations (citation_hints): You must provide targeted citation
hints for all external dependencies. Every hint must point to a single,
unambiguous canonical paper.
- Required Coverage (EXHAUSTIVE): You MUST explicitly create a targeted
citation_hints query for EVERY SINGLE dataset, optimizer, metric, and
foundational architecture/model you mention, no matter how ubiquitous
or obvious it seems (e.g., AdamW, ResNet, ImageNet, CLIP, Transformer,
LLaMA, GPT, LLaVA). If it is in the experimental_log.md or idea.md,
it MUST have a citation hint.
1. All baseline methods compared against.
2. All datasets evaluated on.
3. All standard metrics utilized.
4. All foundational algorithms (e.g., ResNet, Transformer, Diffusion
models), foundational models (e.g., LLMs, VLMs), optimizers
(e.g., AdamW), or frameworks built upon.
- Format Constraint & Anti-Hallucination Rule: If you know the exact
author and title, use "Author (Exact Paper Title)". DO NOT guess or
hallucinate authors. If you do not know the exact author, use this
format: "research paper or technical report introducing '[Exact
Model/Dataset/Metric Name]'".
- Output Focus: Populate the section_plan key.
Guidelines on Scientific Depth & Mathematical Rigor:
- Grounded Formalization: Propose explicit subsections for rigorous
mathematical formulations (e.g., loss functions, core algorithms,
theoretical proofs). You must base these strictly on idea.md and
experimental_log.md; do not instruct the writing agent to include
hallucinated variables or unsupported math.
Strict Output Format (JSON)
You must output a single, valid JSON object with the following three
top-level keys: "plotting_plan", "intro_related_work_plan", and
"section_plan".The full example output JSON from the paper (App. F.1, pp. 43–44) is at example-output.json.
#!/usr/bin/env python3
"""
validate_outline.py — Validate workspace/outline.json against the JSON Schema.
Performs the structural checks defined in
`references/outline_schema.json` plus a few semantic checks the schema
cannot express:
- figure_id snake_case + no "figure" infix
- "no orphaned subsection" hierarchy rule
- related_work_strategy.subsections has 2-4 entries (paper rule)
- emits WARNINGS for missing-but-recommended things, not just ERRORS
Exit codes:
0 valid (warnings allowed)
1 schema or semantic errors
Usage:
python validate_outline.py /path/to/outline.json
"""
import json
import os
import re
import sys
try:
import jsonschema
from jsonschema import Draft202012Validator
except ImportError:
print("ERROR: jsonschema is required. Install with: pip install jsonschema",
file=sys.stderr)
sys.exit(2)
HERE = os.path.dirname(os.path.abspath(__file__))
SCHEMA_PATH = os.path.normpath(
os.path.join(HERE, "..", "references", "outline_schema.json")
)
FIGID_RE = re.compile(r"^[a-z0-9_]+$")
def load_schema() -> dict:
with open(SCHEMA_PATH) as f:
return json.load(f)
def semantic_checks(outline: dict) -> tuple[list[str], list[str]]:
"""Return (errors, warnings)."""
errors: list[str] = []
warnings: list[str] = []
# Figure ID checks
for i, fig in enumerate(outline.get("plotting_plan", [])):
fid = fig.get("figure_id", "")
if not FIGID_RE.match(fid):
errors.append(
f"plotting_plan[{i}].figure_id={fid!r} is not snake_case "
f"matching ^[a-z0-9_]+$"
)
# The word "figure" must not appear (allow the conventional fig_ prefix)
body = re.sub(r"^fig_", "", fid)
if "figure" in body.lower():
errors.append(
f"plotting_plan[{i}].figure_id={fid!r} contains 'figure' — "
f"forbidden by Outline Agent prompt (App. F.1)"
)
# If plot_type is "plot", objective should mention a specific chart type
if fig.get("plot_type") == "plot":
obj = fig.get("objective", "").lower()
chart_keywords = [
"radar", "bar", "line", "scatter", "box", "violin",
"heatmap", "histogram", "pie", "stacked", "grouped",
"ridge", "density", "convergence", "training curve",
]
if not any(k in obj for k in chart_keywords):
warnings.append(
f"plotting_plan[{i}] is plot_type='plot' but objective "
f"does not name a specific chart type. The Outline Agent "
f"prompt requires this."
)
# Section hierarchy: no orphan subsections
for i, section in enumerate(outline.get("section_plan", [])):
subs = section.get("subsections") or []
if len(subs) == 1:
warnings.append(
f"section_plan[{i}] {section.get('section_title')!r} has only "
f"one subsection — this is an orphan. The Outline Agent prompt "
f"requires that if X.1 exists, X.2 must too."
)
return errors, warnings
def main() -> int:
if len(sys.argv) != 2:
print(__doc__, file=sys.stderr)
return 2
outline_path = sys.argv[1]
if not os.path.isfile(outline_path):
print(f"ERROR: {outline_path} not found", file=sys.stderr)
return 1
try:
with open(outline_path) as f:
outline = json.load(f)
except json.JSONDecodeError as e:
print(f"ERROR: {outline_path} is not valid JSON: {e}", file=sys.stderr)
return 1
schema = load_schema()
validator = Draft202012Validator(schema)
schema_errors = sorted(validator.iter_errors(outline), key=lambda e: e.path)
sem_errors, sem_warnings = semantic_checks(outline)
if schema_errors:
print("SCHEMA ERRORS:", file=sys.stderr)
for e in schema_errors:
path = "/".join(str(p) for p in e.absolute_path) or "(root)"
print(f" {path}: {e.message}", file=sys.stderr)
if sem_errors:
print("SEMANTIC ERRORS:", file=sys.stderr)
for msg in sem_errors:
print(f" {msg}", file=sys.stderr)
if sem_warnings:
print("WARNINGS:")
for msg in sem_warnings:
print(f" {msg}")
if schema_errors or sem_errors:
print(f"\nFAIL: {outline_path}", file=sys.stderr)
return 1
n_figs = len(outline.get("plotting_plan", []))
n_relwork = len(
outline.get("intro_related_work_plan", {})
.get("related_work_strategy", {})
.get("subsections", [])
)
n_sections = len(outline.get("section_plan", []))
n_hints = sum(
len(sub.get("citation_hints", []))
for sec in outline.get("section_plan", [])
for sub in (sec.get("subsections") or [])
)
print(
f"OK: {outline_path} — {n_figs} figures, "
f"{n_relwork} related-work clusters, {n_sections} sections, "
f"{n_hints} citation hints"
)
return 0
if __name__ == "__main__":
sys.exit(main())