Codex Retrospective

Codex Retrospective

A structured self-improvement loop for Codex.

This skill turns Codex from a one-off collaborator into a system that gets meaningfully better at working with you over time by systematically eating its own usage history.

Why This Exists

Most people improve their agent usage by manually maintaining AGENTS.md or skills when they notice problems. This skill makes that process deliberate, regular, and high-leverage.

It is directly inspired by strong practices from heavy users (especially Greg Brockman's emphasis on updating the "constitution" from real failures and friction), but executed as a repeatable arsenal-style workflow.

Core Principle

Minimal effective change, grounded in evidence from actual history.

Every output must be:

• The smallest possible useful addition or edit
• Backed by specific patterns observed in recent sessions
• Designed to prevent repetition of the same friction

When to Trigger

• After a painful or repetitive session ("Codex should have known better by now")
• Periodically (weekly or monthly): "Do a retrospective on the last 7/30 days of my Codex usage"
• When codex-fluent reports that Codex is constantly re-asking for the same context or preferences
• After you have manually corrected the same class of mistake multiple times

The Retrospective Process

1. Scope Definition

You specify the time window or focus area:

• "Last 14 days"
• "The three big auth + payments threads from last month"
• "All sessions involving the new Go service"

2. Evidence Collection Preflight

Before proposing any AGENTS.md update or tiny skill, Codex must build a small evidence inventory:

• Scope: repo/project, time window, and focus area.
• Sources checked: current AGENTS.md, relevant existing skills, repo docs, Memories/Chronicle, and local Codex session files when available.
• Evidence handles: session IDs, memory IDs, file paths, PR/issue numbers, or dated incidents that can be checked later.
• Runtime availability: distinguish "skill exists in source" from "skill is installed and triggerable in the active runtime" when that matters.
• No-change path: if evidence is weak, stop with a concise "no high-confidence change" result instead of manufacturing proposals.

Every proposed update must map back to at least one concrete evidence handle.

3. History Analysis (The Skill Guides Codex)

Codex is instructed to look for:

• Recurring mistakes or inefficient patterns
• Things the user had to explain or correct repeatedly
• High-friction moments (lots of back-and-forth, context loss, wrong assumptions)
• Successful patterns worth encoding so they happen by default
• Opportunities to extract tiny, reusable skills

4. Output — Strict Format

The skill forces Codex to produce output in this order:

0. Evidence Inventory (scope, sources checked, and concrete handles) 1. Retrospective Summary (short, evidence-based) 2. Proposed AGENTS.md Updates (exact diff or append text only) 3. New or Refined Minimal Skills (at most 1-2 tiny ones, full SKILL.md with frontmatter) 4. Rationale + Evidence (which sessions/patterns drove each proposal) 5. Application Plan (how to safely apply the changes)

5. Human Gate + Application

You review. The skill then helps you apply the minimal changes cleanly (never blindly overwriting large sections of AGENTS.md).

Relationship with codex-fluent

These two skills are designed to be used together:

• codex-retrospective finds behavioral and knowledge improvements (better defaults, new rules, extracted skills).
• codex-fluent finds state and context hygiene improvements (session bloat, missing handoffs, archive opportunities).

A good monthly ritual for serious users: 1. Run codex-retrospective on the last 30 days. 2. Run codex-fluent diagnosis. 3. Apply the best changes from both.

Hard Constraints on Output

• Never propose large rewrites of AGENTS.md.
• Never create big new skills. Tiny, focused, high-ROI only.
• Every proposal must reference concrete history ("In the payment retry thread on May 3rd and the similar incident on May 18th...").
• Do not propose changes until the evidence inventory exists and each proposal maps to a concrete handle.
• If nothing high-confidence was found, say so clearly instead of manufacturing improvements.

Gotchas

• Do not turn a single annoying session into a permanent rule unless it was

unusually high impact. Prefer "no change" over a weak constitution update.

• Do not rely on memory vibes. Each proposed update needs a concrete session,

date range, file, PR, or repeated correction pattern that can be checked.

• Keep generated AGENTS.md text short. If the proposed rule needs multiple

paragraphs, it probably belongs in a tiny skill or a project doc instead.

• Application is gated by the user. The retrospective can propose exact diffs,

but it must not silently rewrite AGENTS.md or existing skills.

References

• references/retrospective-prompt.md — The core prompt template used to drive Codex's self-analysis
• references/agents-md-update-rules.md — Strict rules for what kind of changes are acceptable
• references/minimal-skill-criteria.md — What qualifies as a "tiny useful skill" worth extracting
• references/examples/ — Real (sanitized) retrospective outputs and the resulting AGENTS.md diffs

Success Looks Like

After 4–8 weeks of regular use:

• Codex makes fewer basic assumption errors in your domain
• You spend less time re-explaining preferences and constraints
• Your AGENTS.md and skills folder feel like they were written by someone who has worked with you for a long time (because they were)
• New projects ramp up faster because the constitution already encodes hard-won lessons

Start with a focused 7- or 14-day retrospective on a project where you've felt the most friction recently. The pattern will become natural quickly.

Rules for Updating AGENTS.md via Retrospective

These rules are enforced by the codex-retrospective skill.

Acceptable Changes (Only These)

• Adding a short, specific "do X instead of Y" rule that directly addresses a repeated mistake.
• Adding or tightening a constraint in an existing section.
• Adding a small "preferred pattern" example (2–4 lines max) when the same pattern has been manually corrected multiple times.
• Updating the "Common Friction Points" or equivalent section with a newly observed pattern + mitigation.
• Adding a one-line pointer to a new tiny skill.

Forbidden Changes

• Large rewrites or reorganizations of AGENTS.md.
• Adding vague or aspirational principles ("be more careful", "think harder").
• Adding rules based on single incidents (needs repetition or very high impact).
• Creating long lists of "don'ts".
• Changing the overall structure or voice of the document without strong justification.

Quality Bar

A good retrospective update should make the next similar task noticeably smoother for both you and the user, with very little added cognitive load when reading AGENTS.md.

If a proposed change would make AGENTS.md feel heavier or more bureaucratic, reject it.

After Applying

The user should be able to point to a specific recent painful session and say "because of this line we added, that would have gone better."

If they can't, the change was probably too weak or too broad.

Example of a Good Retrospective Output (Sanitized)

Window: Last 18 days, focused on the new Go service work.

Evidence Inventory

• Scope: new Go service work, last 18 days.
• Sources checked: current AGENTS.md, service-specific skills, local session summaries, and three Codex session files.
• Evidence handles: sessions from May 4, May 9, May 12, and May 15; AGENTS.md "Core Implementation Principles"; service money-movement skill.

Retrospective Summary

• Repeatedly started implementation before fully clarifying the exact failure modes the user cared about most (happened in at least 3 sessions).
• Strong pattern: when the user said "make it robust", Codex defaulted to broad retries instead of the specific idempotency + compensation model preferred here.
• User had to re-explain the "Money type + no raw amounts" rule twice in different files.
• Excellent results when Codex first wrote a tiny state machine sketch before coding — this pattern is worth encoding.

Proposed AGENTS.md Updates

Location: Under "Core Implementation Principles" (new bullet)

+ - When the user asks for something to be "robust", "safe", or "production ready" in this service, first explicitly list the 2-4 concrete failure modes we are optimizing for before writing any code. Do not assume broad retries or circuit breakers are the answer.

Proposed Tiny Skill

File: skills/go-service-idempotency.md

(Full small skill with frontmatter for handling money movements with explicit compensation. Omitted here for brevity — it was 2 focused sections + 1 example.)

Rationale

The three sessions on May 4, May 9, and May 15 all had the same pattern: Codex jumped into "add retry + backoff" while the actual requirement was explicit ledger entries + compensation handlers. The Money type issue appeared in the May 9 and May 15 threads.

The state machine sketch technique from the May 12 session produced the cleanest result with the least back-and-forth.

Application Plan

1. Apply the AGENTS.md addition first (low risk). 2. Review the extracted skill together. 3. In the next similar task, explicitly invoke the new skill when relevant. 4. Re-run a short retrospective in 10 days focused only on this area to measure improvement.

Criteria for Extracting a Tiny Skill During Retrospective

The bar for creating a new skill file from a retrospective is intentionally high.

Must Meet All of These

• The pattern appeared in at least two meaningfully different sessions in the review window (or one extremely high-cost incident).
• Encoding it as a small skill would have saved the user noticeable time, context, or corrections.
• The skill is small and focused (ideally under 80–100 lines when first created).
• It has a clear, natural trigger phrase the user is likely to say or that Codex can reliably recognize.
• It is more valuable as a standalone file than as a short section in AGENTS.md.

Should Usually Be Rejected If

• It only saves one or two minor steps.
• It is very specific to a single project or temporary situation.
• It duplicates logic that already exists in AGENTS.md or another skill.
• It would require frequent updates (better to keep in AGENTS.md for now).
• The main value is "reminding Codex of something" rather than providing a reusable procedure or checklist.

Good Examples of Tiny Skills Worth Extracting

• A specific debugging ritual for a tricky class of production incidents in one service.
• A strict code review checklist for security-sensitive changes that the team cares about.
• A reliable way to produce high-quality handoff documents for a particular type of long-running work.
• A pattern for safely introducing a new database migration in this specific stack.

Anti-Examples

• "Be more careful with error handling" → too vague.
• A 300-line "perfect Go web service generator" → way too big for a retrospective extraction.
• Something that only applied to last week's incident.

Final Test

After writing the tiny skill, ask: "If I had this skill two weeks ago, would at least one of the painful sessions in this retrospective have gone clearly better?"

If the honest answer is no or "maybe a little", do not create the file. Put the insight in AGENTS.md instead.

Codex Retrospective Prompt Template

This is the core prompt used by the codex-retrospective skill. It is deliberately strict.

---

You are performing a high-signal retrospective on your own recent usage history as Codex.

Time window: [user specifies, e.g. last 14 days / the auth and payments work from May]

Your goal: Make yourself meaningfully better at working with this user by identifying the smallest number of high-leverage changes to AGENTS.md and/or tiny new skills.

Strict rules you must follow:

1. Only propose changes when you have clear, repeated evidence from the history. 2. Every proposal must be the minimal useful intervention. 3. Prefer updating or slightly extending existing rules over adding new ones. 4. Only extract a new skill if the pattern is genuinely reusable and would have saved significant time/friction in multiple places. 5. If you cannot find strong evidence for a useful change, say so honestly. 6. Do not propose changes until you have listed the evidence sources checked and concrete handles for the evidence you rely on.

Process:

First, define the scope and build an evidence inventory:

• Scope: repo/project, time window, and focus area.
• Sources checked: current AGENTS.md, relevant existing skills, repo docs, Memories/Chronicle, and local Codex session files when available.
• Evidence handles: session IDs, memory IDs, file paths, PR/issue numbers, or dated incidents that can be checked later.
• Runtime availability: distinguish "skill exists in source" from "skill is installed and triggerable in the active runtime" when that matters.

Then deeply review the relevant session history, any available Memories or Chronicle, and the current AGENTS.md + skills.

Then output exactly in this structure:

0. Evidence Inventory

• Scope reviewed.
• Sources checked.
• Concrete evidence handles used for the proposals.
• Any source/runtime availability distinctions that matter.

If this section cannot list enough evidence for a high-confidence change, stop after section 1 with a clear no-change recommendation.

1. Retrospective Summary

• 3–6 bullet points of the most important patterns observed (both problems and strong successes).
• For each point, reference specific sessions or time periods.

2. Proposed AGENTS.md Updates

Only include this section if you have high-confidence minimal changes.

For each change:

• Exact location (or "append at the end of section X")
• The precise text to add or replace (as a clean diff or quoted block)
• One-sentence rationale tied to observed history

3. Proposed New or Updated Tiny Skills (0–2 maximum)

For each:

• Suggested filename (e.g. skills/codex-payment-idempotency.md)
• Complete YAML frontmatter + content
• Why this tiny skill is worth the cost of another file

4. Evidence & Rationale

For every proposal above, point to the concrete patterns in history that justify it. Map every proposed AGENTS.md update or tiny skill to at least one evidence handle from section 0.

5. Application Recommendation

• Suggested order of applying changes
• Any verification steps the user should do afterward
• Whether a follow-up retrospective in 2 weeks would be useful

Remember: Your future self (and the user) will thank you for ruthless minimalism. Large or speculative changes are worse than no change.