Self Improving Agent

Self-Improving Agent

"An AI agent that learns from every interaction, accumulating patterns and insights to continuously improve its own capabilities." — Based on 2025 lifelong learning research

Overview

This is a universal self-improvement system that learns from ALL skill experiences, not just PRDs. It implements a complete feedback loop with:

• Multi-Memory Architecture: Semantic + Episodic + Working memory
• Self-Correction: Detects and fixes skill guidance errors
• Self-Validation: Periodically verifies skill accuracy
• Hooks Integration: Auto-triggers on skill events (before_start, after_complete, on_error)
• Evolution Markers: Traceable changes with source attribution

Research-Based Design

Based on 2025 research:

The Self-Improvement Loop

┌─────────────────────────────────────────────────────────────────┐
│                    UNIVERSAL SELF-IMPROVEMENT                    │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│   Skill Event → Extract Experience → Abstract Pattern → Update  │
│        │                  │                │         │          │
│        ▼                  ▼                ▼         ▼          │
│   ┌─────────────────────────────────────────────────────┐       │
│   │              MULTI-MEMORY SYSTEM                      │       │
│   ├─────────────────────────────────────────────────────┤       │
│   │  Semantic Memory   │  Episodic Memory  │ Working Memory │  │
│   │  (Patterns/Rules)  │  (Experiences)    │  (Current)     │  │
│   │  memory/semantic/  │  memory/episodic/ │  memory/working/│  │
│   └─────────────────────────────────────────────────────┘       │
│                                                                 │
│   ┌─────────────────────────────────────────────────────┐       │
│   │              FEEDBACK LOOP                            │       │
│   │  User Feedback → Confidence Update → Pattern Adapt   │       │
│   └─────────────────────────────────────────────────────┘       │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

When This Activates

Automatic Triggers (via hooks)

Manual Triggers

• User says "自我进化", "self-improve", "从经验中学习"
• User says "分析今天的经验", "总结教训"
• User asks to improve a specific skill

Evolution Priority Matrix

Trigger evolution when new reusable knowledge appears:

Multi-Memory Architecture

1. Semantic Memory (memory/semantic-patterns.json)

Stores abstract patterns and rules reusable across contexts:

{
  "patterns": {
    "pattern_id": {
      "id": "pat-2025-01-11-001",
      "name": "Pattern Name",
      "source": "user_feedback|implementation_review|retrospective",
      "confidence": 0.95,
      "applications": 5,
      "created": "2025-01-11",
      "category": "prd_structure|react_patterns|async_patterns|...",
      "pattern": "One-line summary",
      "problem": "What problem does this solve?",
      "solution": { ... },
      "quality_rules": [ ... ],
      "target_skills": [ ... ]
    }
  }
}

2. Episodic Memory (memory/episodic/)

Stores specific experiences and what happened:

memory/episodic/
├── 2025/
│   ├── 2025-01-11-prd-creation.json
│   ├── 2025-01-11-debug-session.json
│   └── 2025-01-12-refactoring.json

{
  "id": "ep-2025-01-11-001",
  "timestamp": "2025-01-11T10:30:00Z",
  "skill": "debugger",
  "situation": "User reported data not refreshing after form submission",
  "root_cause": "Empty callback in onRefresh prop",
  "solution": "Implement actual refresh logic in callback",
  "lesson": "Always verify callbacks are not empty functions",
  "related_pattern": "callback_verification",
  "user_feedback": {
    "rating": 8,
    "comments": "This was exactly the issue"
  }
}

3. Working Memory (memory/working/)

Stores current session context:

memory/working/
├── current_session.json   # Active session data
├── last_error.json        # Error context for self-correction
└── session_end.json       # Session end marker

Self-Improvement Process

Phase 1: Experience Extraction

After any skill completes, extract:

What happened:
  skill_used: {which skill}
  task: {what was being done}
  outcome: {success|partial|failure}

Key Insights:
  what_went_well: [what worked]
  what_went_wrong: [what didn't work]
  root_cause: {underlying issue if applicable}

User Feedback:
  rating: {1-10 if provided}
  comments: {specific feedback}

Phase 2: Pattern Abstraction

Convert experiences to reusable patterns:

Abstraction Rules:

If experience_repeats 3+ times:
  pattern_level: critical
  action: Add to skill's "Critical Mistakes" section

If solution_was_effective:
  pattern_level: best_practice
  action: Add to skill's "Best Practices" section

If user_rating >= 7:
  pattern_level: strength
  action: Reinforce this approach

If user_rating <= 4:
  pattern_level: weakness
  action: Add to "What to Avoid" section

Phase 3: Skill Updates

Update the appropriate skill files with evolution markers:

<!-- Evolution: 2025-01-12 | source: ep-2025-01-12-001 | skill: debugger -->

## Pattern Added (2025-01-12)

**Pattern**: Always verify callbacks are not empty functions

**Source**: Episode ep-2025-01-12-001

**Confidence**: 0.95

### Updated Checklist
- [ ] Verify all callbacks have implementations
- [ ] Test callback execution paths

Correction Markers (when fixing wrong guidance):

<!-- Correction: 2025-01-12 | was: "Use callback chain" | reason: caused stale refresh -->

## Corrected Guidance

Use direct state monitoring instead of callback chains:

// ✅ Do: Direct state monitoring const prevPendingCount = usePrevious(pendingCount);

Phase 4: Memory Consolidation

1. Update semantic memory (memory/semantic-patterns.json) 2. Store episodic memory (memory/episodic/YYYY-MM-DD-{skill}.json) 3. Update pattern confidence based on applications/feedback 4. Prune outdated patterns (low confidence, no recent applications)

Promotion Policy

Self-improvement has two separate jobs:

1. Capture facts, corrections, failed assumptions, and reusable patterns as memory or proposal artifacts. 2. Promote only validated patterns into SKILL.md, AGENTS.md, docs, or CLI behavior.

Default to capture-first. Promote a change only when one of these is true:

• The user explicitly asks to update a skill or repository instruction.
• The same pattern recurs across multiple episodes.
• A focused test or review proves the current guidance is wrong or incomplete.
• The change is low-risk documentation that preserves existing behavior and is clearly traceable.

Promotion targets:

Self-Correction (on_error hook)

Triggered when:

• Bash command returns non-zero exit code
• Tests fail after following skill guidance
• User reports the guidance produced incorrect results

Process:

## Self-Correction Workflow

1. Detect Error
   - Capture error context from working/last_error.json
   - Identify which skill guidance was followed

2. Verify Root Cause
   - Was the skill guidance incorrect?
   - Was the guidance misinterpreted?
   - Was the guidance incomplete?

3. Create Proposal
   - Write a proposal with evidence, affected skill names, and expected behavior
   - Add correction marker text in the proposal, not directly in the skill yet
   - Update related patterns in semantic memory with low initial confidence

4. Validate Fix
   - Test the corrected guidance
   - Ask user to verify

5. Promote
   - Apply the skill/doc/code change after validation or explicit approval
   - Keep the source episode/proposal id in the change note

Example:

<!-- Correction: 2025-01-12 | was: "useMemo for claimable ids" | reason: stale data at click time -->

## Self-Correction: Click-Time Computation

**Issue**: Using useMemo for claimable IDs caused stale data
**Fix**: Compute at click time for always-fresh data
**Pattern**: click_time_vs_open_time_computation

Self-Validation

Use the validation template in references/appendix.md when reviewing updates.

Hooks Integration

Runtime Trigger Source

agent-playbook self-improve reads skill chaining from each skill's SKILL.md frontmatter:

metadata:
  hooks:
    after_complete:
      - trigger: self-improving-agent
        mode: background
        reason: "Extract patterns"

Treat metadata.hooks as the source of truth. Do not maintain a second hardcoded hook map in runtime code. This keeps skill behavior auditable and lets Skill Creator style reviews inspect the same file that the agent executes.

Wiring Hooks in Claude Code Settings

For Claude Code, install hooks through agent-playbook init --hooks when possible. If you need manual setup, add hook entries to Claude Code settings at the appropriate user or project scope.

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash|Write|Edit",
        "hooks": [
          {
            "type": "command",
            "command": "bash ${SKILLS_DIR}/self-improving-agent/hooks/pre-tool.sh \"$TOOL_NAME\" \"$TOOL_INPUT\""
          }
        ]
      }
    ],
    "PostToolUse": [
      {
        "matcher": "Bash",
        "hooks": [
          {
            "type": "command",
            "command": "bash ${SKILLS_DIR}/self-improving-agent/hooks/post-bash.sh \"$TOOL_OUTPUT\" \"$EXIT_CODE\""
          }
        ]
      }
    ],
    "Stop": [
      {
        "matcher": "",
        "hooks": [
          {
            "type": "command",
            "command": "bash ${SKILLS_DIR}/self-improving-agent/hooks/session-end.sh"
          }
        ]
      }
    ]
  }
}

Replace ${SKILLS_DIR} with your actual skills path.

Additional References

See references/appendix.md for memory structure, workflow diagrams, metrics, feedback templates, and research links.

Best Practices

DO

• ✅ Learn from EVERY skill interaction
• ✅ Extract patterns at the right abstraction level
• ✅ Update multiple related skills
• ✅ Track confidence and apply counts
• ✅ Ask for user feedback on improvements
• ✅ Use evolution/correction markers for traceability
• ✅ Validate guidance before applying broadly
• ✅ Write proposals before mutating durable skill guidance
• ✅ Keep hook routing in metadata.hooks

DON'T

• ❌ Over-generalize from single experiences
• ❌ Update skills without confidence tracking
• ❌ Ignore negative feedback
• ❌ Make changes that break existing functionality
• ❌ Create contradictory patterns
• ❌ Update skills without understanding context
• ❌ Silently promote self-improvement findings into repo rules
• ❌ Duplicate hook definitions in CLI code and skill frontmatter

Quick Start

After a high-signal skill workflow completes, this agent can:

1. Analyzes what happened 2. Extracts patterns and insights 3. Writes memory and proposal artifacts 4. Promotes validated improvements only when approval or evidence is sufficient 5. Reports summary to user

References

• SimpleMem: Efficient Lifelong Memory for LLM Agents
• A Survey on the Memory Mechanism of Large Language Model Agents
• Lifelong Learning of LLM based Agents
• Evo-Memory: DeepMind's Benchmark
• Let's Build a Self-Improving AI Agent
• OpenCrabs local self-improving agent
• ELL-StuLife experience-driven lifelong learning

Self-Improving Agent

A self-improvement system that captures learning artifacts from skill experiences and proposes validated updates.

Overview

This agent captures reusable evidence from skill interactions. It implements a feedback loop with memory artifacts, self-correction proposals, and evolution markers. Durable skill or code changes still require validation or explicit approval.

Key Features

• Multi-Memory Architecture: Semantic + Episodic + Working memory
• Evidence-Gated Learning: Captures reusable lessons from skill workflows
• Pattern Extraction: Converts experiences into reusable patterns
• Self-Correction: Fixes skill guidance when errors occur
• Self-Validation: Periodically verifies skill accuracy
• Proposal Artifacts: Writes proposed updates before durable skill changes
• Confidence Tracking: Measures pattern reliability over time
• Human-in-the-Loop: Collects feedback to validate improvements

Memory System

Current Claude Code hook integration writes to:

~/.claude/memory/
├── semantic/       # Patterns, rules, best practices
├── episodic/       # Specific experiences and episodes
└── working/        # Current session context

How It Works

Any Skill Completes
        ↓
Extract Experience → Identify Patterns → Write Proposals → Consolidate Memory
        ↓                     ↓                  ↓              ↓
   What happened?    What can we reuse?   Which proposals? Track metrics

Installation

apb skills add ./skills/self-improving-agent --scope global --target all --link

Hooks (Optional)

Wire hooks to capture errors and session-end signals:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash|Write|Edit",
        "hooks": [
          { "type": "command", "command": "bash ${SKILLS_DIR}/self-improving-agent/hooks/pre-tool.sh \"$TOOL_NAME\" \"$TOOL_INPUT\"" }
        ]
      }
    ],
    "PostToolUse": [
      {
        "matcher": "Bash",
        "hooks": [
          { "type": "command", "command": "bash ${SKILLS_DIR}/self-improving-agent/hooks/post-bash.sh \"$TOOL_OUTPUT\" \"$EXIT_CODE\"" }
        ]
      }
    ],
    "Stop": [
      {
        "matcher": "",
        "hooks": [
          { "type": "command", "command": "bash ${SKILLS_DIR}/self-improving-agent/hooks/session-end.sh" }
        ]
      }
    ]
  }
}

Triggering

Host-Supported Follow-up

When the host runtime supports hook follow-ups, this skill can be recorded or run after high-signal workflows such as:

• prd-planner
• code-reviewer
• debugger
• refactoring-specialist
• etc.

Manual

"自我进化"
"self-improve"
"分析今天的经验"
"总结这次教训"

Example Learning

Episode

Skill: debugger
Situation: Form submission doesn't refresh data
Root Cause: Empty callback function
Pattern: Always verify callbacks have implementations
Confidence: 0.95 → Proposals: debugger, prd-implementation-precheck

Skill Update

## Proposed Update (2025-01-11)

### Pattern Added
**Callback Verification**: Always verify that callback functions
passed as props are not empty and actually execute logic.

**Source**: Episode ep-2025-01-11-003 (3 occurrences)
**Action**: Propose adding to debugger checklist

Research Basis

• SimpleMem: Efficient Lifelong Memory
• ACM Memory Mechanisms Survey
• Lifelong Learning of LLM Agents

Templates

Reusable templates live in skills/self-improving-agent/templates:

• pattern-template.md
• correction-template.md
• validation-template.md

License

MIT

Appendix

Self-Validation

Validation Report Template

## Validation Report Template

**Date**: [YYYY-MM-DD]
**Scope**: [skill(s) validated]

### Checks
- [ ] Examples compile or run
- [ ] Checklists match current repo conventions
- [ ] External references still valid
- [ ] No duplicated or conflicting guidance

### Findings
- [Finding 1]
- [Finding 2]

### Actions
- [Action 1]
- [Action 2]

Memory File Structure

~/.claude/memory/
├── semantic/
│   └── patterns.json
├── episodic/
│   ├── 2025/
│   │   ├── 2025-01-11-prd-creation.json
│   │   └── 2025-01-11-debug-session.json
│   └── episodes.json
├── working/
│   ├── current_session.json
│   ├── last_error.json
│   └── session_end.json
└── index.json

Automatic Workflow Integration

Any Skill Run
  -> workflow-orchestrator
    -> self-improving-agent (background)
    -> create-pr (ask_first)
    -> session-logger (auto)

Continuous Learning Metrics

{
  "metrics": {
    "patterns_learned": 47,
    "patterns_applied": 238,
    "skills_updated": 12,
    "avg_confidence": 0.87,
    "user_satisfaction_trend": "improving",
    "error_rate_reduction": "-35%",
    "self_corrections": 8
  }
}

Human-in-the-Loop

Feedback Collection

## Self-Improvement Summary

I've learned from our session and updated:

### Updated Skills
- `debugger`: Added callback verification pattern
- `prd-planner`: Enhanced UI/UX specification requirements

### Patterns Extracted
1. **state_monitoring_over_callbacks**: Use usePrevious for state-driven side effects
2. **ui_ux_specification_granularity**: Explicit visual specs prevent rework

### Confidence Levels
- New patterns: 0.85 (needs validation)
- Reinforced patterns: 0.95 (well-established)

### Your Feedback
Rate these improvements (1-10):
- Were the updates helpful?
- Should I apply this pattern more broadly?
- Any corrections needed?

Feedback Integration

User Feedback:
  positive (rating >= 7):
    action: Increase pattern confidence
    scope: Expand to related skills

  neutral (rating 4-6):
    action: Keep pattern, gather more data
    scope: Current skill only

  negative (rating <= 3):
    action: Decrease confidence, revise pattern
    scope: Remove from active patterns

Templates

References

• SimpleMem: Efficient Lifelong Memory for LLM Agents
• A Survey on the Memory Mechanism of Large Language Model Agents
• Lifelong Learning of LLM based Agents
• Evo-Memory: DeepMind's Benchmark
• Let's Build a Self-Improving AI Agent

Correction Template

Issue Summary

Previous Guidance

Corrected Guidance

Root Cause

Follow-up Actions

Pattern Template

Pattern Name

Context

• Source skill:
• Situation:

Guidance

Examples

Confidence

• Initial confidence:
• Validation notes:

Validation Template

Date

Scope

Checks

• [ ] Examples compile or run
• [ ] Guidance matches current repo conventions
• [ ] External references still valid

Findings

Actions