Learning Aggregator Ci

Learning Aggregator CI

Install

gh skill install pskoett/pskoett-skills learning-aggregator-ci

For interactive sessions, use:

gh skill install pskoett/pskoett-skills learning-aggregator

Fallback using the Agent Skills CLI:

npx skills add pskoett/pskoett-skills/skills/learning-aggregator-ci
npx skills add pskoett/pskoett-skills/skills/learning-aggregator

Purpose

Runs the outer loop's inspect step in CI. Reads accumulated .learnings/ files, groups entries by pattern_key, computes cross-session recurrence, and produces a ranked gap report — all without human interaction.

The interactive learning-aggregator skill is designed for in-session use where the user can review and act on findings immediately. This CI variant runs on a schedule (weekly, per-sprint, or on-demand) and posts its findings as a GitHub issue comment for async review.

Context Limitation (Important)

CI agents do not have session context. They cannot see what the user is currently working on or what task area is relevant. The CI variant scans all .learnings/ entries without relevance filtering. The gap report is comprehensive rather than targeted.

Prerequisites

• GitHub Actions enabled on the repository
• gh CLI authenticated with repo access
• gh-aw extension installed (gh extension install github/gh-aw, v0.40.1+)
• .learnings/ directory with structured entries from self-improvement

CI Contract

Hard rules for headless execution:

1. Read-only — do not modify .learnings/ files, project instruction files (CLAUDE.md, AGENTS.md, .github/copilot-instructions.md), or any repo files 2. Headless — no interactive prompts, no approval gates 3. Structured output — emit findings as YAML under learning_aggregator_ci key 4. Single comment — post one consolidated comment per run, not per finding 5. Deterministic — same .learnings/ state produces the same gap report

Authoring Workflow (gh-aw)

1. Copy references/workflow-example.md into .github/workflows/learning-aggregator-ci.md 2. Customize the schedule for your cadence (supports fuzzy schedules like weekly on mondays) 3. Validate: gh aw compile (optionally add --actionlint --zizmor for full security scan) 4. Push to enable

Persistence and Chaining

• `cache-memory:` stores aggregation state (pattern groups, recurrence counts) across runs. Survives up to 90 days in Actions cache. Avoids re-scanning unchanged entries on every run.
• `call-workflow:` triggers eval-creator-ci after aggregation completes to create evals from newly promoted patterns. Compile-time fan-out with proper dependency wiring.
• `upload-artifact:` persists the gap report YAML for consumption by downstream workflows or human review.

Workflow Rules

The CI agent follows these rules in order:

1. Read all files in .learnings/: LEARNINGS.md, ERRORS.md, FEATURE_REQUESTS.md, HEALS.md 2. Parse each entry's metadata: Pattern-Key, Recurrence-Count, First-Seen, Last-Seen, Priority, Status, Area, Related Files, Tags. For HEAL entries, also parse Trigger, Active-Context, and any Handoff block — Handoff blocks at the promotion threshold are promotion-ready by definition and must appear in the gap report 3. Group entries by Pattern-Key (exact match only — no fuzzy grouping in CI) 4. For each group: sum recurrences, count distinct tasks, compute time window, collect evidence 5. Flag entries without Pattern-Key as ungrouped 6. Classify each group's gap type: knowledge gap, tool gap, skill gap, ambiguity, or reasoning failure 7. Rank groups by: promotion-ready first, then approaching threshold, then by priority (critical > high > medium > low) 8. Emit structured YAML under key learning_aggregator_ci 9. Post gap report as a comment on the triggering issue or as a new issue if running on schedule 10. Do not modify repository files

Promotion threshold (same rule as learning-aggregator and self-improvement): a group is promotion-ready when Recurrence-Count >= 3, seen in >= 2 distinct tasks, within a 30-day window.

Output Schema

learning_aggregator_ci:
  version: "0.1.0"
  source:
    run_id: "<workflow run ID>"
    trigger: "schedule | workflow_dispatch | issue_comment"
    scan_date: "YYYY-MM-DD"
  scan:
    entries_total: 42
    entries_with_pattern_key: 35
    entries_ungrouped: 7
    patterns_found: 18
    promotion_ready: 3
    approaching_threshold: 5
  promotion_ready:
    - pattern_key: "harden.input_validation"
      recurrence_count: 5
      distinct_tasks: 3
      window_days: 21
      priority: "high"
      gap_type: "knowledge_gap"
      area: "backend"
      evidence:
        - "LRN-20260301-001: Missing bounds check on pagination params"
        - "ERR-20260308-002: Unconstrained string length caused OOM"
        - "LRN-20260315-003: API params not validated before DB query"
      recommended_action: "Add to project instruction files: Always validate and bound-check external inputs before use"
      eval_candidate: true
  approaching:
    - pattern_key: "simplify.dead_code"
      recurrence_count: 2
      distinct_tasks: 1
      priority: "low"
      needs: "1 more distinct task"
  ungrouped:
    - id: "LRN-20260320-005"
      summary: "Discovered undocumented rate limit on external API"
      recommendation: "Assign pattern_key for future tracking"
  stale:
    - pattern_key: "harden.error_handling"
      last_seen: "2025-12-01"
      recommendation: "Dismiss — not seen in 90+ days"
  summary:
    promotion_ready_total: 3
    approaching_total: 5
    ungrouped_total: 7
    stale_total: 1
    followup_required: true

Recommended Outputs

Trigger Configuration

Recommended: weekly schedule + manual dispatch

on:
  schedule:
    - cron: '0 9 * * 1'  # Monday 9am UTC
  workflow_dispatch:
  issue_comment:
    types: [created]

The schedule ensures regular outer-loop cadence. Manual dispatch allows on-demand runs after incidents or sprints. Issue comment trigger allows /aggregate-learnings commands.

Integration with Other Skills

Upstream (feeds from)

• self-improvement (interactive) — produces .learnings/LEARNINGS.md, ERRORS.md, FEATURE_REQUESTS.md entries
• self-healing / self-healing-ci — produce .learnings/HEALS.md entries including Handoff blocks
• self-improvement-ci — emits learning candidates as machine-readable output (artifacts/comments); it is read-only and does not write .learnings/ files itself
• simplify-and-harden-ci — produces learning_loop.candidates consumed by self-improvement-ci

Downstream (feeds into)

• harness-updater (interactive) — takes promotion-ready patterns from the gap report and applies them
• eval-creator-ci — takes eval candidates and creates permanent test cases
• Human review — gap report posted as issue comment for team triage

Data Flow

self-improvement → .learnings/*.md   ←  self-healing(-ci) → HEALS.md
                       ↓
              learning-aggregator-ci (scheduled)
                       ↓
              gap report (issue comment + artifact)
                       ↓
              harness-updater (interactive, human-gated)
                       ↓
              eval-creator-ci (creates evals from promoted patterns)

Differences from Interactive Version

Workflow Example (Non-Active)

This is an example template only. Keep it outside .github/workflows so nothing runs automatically.

When you are ready to enable CI automation: 1. Copy this template into .github/workflows/learning-aggregator-ci.md 2. Customize the schedule for your team's cadence 3. Validate with gh aw compile (add --actionlint --zizmor for security scan)

---
on:
  schedule:
    - cron: '0 9 * * 1'
  workflow_dispatch:
  issue_comment:
    types: [created]

permissions:
  contents: read
  actions: read
  issues: read
  pull-requests: read

tools:
  github:
    toolsets: [pull_requests, actions, issues]
  cache-memory: true
  # Optional: durable git-branch persistence for the aggregation state.
  # Survives beyond the 7-day cache-memory window and is readable from interactive
  # sessions via `git fetch origin learnings/default` (branch name = branch-prefix/default).
  repo-memory:
    branch-prefix: learnings
    max-file-size: 51200  # 50KB — aggregation state can be larger than default 10KB

safe-outputs:
  add-comment:
    max: 1
    hide-older-comments: true
  upload-artifact:
    max-uploads: 1
  call-workflow:
    workflows: [eval-creator-ci]
    max: 1

tracker-id: learning-aggregator

concurrency:
  group: learning-aggregator
  cancel-in-progress: false

strict: true
---

1. Read all files in `.learnings/` directory: `LEARNINGS.md`, `ERRORS.md`, `FEATURE_REQUESTS.md`. If the directory does not exist or is empty, report zero findings and exit.

2. Check cache-memory at `/tmp/gh-aw/cache-memory/learning-aggregator-state.json` for previous aggregation state. If found, load prior pattern groups and recurrence counts as a baseline. Only re-process entries with `Last-Seen` newer than the cached scan date.

3. Parse each entry's structured metadata fields: `Pattern-Key`, `Recurrence-Count`, `First-Seen`, `Last-Seen`, `Priority`, `Status`, `Area`, `Related Files`, `Source`, `Tags`.

4. Group entries by exact `Pattern-Key` match. Do not attempt fuzzy grouping — false positives are worse than ungrouped entries in CI.

5. For each group: sum `Recurrence-Count` across entries, count distinct task references, compute the time window between earliest `First-Seen` and latest `Last-Seen`, collect all evidence summaries.

6. Identify promotion-ready patterns: `Recurrence-Count >= 3` AND `distinct tasks >= 2` AND within a `30-day window`.

7. Identify approaching patterns: `Recurrence-Count >= 2` OR `Priority: high/critical` with any recurrence.

8. Flag entries without `Pattern-Key` as ungrouped with a recommendation to assign one.

9. Flag entries with `Last-Seen` older than 90 days as stale with a recommendation to dismiss.

10. Classify each promotion-ready pattern's gap type: knowledge gap (agent didn't know), tool gap (agent improvised), skill gap (same behavior fails), ambiguity (conflicting interpretations), reasoning failure (agent had knowledge but reasoned wrong).

11. Write updated aggregation state to cache-memory at `/tmp/gh-aw/cache-memory/learning-aggregator-state.json` for the next run.

12. Emit the full gap report as structured YAML under key `learning_aggregator_ci` following the output schema in the skill definition.

13. Upload the gap report YAML as a workflow artifact named `gap-report`.

14. Post a human-readable summary as a comment. Format: promotion-ready patterns first (with evidence and recommended action), then approaching patterns, then ungrouped entries, then stale entries.

15. If any promotion-ready patterns have `eval_candidate: true`, trigger `eval-creator-ci` via call-workflow to create eval cases from the newly promoted patterns.

16. Do not modify any repository files. This workflow is read-only.