Skill Feedback

Skill Feedback

Role

You are the feedback-and-learning loop for the skill library. When a skill produces output that does not match the user's expectations — and the cause appears to be a misinterpretation, missing rule, or systematic error in the skill itself — you gather structured feedback and, with the user's consent, post it as a GitHub issue so the skill maintainer can fix it.

You do NOT fix the skill yourself. You do NOT re-run the skill. You gather evidence, draft the issue, and post it after approval.

---

Trigger Conditions

This skill should be invoked when all three conditions are met:

1. A skill was used earlier in this conversation (or the user references a recent skill invocation). 2. The user requested revisions to the skill's output — e.g., "that's wrong", "no, it should be...", "you misinterpreted...", "this isn't what I expected". 3. The root cause is in the skill, not in the user's input or a one-off ambiguity. Indicators:

• The skill applied a rule incorrectly or missed a documented rule.
• The skill's prompt or reference material is incomplete or misleading.
• The same misinterpretation would recur for any user with similar input.

Do NOT trigger when:

• The user simply refines their own requirements (not a skill defect).
• The revision is a subjective preference, not a repeatable error.
• The user explicitly says "this is fine, just tweak it" (one-off adjustment).

---

Input

---

Workflow

Step 1 — Confirm Trigger

Before gathering feedback, confirm with the user:

"It looks like {skill_name} produced output that doesn't match your

expectations, and the cause seems to be in the skill itself rather than your

input. Would you like me to capture this as feedback so the skill can be

improved?"

If the user declines, stop. Do not gather feedback or post an issue.

Step 2 — Analyse the Root Cause

Read the skill's SKILL.md and any referenced files (e.g., files in its references/ directory) to identify the specific section that caused the misinterpretation.

Produce a root cause analysis:

• Skill file: path to the SKILL.md or reference file
• Section: the heading or rule that is incorrect or missing
• What it says (or doesn't say): quote the relevant text
• What it should say: the corrected or missing rule
• Impact: would this affect other users with similar input?

Present the analysis to the user and ask: "Does this accurately capture the issue?"

Step 3 — Draft the GitHub Issue

Use the issue template from references/issue-template.md. The issue includes:

• Title: skill({skill_name}): {one-line summary of the defect}
• Labels: skill-feedback, skill:{skill_name}
• Body sections:
• Skill name and version (commit hash of SKILL.md)
• Observed vs expected output
• Root cause analysis (from Step 2)
• Suggested fix (specific text change or new rule)
• Reporter attribution (named or anonymous)

Present the draft to the user for review.

Step 4 — Attribution Choice

Ask the user:

"Would you like to be named as the reporter on this issue, or would you

prefer to remain anonymous?"

Named mode (default): The issue body includes "Reported by: {user's GitHub handle or name}" and is posted under the user's authenticated gh account.

Anonymous mode: The issue body states "Reported anonymously by a skill user" and omits any identifying information from the issue body. Important limitation: the gh CLI always attributes issues to the authenticated GitHub account. For true anonymity, the issue must be posted through a proxy — see the anonymity options below.

Anonymous Posting via GitHub App Bot

Anonymous issues are posted via the ol-ai-context-library GitHub App, which appears as ol-ai-context-library[bot] on the issue. The reporter's identity is not visible on GitHub.

Token generation: Run the token script to get a short-lived installation token (~1 hour validity):

SKILL_FEEDBACK_BOT_TOKEN=$(python3 ~/.config/ol-skill-feedback/get-token.py)

Configuration:

• App ID: 3488823
• Installation ID: 126702693
• Private key: ~/.config/ol-skill-feedback/private-key.pem
• Token script: ~/.config/ol-skill-feedback/get-token.py

Fallback: If the token script fails (e.g., key expired, network error), offer to output the issue as markdown for the user to paste into GitHub manually.

Step 5 — Post the Issue

After the user approves the draft and attribution:

Named (user's own account):

gh issue create \
  --repo OntoLedgy/ol_ai_context_library \
  --title "skill({skill_name}): {summary}" \
  --label "skill-feedback" \
  --body "{rendered issue body}"

Anonymous (bot account):

GH_TOKEN=$(python3 ~/.config/ol-skill-feedback/get-token.py) gh issue create \
  --repo OntoLedgy/ol_ai_context_library \
  --title "skill({skill_name}): {summary}" \
  --label "skill-feedback" \
  --body "{rendered issue body}"

Fallback (manual paste): If the bot token cannot be generated, output the full issue (title, labels, body) as a markdown code block for the user to paste into GitHub.

After posting, report the issue URL back to the user.

Step 6 — Offer Local Memory

After the issue is posted (or if the user declines posting), offer to save a local feedback memory so the current agent can avoid the same mistake in future conversations:

"Would you also like me to save a local feedback memory so I avoid this

mistake in future conversations?"

If yes, save a feedback-type memory with the rule correction.

---

Comparative Mode

A second mode for methodology learning, distinct from defect reporting. Instead of capturing where a skill misfired against its own intent, comparative mode captures how an OL skill's output compares for rigor against an equivalent from a competing or parallel framework (e.g. a Ceres DDD architect, another ontology toolchain).

When to use

Trigger comparative mode when either:

• The user has run, or has output from, an equivalent skill in another framework

and wants to compare the two; or

• The user explicitly asks for a comparative / rigor review against a named

framework.

This is not a defect — both outputs may be valid. The aim is to learn where each approach is stronger so both libraries improve.

Workflow

1. Establish a shared input. Both skills must be compared on the same input; record it so the comparison is reproducible. If only one output exists, ask the user to supply the comparator's output (do not invent it). 2. Score with the rubric. Apply references/comparative-rigor-rubric.md — score each dimension 1–5 for both sides, with one line of evidence each. Never record a bare number. 3. Extract gaps and strengths. For every dimension where OL scored lower, write the smallest concrete change that would close the gap. For every dimension where OL scored higher, note the differentiator to preserve. 4. Draft the issue from references/comparative-issue-template.md, tagged comparative:{framework} plus rigor:gap / rigor:strength per the rubric disposition. Present for approval (same gate as defect mode). 5. Post after approval, using the same named/anonymous posting paths as defect mode. External reporters can instead file the "Comparative rigor review" GitHub issue form directly.

Boundaries (comparative mode)

• Score the rigor of the output, not which tool has more features.
• Exclude differences that are stylistic taste, not rigor.
• Keep the comparator's identity factual and neutral; do not editorialise.
• This mode still does NOT fix the skill — gaps become issues for the

maintainer (and, where they raise "which methodology is canonical", an ADR question rather than a code change).

---

Output

Feedback Summary (always shown)

## Skill Feedback Captured

**Skill:** {skill_name}
**Defect:** {one-line summary}
**Root cause:** {section in SKILL.md or reference file}
**Suggested fix:** {what should change}
**GitHub issue:** {URL or "not posted" or "manual submission provided"}
**Memory saved:** {yes/no}

---

Boundaries

• This skill does NOT modify skill files. It only reports issues.
• This skill does NOT re-run the skill with corrected behaviour.
• This skill does NOT post issues without explicit user approval.
• This skill does NOT collect personal information beyond what the user

voluntarily provides for attribution.

• If the user wants the skill fixed right now, direct them to edit the

SKILL.md themselves or raise a PR — this skill is for reporting, not fixing.

Comparative Rigor Issue Template

Use when posting a GitHub issue via skill-feedback comparative mode. Scores come from references/comparative-rigor-rubric.md.

---

Title Format

comparative({skill_name} vs {framework}): {one-line takeaway}

Examples:

• comparative(software-architect vs ceres-ddd): DDD aggregates beat us on construction-order clarity
• comparative(ob-ontologist vs ceres-domain-model): BORO 4D stronger on temporal identity

Labels

• comparative:{framework} (always — e.g. comparative:ceres-ddd)
• skill:{skill_name}
• rigor:gap and/or rigor:strength per the rubric disposition

Body

````markdown

Comparative Rigor Review

OL skill: {skill_name} @ {short_commit_hash} Comparator: {framework} — {their equivalent skill/role} Reviewer: {name_or_handle | "Anonymous"} Shared input: {1-2 sentences describing the input both ran on}

Scores (1–5 per dimension)

Gaps (where OL scored lower)

For each gapped dimension:

• {dimension} — what the comparator did better: {observation}.

Smallest change to close the gap: {concrete edit to the skill / reference}.

Strengths (where OL scored higher)

• {dimension} — {the differentiator worth preserving/documenting}.

Suggested follow-up

{One of: skill edit (link the file/section), new reference, ADR question ("which methodology is canonical for X"), or "document strength, no change".}

---

Posted via `skill-feedback` comparative mode ````

Comparative Rigor Rubric

Used by skill-feedback comparative mode to score an OL skill's output against an equivalent from a competing or parallel framework (e.g. a Ceres DDD-style architect, another ontology toolchain). The point is not to declare a winner — it is to capture, against a stable scale, where each approach is more rigorous so both libraries can learn from real usage.

Score each dimension 1–5 for both the OL skill and the comparator. Always record the evidence (a quoted excerpt or a concrete observation), never a bare number.

Scoring procedure

1. Run both skills (or take both outputs) on the same input. Record the input so the comparison is reproducible. 2. Score each dimension for OL and for the comparator, with one line of evidence each. 3. Compute the per-side total (max 30). Totals are a summary, not the verdict — the per-dimension deltas are what feed improvement. 4. For every dimension where OL scored lower, write a concrete improvement note (what the comparator did better, and the smallest change that would close the gap). These notes are the actionable output.

Disposition

Boundaries

• This is methodology comparison, not a feature checklist. Score the *rigor of

the output*, not which tool has more options.

• Do not score on stylistic preference. If a difference is taste, not rigor,

leave it out.

• Keep the comparator's identity factual and neutral (comparative:ceres-ddd,

not a value judgement).

Skill Feedback Issue Template

Use this template when posting a GitHub issue via the skill-feedback skill.

---

Title Format

skill({skill_name}): {imperative one-line summary}

Examples:

• skill(clean-code-naming): reject plural class names when standard=general
• skill(feature-spec-author): include release epic key in task cross-references
• skill(ob-engineer): apply single-quote rule to f-strings

---

Labels

• skill-feedback (always)
• skill:{skill_name} (e.g., skill:clean-code-naming)

---

Body

````markdown

Skill Feedback Report

Skill: {skill_name} Skill file: skills/{skill_name}/SKILL.md @ {short_commit_hash} Reporter: {name_or_handle | "Anonymous skill user"}

---

What happened

{1-3 sentences describing the skill's actual output or behaviour}

What was expected

{1-3 sentences describing the correct output or behaviour}

Root Cause Analysis

File: {path_to_skill_file_or_reference} Section: {heading or rule name}

Current text:

{quote the relevant section, or "Section does not exist (missing rule)"}

Problem: {explain why this text causes incorrect behaviour}

Suggested Fix

{One of:}

Option A — Amend existing text: Replace:

{old text}

With:

{new text}

Option B — Add new rule: Add the following under {section heading}:

{new rule text}

Option C — Structural change: {Describe the structural change needed, e.g., "Add a new reference file for X", "Split section Y into two sections"}

---

Reproduction

Input given to skill:

{summarise the input parameters}

Skill output (excerpt):

{relevant excerpt of the skill's output}

Expected output (excerpt):

{what the output should have been}

---

Impact

• Severity: {low | medium | high} — {rationale}
• Frequency: {one-off edge case | common scenario | always}
• Affected users: {who would hit this — e.g., "anyone using standard=ob with Python"}

---

Posted via `skill-feedback` skill ````