Agentic Eval

Name: Agentic Eval
Author: practicalswan

practicalswan/agent-skills

4 installs
7 repo stars
Updated August 2, 2026
practicalswan/agent-skills

agentic-eval is a Claude Code skill for ai & agent building.

About

Provides a structure for building evaluator-optimizer pipelines that score output against rubrics and iterate until acceptance gates pass. A developer uses it when adding self-critique or acceptance gates to code, docs, analysis, or plan generation.

Explicit rubrics with reflection loops and stop conditions
Targets evaluator-optimizer and acceptance-gate patterns

Agentic Eval by the numbers

4 all-time installs (skills.sh)
Ranked #13,348 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
Data as of Aug 4, 2026 (Skillselion catalog sync)

npx skills add https://github.com/practicalswan/agent-skills --skill agentic-eval

Add your badge

Show developers this skill is listed on Skillselion. Paste this into your README.

[![Listed on Skillselion](https://skillselion.com/badge/skills/practicalswan/agent-skills/agentic-eval.svg)](https://skillselion.com/skills/practicalswan/agent-skills/agentic-eval)

Installs	4
repo stars	★ 7
Last updated	August 2, 2026
Repository	practicalswan/agent-skills ↗

How do I helps with ai & agent building tasks.?

Evaluates and improves AI-generated output using explicit rubrics, reflection loops, and stop conditions for self-critique workflows.

Who is it for?

A solo builder working on ai & agent building tasks who needs structured help with agentic eval.

Skip if: Teams with no ai & agent building needs, or anyone wanting a generic chat assistant without this specific workflow.

When should I use this skill?

When you need to helps with ai & agent building tasks., or when agentic-eval is a claude code skill for ai & agent building.

What you get

Structured output aligned to agentic-eval: agentic-eval, AI & Agent Building.

Files

SKILL.mdMarkdownGitHub ↗

Agentic Eval

Use structured evaluation loops to improve important outputs before you call them done.

Leverage native parallel subagent dispatch and 200k+ context windows where available.

When to Use

Use symptom -> action triggers: when one matches, apply this skill and verify with the protocol below.

A task is quality-critical and a single pass is too risky.
You need repeatable acceptance criteria for code, docs, analysis, or plans.
You want a reviewer or judge step that is separate from generation.
You need to compare multiple candidate outputs against the same rubric.

Core Loop

1. Define the artifact being judged. 2. Define a rubric with weighted dimensions. 3. Generate or collect the candidate output. 4. Evaluate it against the rubric. 5. Convert the feedback into concrete changes. 6. Re-run until the score crosses the threshold or the iteration budget is exhausted.

Evaluation Patterns

1. Self-Reflection

Use the same agent to critique its own work when the task is moderate risk and the rubric is precise.

Best for:

formatting checks
completeness checks
first-pass code or doc refinement

2. Evaluator-Optimizer Split

Separate generation from evaluation when you want clearer responsibilities.

Best for:

high-value outputs
rubric-based acceptance checks
comparing multiple candidates fairly

3. Evidence-Based Evaluation

Back the score with tests, logs, benchmarks, or direct verification.

Best for:

code generation
migration plans
architecture recommendations
security or compliance review

Rubric Design Rules

Keep dimensions few and concrete.
Weight the business-critical dimension highest.
Define what a passing score means before evaluation starts.
Require written evidence for any failing dimension.
Stop when you are no longer learning new fixes.

Suggested dimensions:

correctness
completeness
clarity
maintainability
risk management
evidence quality

Stop Conditions

Stop the loop when one of these becomes true:

the overall threshold is met
the failing dimensions are now low-impact only
tests or verification evidence already prove the output is acceptable
the score has stopped improving and more iterations are likely noise

Output Format

Use a structure like this when reporting an evaluation:

## Evaluation Summary

### Artifact
- Short description of what was evaluated

### Rubric Results
| Dimension | Weight | Score | Notes |
|-----------|--------|-------|-------|
| correctness | 0.40 | 4/5 | Main logic is sound |

### Overall
- Weighted score: 0.84
- Threshold: 0.80
- Result: PASS

### Required Improvements
- Tighten edge-case handling around ...
- Add verification evidence for ...

Self-Verification Phase-Gate Questions

Before you claim the evaluation is complete, the evaluating agent must ask:

Did I define the rubric, threshold, and evidence sources explicitly enough for another agent to rerun the check?
Did every failing dimension produce a concrete improvement action instead of a vague critique?
Did I stop because the result is acceptable, or only because I ran out of patience?
Can I point to tests, logs, screenshots, or scorecards that support the final PASS or FAIL decision?

Anti-Patterns

Delegating or evaluating without a scoped success condition: The output becomes hard to review and easy to overbuild.
Skipping the evidence step: A workflow that cannot be re-checked quickly is not ready for handoff.
Bundling unrelated subtasks together: It creates noisy prompts, weaker ownership, and avoidable integration risk.

Verification Protocol

Before claiming "skill applied successfully":

1. Pass/fail: The Agentic Eval workflow names the agent boundary, delegated scope, and expected return artifact. 2. Pass/fail: Context passed to helpers is minimal, task-local, and free of hidden expected answers. 3. Pass/fail: Results are integrated only after evidence, diffs, or citations are checked by the controller. 4. Pressure-test scenario: Run the workflow on two similar tasks that must not share assumptions or leaked context. 5. Success metric: Zero context leakage; every delegated output is independently reviewable.

Scripts And References

Rubric Template
Example Scores
Rubric Scorecard Helper

Best Practices

Keep the rubric stable across iterations so the score means something.
Prefer evidence-backed criteria over taste-based criteria.
Store the final rubric and score with the task when the output matters later.
Pair with tests or direct verification whenever the artifact can be executed.
If you use an LLM judge, constrain the output format so it can be parsed and compared.

Cross-Client Portability

This skill is written to stay usable across GitHub Copilot, Claude Code, Codex, and Gemini CLI.

GitHub Copilot: keep the folder in a Copilot-visible skill or plugin path, or wrap the workflow as project instructions if the host does not support portable skill folders directly.
Claude Code: keep the folder in a local skills directory or a compatible plugin or marketplace source.
Codex: install or sync the folder into $CODEX_HOME/skills/<skill-name> and restart Codex after major changes.
Gemini CLI: this repository generates a project command named /skills:agentic-eval from this skill. Rebuild commands with python scripts/export-gemini-skill.py agentic-eval and then run /commands reload inside Gemini CLI.

MCP Availability And Fallback

Preferred MCP Server: None required

Fallback prompt: "Use the Agentic Eval skill without MCP. Rely on the local SKILL.md, bundled references or scripts, and manual verification. Show the exact commands, evidence, and final checks you used before concluding."
If the current host does not expose a matching server, use the bundled references, scripts, native toolchain, and manual workflow already described in this skill.
Treat direct local verification, rendered output, logs, tests, or screenshots as the fallback evidence path before completion.

Related Skills

agent-task-mapping: Use it when the workflow also needs task-to-agent routing decisions.
custom-agent-usage: Use it when the workflow also needs loading and invoking custom agent definitions safely.
subagent-delegation: Use it when the workflow also needs safe, scoped delegation to helper agents.
subagent-driven-development: Use it when the workflow also needs plan-driven implementation with reviewer loops.

{
  "max_score": 5,
  "dimensions": {
    "correctness": {
      "weight": 0.4,
      "description": "Does the artifact solve the actual task without factual or behavioral errors?"
    },
    "completeness": {
      "weight": 0.25,
      "description": "Does it cover the full requested scope and major edge cases?"
    },
    "clarity": {
      "weight": 0.15,
      "description": "Is the result easy to understand and act on?"
    },
    "maintainability": {
      "weight": 0.2,
      "description": "Will the result be easy to evolve, verify, and support later?"
    }
  }
}

#!/usr/bin/env python3
from __future__ import annotations

import argparse
import json
from pathlib import Path


def load_json(path: Path) -> dict:
    return json.loads(path.read_text(encoding="utf-8"))


def compute_score(rubric: dict, scores: dict) -> dict:
    max_score = rubric.get("max_score", 5)
    dimensions = rubric["dimensions"]
    total = 0.0
    missing = []
    per_dimension = {}

    for name, config in dimensions.items():
        weight = float(config["weight"])
        if name not in scores:
            missing.append(name)
            continue
        raw_score = float(scores[name])
        normalized = raw_score / max_score
        weighted = normalized * weight
        total += weighted
        per_dimension[name] = {
            "raw_score": raw_score,
            "max_score": max_score,
            "weight": weight,
            "weighted_score": round(weighted, 4),
        }

    return {
        "overall_score": round(total, 4),
        "missing_dimensions": missing,
        "dimensions": per_dimension,
    }


def main() -> int:
    parser = argparse.ArgumentParser(description="Score a candidate artifact against a weighted rubric.")
    parser.add_argument("--rubric", required=True, help="Path to a rubric JSON file.")
    parser.add_argument("--scores", required=True, help="Path to a scores JSON file.")
    parser.add_argument("--threshold", type=float, default=0.8, help="Passing threshold on a 0-1 scale.")
    args = parser.parse_args()

    rubric = load_json(Path(args.rubric))
    scores = load_json(Path(args.scores))
    result = compute_score(rubric, scores)
    result["threshold"] = args.threshold
    result["result"] = "PASS" if result["overall_score"] >= args.threshold and not result["missing_dimensions"] else "FAIL"

    print(json.dumps(result, indent=2))
    return 0 if result["result"] == "PASS" else 1


if __name__ == "__main__":
    raise SystemExit(main())