Plugin Review

Plugin Review

Tiered quality review of plugins with dependency-aware scoping.

Table of Contents

• Tiers
• Orchestration
• Scope Detection
• Module Loading
• Verdict
• Output Format
• Quality Gate Mode
• Configuration

Tiers

Orchestration

1. Detect scope: parse --tier flag, find affected plugins from git diff, resolve related plugins from docs/plugin-dependencies.json 2. Plan: build check matrix (tier x plugin x role) 3. Execute: run checks per tier definition 4. Report: per-plugin table, aggregate verdict

Scope Detection

Affected plugins: git diff main --name-only filtered to plugins/*/.

Related plugins: load docs/plugin-dependencies.json, look up each affected plugin's reverse index to find dependents. Mark as "related" (lighter checks).

If --tier release or no git diff available, scope to all plugins.

Module Loading

• Always: this SKILL.md (orchestration logic)
• branch tier: load modules/tier-branch.md
• pr tier: load modules/tier-branch.md then

modules/tier-pr.md

• release tier: load all tier modules plus

modules/tier-release.md

• When resolving deps: load

modules/dependency-detection.md

Verdict

Output Format

Plugin Review (<tier> tier)
Affected: <list>
Related:  <list> (<reason>)

Plugin          test  lint  type  reg   verdict
<name>          PASS  PASS  PASS  PASS  PASS
...

Verdict: <PASS|PASS-WITH-WARNINGS|FAIL> (N/N plugins healthy)

PR and release tiers add scorecard sections.

Quality Gate Mode

The --quality-gate flag enables CI/CD integration with exit codes that distinguish warnings from failures:

• 0: all quality gates passed
• 1: warnings present but gates passed (non-blocking)
• 2: quality gate failures (blocking)
• 3: critical issues found (blocking)

Use --fail-on warning to treat warnings as blocking.

Configuration

Place a .plugin-review.yaml file in the plugin root to customize thresholds and focus areas:

plugin_review:
  quality_gates:
    structure_min: 80
    skills_min: 75
    hooks_min: 70
    tokens_max_total: 50000
    bloat_max_percentage: 15
  focus_areas:
    - skills
    - hooks
    - tokens
  exclude_patterns:
    - "*/legacy/*"
    - "*/deprecated/*"
  severity_overrides:
    missing_description: warning
    large_file: info

See the /plugin-review command reference for full usage examples.

Dependency Detection

How to resolve affected and related plugins for scoped review.

Affected Plugins

Detect from git diff against the base branch:

git diff main --name-only | \
  grep '^plugins/' | \
  sed 's|^plugins/\([^/]*\)/.*|\1|' | \
  sort -u

If no diff available (detached HEAD, no main branch), fall back to all plugins.

Related Plugins

Load docs/plugin-dependencies.json and for each affected plugin, look up who depends on it:

1. Read the dependencies section to find the affected plugin 2. If dependents is ["*"], ALL other plugins are related 3. Otherwise, dependents lists the specific related plugins

Example: if abstract is affected and has dependents: ["*"], then all 16 other plugins are related.

If leyline is affected and has dependents: ["conjure"], then only conjure is related.

Deduplication

A plugin that is both affected (has direct changes) AND related (depends on another changed plugin) is treated as affected. The "affected" role always wins.

Special Cases

• Root-level changes (pyproject.toml, Makefile, scripts/):

These affect the workspace, not individual plugins. At branch tier, skip. At pr/release tier, flag for manual review.

• No plugins changed: If the diff has no plugin changes,

report "No plugin changes detected" and exit with PASS.

• New plugin: If a plugin directory exists on disk but

is not in plugin-dependencies.json, flag it and suggest running scripts/generate_dependency_map.py to update.

Branch Tier Checks

The branch tier runs quick quality gates on affected and related plugins. Designed for fast feedback during development.

Checks for Affected Plugins

Run these sequentially for each affected plugin:

1. Plugin Structure Validation

Run the plugin validator to check plugin.json schema, skill/agent frontmatter, hook event types, and path references against the official Claude Code spec:

python3 plugins/abstract/scripts/validate_plugin.py \
  plugins/<plugin>

If critical issues found, mark plugin as FAIL. Warnings are non-blocking at branch tier.

2. Registration Audit

Only available in night-market repo. Check script exists first:

# Check if the script exists (night-market only)
if [[ -f "plugins/sanctum/scripts/update_plugin_registrations.py" ]]; then
  python3 plugins/sanctum/scripts/update_plugin_registrations.py \
    <plugin-name> --dry-run
else
  echo "Registration audit skipped - not running in night-market"
fi

Report any missing or stale registrations. If skipped, note in the result table that registration audit is unavailable.

3. Test Gate

cd plugins/<plugin> && make test

Capture pass/fail and test count. If tests fail, mark plugin as FAIL.

4. Lint Gate

cd plugins/<plugin> && make lint

Capture pass/fail. If lint fails, mark as WARNING (not blocking at branch tier).

5. Typecheck Gate

cd plugins/<plugin> && make typecheck

Capture pass/fail. If typecheck fails, mark as WARNING.

6. Diff Analysis

Run git diff main -- plugins/<plugin>/ to identify:

• New files (commands, skills, hooks, agents)
• Deleted files
• Modified production code vs test-only changes

Flag high-risk patterns:

• Hook changes (security surface)
• __init__.py export changes (API surface)
• pyproject.toml dependency changes

Checks for Related Plugins

Run a lighter subset:

1. Registration Audit

Same as affected plugins.

2. Test Gate

cd plugins/<plugin> && make test

Tests must pass. This catches side-effect breakage from dependency changes.

Result Table

Build a table with columns: plugin, test, lint, type, reg, verdict. Use -- for skipped checks on related plugins.

Verdict Rules

• Any test FAIL on affected or related: overall FAIL
• Any lint/type FAIL on affected: overall PASS-WITH-WARNINGS
• All green: PASS

PR Tier Checks

The PR tier adds quality scoring on top of branch tier gates. Runs before merge.

Additional Checks for Affected Plugins

After all branch tier checks pass, run these:

1. Skills Evaluation

Invoke Skill(abstract:skills-eval) for each affected plugin. Capture:

• Per-skill quality scores
• Token efficiency ratings
• Compliance status

Report any skill scoring below 70 (MINIMUM_QUALITY_THRESHOLD).

2. Hooks Evaluation

If the plugin has hooks that changed in the diff:

Invoke Skill(abstract:hooks-eval) targeting only changed hook files. Capture:

• Security findings
• Performance concerns
• Compliance status

3. Test Review

Invoke Skill(pensive:test-review) for each affected plugin. Capture:

• Coverage percentage
• Anti-pattern findings
• Missing edge cases

4. Quick Bloat Scan

Invoke Skill(conserve:bloat-detector) at Tier 1 (quick) for each affected plugin. Capture:

• Dead code candidates
• Duplicate content
• Stale files

5. Rules Evaluation

If .claude/rules/ files changed in the diff:

Invoke Skill(abstract:rules-eval). Capture:

• YAML validity
• Pattern specificity
• Content quality

Parallel Execution

Group affected plugins into 2-3 clusters and dispatch agents in parallel. Each agent runs the full PR check suite on its assigned plugins.

Scorecard Output

Add a scorecard section after the branch tier table:

Scorecard (PR tier)

Plugin          skills  hooks  tests  bloat  grade
sanctum         88/100  92/100 93%    90/100 A-
memory-palace   85/100  --     89%    82/100 B+

Top 5 Remediation Actions:
1. [sanctum] skills/commit-messages: missing trigger phrases
2. [memory-palace] hooks/research_interceptor.py: no timeout
...

Verdict Rules

• Any branch-tier FAIL: overall FAIL
• Any skill score below 50: overall FAIL
• Any security finding (HIGH): overall FAIL
• All scores above 70: PASS
• Otherwise: PASS-WITH-WARNINGS with details

Release Tier Checks

The release tier runs a full ecosystem audit across all 17 plugins. Requires plan mode for parallel agent dispatch.

Scope

All 17 plugins, regardless of git diff. This is the pre-release validation gate.

Additional Checks (Beyond PR Tier)

1. Architecture Review

Invoke Skill(pensive:architecture-review) to assess:

• ADR compliance across all plugins
• Cross-plugin coupling analysis
• Module boundary violations
• Design pattern adherence

2. Unified Review

Invoke Skill(pensive:unified-review) for multi-domain orchestration covering API, architecture, test, and code quality dimensions.

3. Deep Bloat Scan

Invoke Skill(conserve:bloat-detector) at Tier 2-3 (deep) for full dead code analysis, cross-file duplication detection, and dependency audit.

4. Token Efficiency Analysis

For each plugin, calculate:

• Total skill description budget usage
• Per-skill token estimates
• Progressive loading compliance
• Module size distribution

Flag any plugin exceeding its proportional token budget allocation.

5. Meta-Evaluation

Run python3 plugins/sanctum/scripts/meta_evaluation.py to validate that evaluation-related skills (skills-eval, hooks-eval, rules-eval) meet their own quality standards.

6. Cross-Plugin Dependency Validation

Regenerate the dependency map and compare with the committed version:

python3 scripts/generate_dependency_map.py --stdout | \
  diff - docs/plugin-dependencies.json

Flag any drift between actual and documented dependencies.

Agent Dispatch Plan

Requires plan mode (4+ agents rule). Suggested grouping:

Each agent runs full PR-tier checks on its plugins plus the release-specific checks.

Full Ecosystem Report

Plugin Review (release tier) - vX.Y.Z
Date: YYYY-MM-DD
Plugins: 17/17

ECOSYSTEM HEALTH
Plugin          test  lint  type  skills hooks  bloat  grade
abstract        PASS  PASS  PASS  92     88     90     A
attune          PASS  PASS  PASS  85     --     82     B+
conjure         PASS  PASS  PASS  88     85     87     A-
...

ARCHITECTURE
ADR compliance: 7/7
Coupling score: 0.06
Boundary violations: 0

TOKEN BUDGET
Used: 15,200 / 17,000 chars (89.4%)
Largest: attune (3,920 chars, 23.1%)

DEPENDENCY MAP
Status: current (no drift)

Verdict: PASS (17/17 plugins healthy)

Verdict Rules

• Any test FAIL: overall FAIL
• Any security finding (HIGH): overall FAIL
• ADR violation without justification: overall FAIL
• Coupling score > 0.7: FAIL
• All plugins grade B or above: PASS
• Otherwise: PASS-WITH-WARNINGS