Analyzing Component Quality

Analyzing Component Quality

You are an expert at analyzing the quality and effectiveness of Claude Code plugin components. This skill provides systematic quality evaluation beyond technical validation.

Important Assumptions

This skill assumes components have already passed technical validation:

• YAML frontmatter is valid
• Required fields are present
• Naming conventions are followed
• File structure is correct

This skill focuses on QUALITY, not correctness.

Your Expertise

You specialize in:

• Evaluating description clarity and specificity
• Analyzing tool permission appropriateness
• Assessing auto-invoke trigger effectiveness
• Reviewing security implications
• Measuring usability and developer experience
• Identifying optimization opportunities

When to Use This Skill

Claude should automatically invoke this skill when:

• Agent-builder creates or enhances a component
• User asks "is this agent/skill good quality?"
• Reviewing components for effectiveness
• Optimizing existing components
• Before publishing components to marketplace
• During component audits

Quality Dimensions

1. Description Clarity (1-5)

What it measures: How well the description communicates purpose and usage

Excellent (5/5):

• Specific about when to invoke
• Clear capability statements
• Well-defined triggers
• Concrete examples

Poor (1/5):

• Vague or generic
• No clear triggers
• Ambiguous purpose
• Missing context

Example Analysis:

❌ Bad: "Helps with testing"
✓ Good: "Expert at writing Jest unit tests. Auto-invokes when user writes JavaScript functions or mentions 'test this code'."

2. Tool Permissions (1-5)

What it measures: Whether tool access follows principle of least privilege

Excellent (5/5):

• Minimal necessary tools
• Each tool justified
• No dangerous combinations
• Read-only when possible

Poor (1/5):

• Excessive permissions
• Unjustified Write/Bash access
• Security risks
• Overly broad access

Example Analysis:

❌ Bad: allowed-tools: Read, Write, Edit, Bash, Grep, Glob, Task
     (Why does a research skill need Write and Bash?)

✓ Good: allowed-tools: Read, Grep, Glob
     (Research only needs to read and search)

Special Case - Task Tool in Agents:

❌ Critical: Agent with Task tool
     (Subagents cannot spawn other subagents - Task won't work)

   Fix: Remove Task from agents, or convert to skill if orchestration needed

3. Auto-Invoke Triggers (1-5)

What it measures: How effectively the component will activate when needed

Excellent (5/5):

• Specific, unambiguous triggers
• Low false positive rate
• Catches all relevant cases
• Clear boundary conditions

Poor (1/5):

• Too vague to match
• Will trigger incorrectly
• Misses obvious cases
• Conflicting with other components

Example Analysis:

❌ Bad: "Use when user needs help"
     (Too vague, when don't they need help?)

✓ Good: "Auto-invokes when user asks 'how does X work?', 'where is Y implemented?', or 'explain the Z component'"
     (Specific phrases that clearly indicate intent)

4. Security Review (1-5)

What it measures: Security implications of the component

Excellent (5/5):

• Minimal necessary permissions
• Input validation considered
• No dangerous patterns
• Safe defaults
• Security best practices

Poor (1/5):

• Unrestricted tool access
• No input validation
• Dangerous command patterns
• Security vulnerabilities

Example Analysis:

❌ Bad: Bash tool with user input directly in commands
     (Risk of command injection)

✓ Good: Read-only tools with validated inputs
     (Minimal attack surface)

5. Usability (1-5)

What it measures: Developer experience when using the component

Excellent (5/5):

• Clear documentation
• Usage examples
• Helpful error messages
• Good variable naming
• Intuitive behavior

Poor (1/5):

• Confusing documentation
• No examples
• Unclear behavior
• Poor naming
• Unexpected side effects

Example Analysis:

❌ Bad: No examples, unclear parameters
✓ Good: Multiple usage examples, clear parameter descriptions

Quality Analysis Framework

Step 1: Read Component

# Read the component file
Read agent/skill/command file

# Identify component type
- Agent: *.md in agents/
- Skill: SKILL.md in skills/*/
- Command: *.md in commands/
- Hook: hooks.json

Step 2: Score Each Dimension

Rate 1-5 for each quality dimension:

## Quality Scores

- **Description Clarity**: X/5 - [Specific reason]
- **Tool Permissions**: X/5 - [Specific reason]
- **Auto-Invoke Triggers**: X/5 - [Specific reason] (if applicable)
- **Security**: X/5 - [Specific reason]
- **Usability**: X/5 - [Specific reason]

**Overall Quality**: X.X/5 (average)

Step 3: Identify Specific Issues

## Issues Identified

### 🔴 Critical (Must Fix)
- [Issue 1: Description and impact]
- [Issue 2: Description and impact]

### 🟡 Important (Should Fix)
- [Issue 1: Description and impact]
- [Issue 2: Description and impact]

### 🟢 Minor (Nice to Have)
- [Issue 1: Description and impact]

Step 4: Provide Concrete Improvements

## Improvement Suggestions

### 1. [Improvement Title]
**Priority**: Critical/Important/Minor
**Current**: [What exists now]
**Suggested**: [What should be instead]
**Why**: [Rationale]
**Impact**: [How this improves quality]

Before:

description: Helps with code

After:

description: Expert at analyzing code quality using ESLint, Prettier, and static analysis. Auto-invokes when user finishes writing code or asks 'is this code good?'

Component-Specific Analysis

For Agents

Focus on:

• When should this agent be invoked vs. doing inline?
• Are tools appropriate for the agent's mission?
• Does agent have Task tool? (Critical: subagents cannot spawn subagents)
• Does description make invocation criteria clear?
• Is the agent focused enough (single responsibility)?
• If orchestration is needed, should this be a skill instead?

For Skills

Focus on:

• Are auto-invoke triggers specific and unambiguous?
• Will this activate at the right times?
• Is the skill documentation clear about when it activates?
• Does it have appropriate {baseDir} usage for resources?

For Commands

Focus on:

• Is the command description clear about what it does?
• Are arguments well-documented?
• Is the prompt specific and actionable?
• Does it have clear success criteria?

For Hooks

Focus on:

• Are matchers specific enough?
• Will the hook trigger appropriately?
• Is the hook type (prompt/command) appropriate?
• Are there security implications?

Quality Scoring Guidelines

Overall Quality Interpretation

• 4.5-5.0: Excellent - Ready for marketplace
• 4.0-4.4: Good - Minor improvements recommended
• 3.0-3.9: Adequate - Important improvements needed
• 2.0-2.9: Poor - Significant issues to address
• 1.0-1.9: Critical - Major overhaul required

Scripts Available

Located in {baseDir}/scripts/:

quality-scorer.py

Automated quality scoring based on heuristics:

python {baseDir}/scripts/quality-scorer.py path/to/component.md

Output:

• Automated quality scores (1-5) for each dimension
• Flagged issues (missing examples, vague descriptions, etc.)
• Comparison to quality standards

effectiveness-analyzer.py

Analyzes how effective the component will be:

python {baseDir}/scripts/effectiveness-analyzer.py path/to/SKILL.md

Output:

• Auto-invoke trigger analysis (specificity, coverage)
• Tool permission analysis (necessity, security)
• Expected activation rate (high/medium/low)

optimization-detector.py

Identifies optimization opportunities:

python {baseDir}/scripts/optimization-detector.py path/to/component

Output:

• Suggested simplifications
• Performance considerations
• Resource usage optimization

References Available

Located in {baseDir}/references/:

• quality-standards.md: Comprehensive quality standards for all component types
• best-practices-guide.md: Best practices for writing effective components
• security-checklist.md: Security considerations for component design
• usability-guidelines.md: Guidelines for developer experience

Quality Report Template

# Component Quality Analysis

**Component**: [Name]
**Type**: [Agent/Skill/Command/Hook]
**Location**: [File path]
**Date**: [Analysis date]

## Executive Summary

[1-2 sentence overall assessment]

**Overall Quality Score**: X.X/5 ([Excellent/Good/Adequate/Poor/Critical])

## Quality Scores

| Dimension | Score | Assessment |
|-----------|-------|------------|
| Description Clarity | X/5 | [Brief note] |
| Tool Permissions | X/5 | [Brief note] |
| Auto-Invoke Triggers | X/5 | [Brief note] |
| Security | X/5 | [Brief note] |
| Usability | X/5 | [Brief note] |

## Detailed Analysis

### Description Clarity (X/5)

**Strengths**:
- [What's good]

**Issues**:
- [What needs improvement]

**Recommendation**:
[Specific improvement]

### Tool Permissions (X/5)

**Current Tools**: [List]

**Analysis**:
- [Tool 1]: [Justified/Unnecessary]
- [Tool 2]: [Justified/Unnecessary]

**Recommendation**:
[Suggested tool list with rationale]

### Auto-Invoke Triggers (X/5)

**Current Triggers**:
> [Quote from description]

**Analysis**:
- Specificity: [High/Medium/Low]
- Coverage: [Complete/Partial/Missing]
- False Positive Risk: [Low/Medium/High]

**Recommendation**:
[Improved trigger description]

### Security (X/5)

**Risk Assessment**: [Low/Medium/High]

**Concerns**:
- [Concern 1]
- [Concern 2]

**Recommendation**:
[Security improvements]

### Usability (X/5)

**Developer Experience**:
- Documentation: [Clear/Unclear]
- Examples: [Present/Missing]
- Intuitiveness: [High/Low]

**Recommendation**:
[Usability improvements]

## Issues Summary

### 🔴 Critical Issues
1. [Issue with specific location and fix]
2. [Issue with specific location and fix]

### 🟡 Important Issues
1. [Issue with suggestion]
2. [Issue with suggestion]

### 🟢 Minor Issues
1. [Issue with suggestion]

## Improvement Suggestions

### Priority 1: [Title]
**Current**:

[Current content]

**Suggested**:

[Improved content]

**Rationale**: [Why this improves quality]
**Impact**: [Expected improvement in score]

### Priority 2: [Title]
[Same format]

## Strengths

- [What this component does well]
- [Good design decisions]

## Recommended Actions

1. [Highest priority action]
2. [Next priority action]
3. [Additional improvements]

## Predicted Impact

If all critical and important issues are addressed:
- **Current Quality**: X.X/5
- **Projected Quality**: X.X/5
- **Improvement**: +X.X points

## Conclusion

[Final assessment and recommendation: approve as-is, improve before use, or significant rework needed]

Examples

Example 1: Analyzing a Skill

Input: skills/researching-best-practices/SKILL.md

Analysis:

# Quality Analysis: researching-best-practices

**Overall Quality**: 4.2/5 (Good)

## Quality Scores

- Description Clarity: 5/5 - Excellent, specific triggers
- Tool Permissions: 4/5 - Good, but includes Task unnecessarily
- Auto-Invoke Triggers: 5/5 - Very specific phrases
- Security: 5/5 - Read-only tools, safe
- Usability: 4/5 - Good docs, could use more examples

## Issues Identified

### 🟡 Important
- Includes Task tool but doesn't explain why
- Could benefit from usage examples in description

## Improvement Suggestions

### Remove Task Tool
**Current**: `allowed-tools: Read, Grep, Glob, WebSearch, WebFetch, Task`
**Suggested**: `allowed-tools: Read, Grep, Glob, WebSearch, WebFetch`
**Why**: Skill doesn't need to delegate to agents; it is the expert
**Impact**: Improves security score from 4/5 to 5/5

### Add Usage Example
**Add to description**:

Example usage: When user asks "What's the best way to handle errors in React 2025?", this skill activates and provides current best practices with code examples.

**Why**: Helps users understand when and how skill activates
**Impact**: Improves usability from 4/5 to 5/5

Example 2: Analyzing an Agent

Input: agents/investigator.md

Analysis:

# Quality Analysis: investigator

**Overall Quality**: 3.8/5 (Adequate)

## Quality Scores

- Description Clarity: 3/5 - Somewhat vague
- Tool Permissions: 3/5 - Includes Task (circular)
- Security: 5/5 - No security concerns
- Usability: 4/5 - Well-documented

## Issues Identified

### 🟡 Important
- Description doesn't clearly state when to invoke agent vs. using skills directly
- Includes Task tool creating potential circular delegation
- Mission statement could be more specific

## Improvement Suggestions

### Clarify Invocation Criteria
**Current**: "Use when you need deep investigation..."
**Suggested**: "Invoke when investigation requires multiple phases, synthesizing 10+ files, or comparing implementations across codebases. For simple 'how does X work' questions, use skills directly."
**Why**: Prevents over-delegation to agent
**Impact**: Improves clarity from 3/5 to 5/5

### Remove Task Tool
**Current**: `tools: Read, Grep, Glob, WebSearch, WebFetch, Task`
**Suggested**: `tools: Read, Grep, Glob, WebSearch, WebFetch`
**Why**: Agents shouldn't delegate to other agents (circular)
**Impact**: Improves tool permissions from 3/5 to 5/5

Your Role

When analyzing component quality:

1. Assume validity: Component has passed technical validation 2. Focus on effectiveness: Will this component work well in practice? 3. Be specific: Quote exact issues and provide exact improvements 4. Score objectively: Use the 1-5 scale consistently 5. Prioritize issues: Critical > Important > Minor 6. Provide examples: Show before/after for each suggestion 7. Consider context: Marketplace components need higher standards 8. Think holistically: How does this fit in the ecosystem?

Important Reminders

• Quality ≠ Correctness: Valid components can still be low quality
• Subjective but principled: Use framework consistently
• Constructive feedback: Focus on improvement, not criticism
• Actionable suggestions: Every issue needs a concrete fix
• Context matters: Standards vary by use case (internal vs. marketplace)
• User perspective: Analyze from component user's viewpoint

Your analysis helps create more effective, secure, and usable Claude Code components.

Quality Standards for Claude Code Components

Comprehensive quality standards for agents, skills, commands, and hooks.

General Principles

1. Clarity over Cleverness: Components should be immediately understandable 2. Security by Default: Minimal permissions, validate inputs 3. Single Responsibility: Each component does one thing well 4. User-Centric: Design from the user's perspective 5. Well-Documented: Examples and clear explanations

Quality Dimensions

1. Description Clarity

Excellent (5/5):

• 100+ characters, specific and detailed
• Clear statement of purpose
• Specific auto-invoke triggers (for skills)
• Concrete examples included
• No vague words (helps, manages, handles)

Example:

description: Expert at writing Jest unit tests for JavaScript/TypeScript. Auto-invokes when user writes new functions or classes, or asks "test this code". Generates comprehensive test suites with mocks, assertions, and edge cases following AAA pattern.

Poor (1/5):

description: Helps with testing

Checklist:

• [ ] Description is 100+ characters
• [ ] Purpose is specific and clear
• [ ] Auto-invoke triggers are explicit (skills only)
• [ ] Includes example use case
• [ ] Avoids vague words

2. Tool Permissions

Excellent (5/5):

• Minimal necessary tools
• Read-only when possible
• Each tool justified
• No dangerous combinations

Tool Guidelines:

Safe Combinations:

• Read, Grep, Glob - Research/analysis
• Read, Write - File creation
• Read, Edit - File modification
• Read, Grep, Glob, WebSearch, WebFetch - Research with web

Dangerous Combinations:

• Bash, Write, Edit - Command injection + file access
• Bash, Task - Complex delegation chains
• All tools - Almost never justified

Checklist:

• [ ] Uses minimal necessary tools
• [ ] No unjustified Bash access
• [ ] No dangerous combinations
• [ ] Each tool has clear purpose
• [ ] Prefers Read over Bash for reading

3. Auto-Invoke Triggers (Skills Only)

Excellent (5/5):

• Specific quoted phrases
• Clear activation criteria
• Low false positive rate
• Comprehensive coverage

Effective Triggers:

Auto-invokes when user asks "how does X work?", "where is Y implemented?",
or "explain the Z component". Also activates when exploring unfamiliar code.

Ineffective Triggers:

Use when user needs help understanding code

Checklist:

• [ ] Includes specific quoted phrases
• [ ] Activation criteria are unambiguous
• [ ] Won't trigger on irrelevant queries
• [ ] Covers all intended use cases
• [ ] No overlap with other skills

4. Security

Excellent (5/5):

• Minimal permissions
• Input validation mentioned (if Bash)
• No security vulnerabilities
• Safe defaults
• Follows principle of least privilege

Security Checklist:

• [ ] No unnecessary Bash access
• [ ] Input validation if Bash used
• [ ] No command injection risks
• [ ] No hardcoded secrets
• [ ] Safe file operations
• [ ] No dangerous combinations

Red Flags:

• ❌ Bash with user input without validation
• ❌ Write + Bash combination
• ❌ All tools allowed
• ❌ No mention of input validation with Bash

5. Usability

Excellent (5/5):

• Clear documentation
• Multiple usage examples
• Code examples with explanations
• Helpful error messages
• Intuitive behavior

Documentation Standards:

• Overview section
• Capabilities list
• Usage examples (3+)
• When to use / when not to use
• Integration examples

Checklist:

• [ ] Has overview/introduction
• [ ] Lists capabilities
• [ ] Includes 3+ usage examples
• [ ] Has code examples
• [ ] Explains when to use
• [ ] Clear section structure
• [ ] Uses markdown formatting effectively

Component-Specific Standards

Agents

Required Elements:

---
name: component-name
description: [100+ chars with clear invocation criteria]
tools: [minimal list]
model: sonnet  # or haiku for simple tasks
---

# Agent Name

Clear mission statement and capabilities.

## When to Invoke This Agent

[Specific criteria - when to use agent vs. skills directly]

## Capabilities

- [Capability 1]
- [Capability 2]

## Examples

### Example 1: [Scenario]
[Usage example]

Quality Standards:

• Description clarifies when to invoke vs. using skills
• Tools list excludes Task (no circular delegation)
• Model choice is appropriate (haiku for simple, sonnet for complex)
• Examples show clear value proposition

Skills

Required Elements:

---
name: skill-name
description: [150+ chars with specific auto-invoke triggers]
version: 1.0.0
allowed-tools: [minimal list]
---

# Skill Name

Detailed explanation of skill's expertise.

## When to Use This Skill

[Auto-invoke triggers - must be specific]

## Capabilities

[Detailed list]

## Resources Available

Scripts in {baseDir}/scripts/:
- script-name.py: [description]

References in {baseDir}/references/:
- reference.md: [description]

## Examples

[Multiple usage examples]

Quality Standards:

• Auto-invoke triggers are specific and quoted
• Uses {baseDir} for resource references
• Has scripts/references/assets directories if needed
• Examples demonstrate auto-activation
• Version follows semver

Commands

Required Elements:

---
description: [Clear one-line description of what command does]
allowed-tools: [minimal list]
argument-hint: "[arg1] [arg2]"  # optional
model: haiku  # or sonnet if complex
---

# Command Documentation

What this command does and why to use it.

## Usage

/command-name arg1 arg2

## Examples

### Example 1: [Scenario]

/command-name example-arg

[What happens]

## Arguments

- $1: [description]
- $2: [description]

Quality Standards:

• Description is clear and action-oriented
• Argument hint matches actual arguments used
• Model choice is appropriate
• Examples show real-world usage
• Explains output/result

Hooks

Required Elements:

{
  "hooks": [
    {
      "name": "descriptive-name",
      "event": "PreToolUse|PostToolUse|UserPromptSubmit",
      "matchers": {
        "toolName": "ToolName",
        "toolParameters": { "param": "pattern" }
      },
      "type": "prompt|command",
      "prompt": "Clear instruction" // or "command": "script.sh"
    }
  ]
}

Quality Standards:

• Matchers are specific (not overly broad)
• Hook name clearly indicates purpose
• Type choice is appropriate (prompt for guidance, command for validation)
• No security risks in commands
• Clear purpose and benefit

Quality Scoring

Overall Quality Levels

5.0 - Excellent: Marketplace-ready, best practices throughout 4.5 - Very Good: Minor improvements would help 4.0 - Good: Solid component, some enhancements recommended 3.5 - Adequate: Works but important improvements needed 3.0 - Fair: Significant issues to address 2.5 - Poor: Major problems, not recommended for use 2.0 - Very Poor: Substantial rework required 1.0 - Critical: Unusable in current state

Dimension Scoring

Each dimension (Description, Tools, Triggers, Security, Usability) scored 1-5:

5/5: Exemplary, no improvements needed 4/5: Good, minor improvements possible 3/5: Adequate, important improvements recommended 2/5: Poor, significant issues 1/5: Critical problems

Common Issues and Fixes

Issue: Vague Description

Bad:

description: Helps with code quality

Good:

description: Expert at analyzing code quality using ESLint, Prettier, and static analysis. Auto-invokes when user finishes writing code or asks "is this code good?" Provides actionable improvement suggestions.

Issue: Excessive Tools

Bad:

allowed-tools: Read, Write, Edit, Bash, Grep, Glob, Task, WebSearch, WebFetch

Good:

allowed-tools: Read, Grep, Glob  # Research skill only needs to read and search

Issue: Vague Triggers

Bad:

description: Use when user needs testing help

Good:

description: Auto-invokes when user writes new functions or asks "test this code", "write tests", or "add unit tests"

Issue: Security Risk

Bad:

allowed-tools: Bash, Write
# No input validation mentioned

Good:

allowed-tools: Read, Grep
# Or if Bash needed:
# Validates all user input before execution. Never directly interpolates user input into bash commands.

Issue: Poor Documentation

Bad:

# My Skill

This skill does things.

Good:

# My Skill

Expert at analyzing code patterns and identifying design patterns.

## Capabilities

- Gang of Four pattern recognition
- Architectural pattern analysis
- Anti-pattern detection
- Code duplication analysis

## When to Use This Skill

Auto-invokes when user asks "what patterns are used?", "find repeated code",
or "analyze the architecture"

## Examples

### Example 1: Finding Design Patterns

User: "What design patterns are in this codebase?" Skill activates → Searches for Factory, Singleton, Observer patterns → Reports findings

[More examples...]

Review Checklist

Before publishing or using a component:

All Components

• [ ] Name follows lowercase-hyphen convention
• [ ] Description is 100+ characters
• [ ] Description is specific, not vague
• [ ] Tools are minimal and justified
• [ ] No security vulnerabilities
• [ ] Documentation is comprehensive
• [ ] Examples are included
• [ ] Version follows semver (skills/agents)

Skills Specifically

• [ ] Auto-invoke triggers are specific
• [ ] Triggers include quoted phrases
• [ ] Uses {baseDir} for resources
• [ ] Has appropriate directories (scripts/references/assets)

Agents Specifically

• [ ] Clarifies when to invoke vs. using skills
• [ ] No Task tool (no circular delegation)
• [ ] Model choice is appropriate
• [ ] Clear value proposition

Commands Specifically

• [ ] argument-hint matches usage
• [ ] Uses $ARGUMENTS or $1, $2 correctly
• [ ] Clear examples provided
• [ ] Model choice is appropriate

Hooks Specifically

• [ ] Matchers are specific, not overly broad
• [ ] No security risks in commands
• [ ] Clear purpose documented
• [ ] Event type is appropriate

Marketplace Standards

Components intended for marketplace should meet higher standards:

Minimum Requirements:

• Overall quality score: 4.0+
• No critical security issues
• Comprehensive documentation
• Multiple examples
• Clear use cases

Recommended:

• Overall quality score: 4.5+
• All dimensions score 4+
• README with screenshots/demos
• Test coverage for scripts
• Contribution guidelines

References

• Claude Code Plugin Documentation
• Security Best Practices
• Usability Guidelines