Skill Tester

Name: Skill Tester
Author: pskoett

pskoett/pskoett-skills

4 installs
272 repo stars
Updated June 12, 2026
pskoett/pskoett-skills

Helps with testing & qa tasks.

About

skill-tester is a Claude Code skill for testing & qa. It helps solo builders move faster with AI-assisted development.

skill-tester
Testing & QA
AI-coding skill

Skill Tester by the numbers

4 all-time installs (skills.sh)
Ranked #1,631 of 2,153 Testing & QA skills by installs in the Skillselion catalog
Data as of Aug 4, 2026 (Skillselion catalog sync)

npx skills add https://github.com/pskoett/pskoett-skills --skill skill-tester

Add your badge

Show developers this skill is listed on Skillselion. Paste this into your README.

[![Listed on Skillselion](https://skillselion.com/badge/skills/pskoett/pskoett-skills/skill-tester.svg)](https://skillselion.com/skills/pskoett/pskoett-skills/skill-tester)

Installs	4
repo stars	★ 272
Last updated	June 12, 2026
Repository	pskoett/pskoett-skills ↗

What it does

Helps with testing & qa tasks.

Files

SKILL.mdMarkdownGitHub ↗

Skill Tester

Validates all interactive (non-CI) skills in this repo. Runs the Anthropic skill-creator's quick_validate.py plus project-specific checks.

When to Use

After adding or modifying a skill
Before committing changes
Before submitting the plugin for Anthropic review
As part of the outer loop when eval-creator needs to verify skill quality

Checks

1. Anthropic Spec Validation

Run quick_validate.py on every skill in skills/ (excluding -ci variants):

for d in skills/*/; do
  skill=$(basename "$d")
  [[ "$skill" == *-ci ]] && continue
  python3 .claude/skills/skill-creator/scripts/quick_validate.py "$d"
done

Pass criteria: Exit code 0 for every skill. Frontmatter has only allowed keys (name, description, license, allowed-tools, metadata, compatibility). Name is kebab-case, max 64 chars. Description max 1024 chars, no angle brackets.

2. Project Convention Checks

For each skill directory:

Check	Rule	Severity
Name matches folder	Frontmatter `name` == directory name	Error
Line limit	SKILL.md at or under 600 lines	Error
No README.md	Skill folders must not contain README.md	Error
Scripts executable	All `.sh` files in `scripts/` must have execute permission	Error
References exist	Files referenced in SKILL.md body actually exist in `references/`	Warning
Description non-empty	Description field is present and non-empty	Error

3. Cross-Reference Validation

Verify that all skills listed in these files actually exist as directories:

CLAUDE.md — Skill References section
AGENTS.md — Skill References section
.github/copilot-instructions.md — Skill References section
README.md — Skills table

Also verify reverse: every skill directory is listed in all four files.

4. Hook Script Testing

For each skill with a scripts/ directory:

# Syntax check
bash -n scripts/*.sh

# Verify shebang
head -1 scripts/*.sh | grep -q "^#!/bin/bash"

# Verify executable
test -x scripts/*.sh

5. Plugin Skill Validation

For skills that exist in both skills/ and plugin/skills/:

Check	Rule
Plugin frontmatter keys	Only Claude Code-specific keys (`hooks`, `user-invocable`, `argument-hint`) added beyond spec
Content alignment	Body content matches or plugin has extracted references
Beta markers consistent	If `skills/` copy has `[Beta]`, plugin copy should too

Output Format

## Skill Test Results

**Date:** YYYY-MM-DD
**Skills tested:** N
**Passed:** N
**Warnings:** N
**Failed:** N

### Failures
- [skill-name]: [check]: [error message]

### Warnings
- [skill-name]: [check]: [warning message]

### All Passed
- [list of clean skills]

Running

Invoke manually:

/skill-tester

Or run the script directly:

bash skills/skill-tester/scripts/run-tests.sh

What This Skill Does NOT Do

Does not test CI skills (use skill-tester-ci for those)
Does not modify skills — reports findings only
Does not run behavioral evals (trigger testing) — use skill-creator's run_eval.py for that
Does not replace the eval-creator regression framework — this tests skill structure, not promoted rules

#!/bin/bash
# Validates all interactive skills against the Agent Skills spec and project conventions.
# Usage: bash skills/skill-tester/scripts/run-tests.sh [skill-name]
#   No args: test all non-CI skills
#   With arg: test only the named skill

set -e

REPO_ROOT="$(cd "$(dirname "$0")/../../.." && pwd)"
SKILLS_DIR="$REPO_ROOT/skills"
PLUGIN_DIR="$REPO_ROOT/plugin/skills"
VALIDATE="$REPO_ROOT/.claude/skills/skill-creator/scripts/quick_validate.py"

pass=0; warn=0; fail=0
failures=""
warnings=""

check_skill() {
  local skill_dir="$1"
  local skill=$(basename "$skill_dir")

  # Skip CI skills
  [[ "$skill" == *-ci ]] && return
  [[ "$skill" == "skill-tester" ]] && return
  [[ "$skill" == "skill-tester-ci" ]] && return

  # 1. Anthropic spec validation
  if [ -f "$VALIDATE" ]; then
    result=$(python3 "$VALIDATE" "$skill_dir" 2>&1)
    code=$?
    if [ $code -ne 0 ]; then
      failures="${failures}\n  ✗ $skill: spec: $result"
      fail=$((fail + 1))
      return
    fi
  fi

  # 2. Name matches folder
  name=$(head -10 "$skill_dir/SKILL.md" | grep "^name:" | sed 's/name: *//' | tr -d '"')
  if [ "$name" != "$skill" ]; then
    failures="${failures}\n  ✗ $skill: name mismatch: frontmatter='$name' folder='$skill'"
    fail=$((fail + 1))
    return
  fi

  # 3. Line count
  lines=$(wc -l < "$skill_dir/SKILL.md" | tr -d ' ')
  if [ "$lines" -gt 600 ]; then
    failures="${failures}\n  ✗ $skill: $lines lines (hard limit 600)"
    fail=$((fail + 1))
    return
  elif [ "$lines" -gt 500 ]; then
    warnings="${warnings}\n  ⚠ $skill: $lines lines (soft limit 500)"
    warn=$((warn + 1))
  fi

  # 4. No README.md
  if [ -f "$skill_dir/README.md" ]; then
    failures="${failures}\n  ✗ $skill: contains README.md (not allowed per spec)"
    fail=$((fail + 1))
    return
  fi

  # 5. Scripts executable
  if [ -d "$skill_dir/scripts" ]; then
    for script in "$skill_dir"/scripts/*.sh; do
      [ -f "$script" ] || continue
      if [ ! -x "$script" ]; then
        failures="${failures}\n  ✗ $skill: $(basename $script) not executable"
        fail=$((fail + 1))
        return
      fi
      # Syntax check
      if ! bash -n "$script" 2>/dev/null; then
        failures="${failures}\n  ✗ $skill: $(basename $script) has syntax errors"
        fail=$((fail + 1))
        return
      fi
    done
  fi

  # 6. Description non-empty
  desc=$(head -10 "$skill_dir/SKILL.md" | grep "^description:" | sed 's/description: *//')
  if [ -z "$desc" ] || [ "$desc" = '""' ]; then
    failures="${failures}\n  ✗ $skill: empty description"
    fail=$((fail + 1))
    return
  fi

  pass=$((pass + 1))
}

echo "## Skill Test Results"
echo
echo "**Date:** $(date +%Y-%m-%d)"

if [ -n "$1" ]; then
  # Test single skill
  if [ -d "$SKILLS_DIR/$1" ]; then
    check_skill "$SKILLS_DIR/$1"
  else
    echo "Skill not found: $1"
    exit 1
  fi
else
  # Test all
  for d in "$SKILLS_DIR"/*/; do
    check_skill "$d"
  done
fi

total=$((pass + warn + fail))
echo "**Skills tested:** $total"
echo "**Passed:** $pass"
echo "**Warnings:** $warn"
echo "**Failed:** $fail"
echo

if [ $fail -gt 0 ]; then
  echo "### Failures"
  echo -e "$failures"
  echo
fi

if [ $warn -gt 0 ]; then
  echo "### Warnings"
  echo -e "$warnings"
  echo
fi

if [ $pass -gt 0 ]; then
  echo "### Passed"
  for d in "$SKILLS_DIR"/*/; do
    skill=$(basename "$d")
    [[ "$skill" == *-ci ]] && continue
    [[ "$skill" == "skill-tester" ]] && continue
    [[ "$skill" == "skill-tester-ci" ]] && continue
    echo "  ✓ $skill"
  done
fi

exit $fail