
Skill Tester
- 128 installs
- 451 repo stars
- Updated July 21, 2026
- borghei/claude-skills
Validate Claude Code skills end-to-end: trigger conditions, run scenarios, check outputs, and report gaps before publishing skill packs to users.
About
Skill tester for Claude agent skill packs: designs scenario matrices, verifies trigger phrases and instructions, runs representative tasks, scores output quality, documents failures, and recommends fixes so skills behave reliably before release to developers.
- Trigger and scope validation
- Scenario-based skill runs
- Output rubric and regression checks
- Edge-case and failure reproduction
- Publish-readiness reports
Skill Tester by the numbers
- 128 all-time installs (skills.sh)
- Ranked #228 of 782 Skill Development skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/borghei/claude-skills --skill skill-testerAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 128 |
|---|---|
| repo stars | ★ 451 |
| Last updated | July 21, 2026 |
| Repository | borghei/claude-skills ↗ |
What it does
Validate Claude Code skills end-to-end: trigger conditions, run scenarios, check outputs, and report gaps before publishing skill packs to users.
Files
Sample Text Processor
Name: sample-text-processor Tier: BASIC Category: Text Processing Dependencies: None (Python Standard Library Only) Author: Claude Skills Engineering Team Version: 1.0.0 Last Updated: 2026-02-16
---
Description
The Sample Text Processor is a simple skill designed to demonstrate the basic structure and functionality expected in the claude-skills ecosystem. This skill provides fundamental text processing capabilities including word counting, character analysis, and basic text transformations.
This skill serves as a reference implementation for BASIC tier requirements and can be used as a template for creating new skills. It demonstrates proper file structure, documentation standards, and implementation patterns that align with ecosystem best practices.
The skill processes text files and provides statistics and transformations in both human-readable and JSON formats, showcasing the dual output requirement for skills in the claude-skills repository.
Features
Core Functionality
- Word Count Analysis: Count total words, unique words, and word frequency
- Character Statistics: Analyze character count, line count, and special characters
- Text Transformations: Convert text to uppercase, lowercase, or title case
- File Processing: Process single text files or batch process directories
- Dual Output Formats: Generate results in both JSON and human-readable formats
Technical Features
- Command-line interface with comprehensive argument parsing
- Error handling for common file and processing issues
- Progress reporting for batch operations
- Configurable output formatting and verbosity levels
- Cross-platform compatibility with standard library only dependencies
Usage
Basic Text Analysis
python text_processor.py analyze document.txt
python text_processor.py analyze document.txt --output results.jsonText Transformation
python text_processor.py transform document.txt --mode uppercase
python text_processor.py transform document.txt --mode title --output transformed.txtBatch Processing
python text_processor.py batch text_files/ --output results/
python text_processor.py batch text_files/ --format json --output batch_results.jsonExamples
Example 1: Basic Word Count
$ python text_processor.py analyze sample.txt
=== TEXT ANALYSIS RESULTS ===
File: sample.txt
Total words: 150
Unique words: 85
Total characters: 750
Lines: 12
Most frequent word: "the" (8 occurrences)Example 2: JSON Output
$ python text_processor.py analyze sample.txt --format json
{
"file": "sample.txt",
"statistics": {
"total_words": 150,
"unique_words": 85,
"total_characters": 750,
"lines": 12,
"most_frequent": {
"word": "the",
"count": 8
}
}
}Example 3: Text Transformation
$ python text_processor.py transform sample.txt --mode title
Original: "hello world from the text processor"
Transformed: "Hello World From The Text Processor"Installation
This skill requires only Python 3.7 or later with the standard library. No external dependencies are required.
1. Clone or download the skill directory 2. Navigate to the scripts directory 3. Run the text processor directly with Python
cd scripts/
python text_processor.py --helpConfiguration
The text processor supports various configuration options through command-line arguments:
--format: Output format (json, text)--verbose: Enable verbose output and progress reporting--output: Specify output file or directory--encoding: Specify text file encoding (default: utf-8)
Architecture
The skill follows a simple modular architecture:
- TextProcessor Class: Core processing logic and statistics calculation
- OutputFormatter Class: Handles dual output format generation
- FileManager Class: Manages file I/O operations and batch processing
- CLI Interface: Command-line argument parsing and user interaction
Error Handling
The skill includes comprehensive error handling for:
- File not found or permission errors
- Invalid encoding or corrupted text files
- Memory limitations for very large files
- Output directory creation and write permissions
- Invalid command-line arguments and parameters
Performance Considerations
- Efficient memory usage for large text files through streaming
- Optimized word counting using dictionary lookups
- Batch processing with progress reporting for large datasets
- Configurable encoding detection for international text
Contributing
This skill serves as a reference implementation and contributions are welcome to demonstrate best practices:
1. Follow PEP 8 coding standards 2. Include comprehensive docstrings 3. Add test cases with sample data 4. Update documentation for any new features 5. Ensure backward compatibility
Limitations
As a BASIC tier skill, some advanced features are intentionally omitted:
- Complex text analysis (sentiment, language detection)
- Advanced file format support (PDF, Word documents)
- Database integration or external API calls
- Parallel processing for very large datasets
This skill demonstrates the essential structure and quality standards required for BASIC tier skills in the claude-skills ecosystem while remaining simple and focused on core functionality.
This is a sample text file for testing the text processor skill.
It contains multiple lines of text with various words and punctuation.
The quick brown fox jumps over the lazy dog.
This sentence contains all 26 letters of the English alphabet.
Some additional content:
- Numbers: 123, 456, 789
- Special characters: !@#$%^&*()
- Mixed case: CamelCase, snake_case, PascalCase
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
Ut enim ad minim veniam, quis nostrud exercitation ullamco.
This file serves as a basic test case for:
1. Word counting functionality
2. Character analysis
3. Line counting
4. Text transformations
5. Statistical analysis
The text processor should handle this content correctly and produce
meaningful statistics and transformations for testing purposes.name,age,city,country
John Doe,25,New York,USA
Jane Smith,30,London,UK
Bob Johnson,22,Toronto,Canada
Alice Brown,28,Sydney,Australia
Charlie Wilson,35,Berlin,Germany
This CSV file contains sample data with headers and multiple rows.
It can be used to test the text processor's ability to handle
structured data formats and count words across different content types.
The file includes:
- Header row with column names
- Data rows with mixed text and numbers
- Various city and country names
- Different age values for statistical analysis{
"file": "assets/sample_text.txt",
"file_size": 855,
"total_words": 116,
"unique_words": 87,
"total_characters": 855,
"lines": 19,
"average_word_length": 4.7,
"most_frequent": {
"word": "the",
"count": 5
}
}Sample Text Processor
A basic text processing skill that demonstrates BASIC tier requirements for the claude-skills ecosystem.
Quick Start
# Analyze a text file
python scripts/text_processor.py analyze sample.txt
# Get JSON output
python scripts/text_processor.py analyze sample.txt --format json
# Transform text to uppercase
python scripts/text_processor.py transform sample.txt --mode upper
# Process multiple files
python scripts/text_processor.py batch text_files/ --verboseFeatures
- Word count and text statistics
- Text transformations (upper, lower, title, reverse)
- Batch file processing
- JSON and human-readable output formats
- Comprehensive error handling
Requirements
- Python 3.7 or later
- No external dependencies (standard library only)
Usage
See SKILL.md for comprehensive documentation and examples.
Testing
Sample data files are provided in the assets/ directory for testing the functionality.
Text Processor API Reference
Classes
TextProcessor
Main class for text processing operations.
__init__(self, encoding: str = 'utf-8')
Initialize the text processor with specified encoding.
Parameters:
encoding(str): Character encoding for file operations. Default: 'utf-8'
analyze_text(self, text: str) -> Dict[str, Any]
Analyze text and return comprehensive statistics.
Parameters:
text(str): Text content to analyze
Returns:
dict: Statistics including word count, character count, lines, most frequent word
Example:
processor = TextProcessor()
stats = processor.analyze_text("Hello world")
# Returns: {'total_words': 2, 'unique_words': 2, ...}transform_text(self, text: str, mode: str) -> str
Transform text according to specified mode.
Parameters:
text(str): Text to transformmode(str): Transformation mode ('upper', 'lower', 'title', 'reverse')
Returns:
str: Transformed text
Raises:
ValueError: If mode is not supported
OutputFormatter
Static methods for output formatting.
format_json(data: Dict[str, Any]) -> str
Format data as JSON string.
format_human_readable(data: Dict[str, Any]) -> str
Format data as human-readable text.
FileManager
Handles file operations and batch processing.
find_text_files(self, directory: str) -> List[str]
Find all text files in a directory recursively.
Supported Extensions:
- .txt
- .md
- .rst
- .csv
- .log
Command Line Interface
Commands
analyze
Analyze text file statistics.
python text_processor.py analyze <file> [options]transform
Transform text file content.
python text_processor.py transform <file> --mode <mode> [options]batch
Process multiple files in a directory.
python text_processor.py batch <directory> [options]Global Options
--format {json,text}: Output format (default: text)--output FILE: Output file path (default: stdout)--encoding ENCODING: Text file encoding (default: utf-8)--verbose: Enable verbose output
Error Handling
The text processor handles several error conditions:
- FileNotFoundError: When input file doesn't exist
- UnicodeDecodeError: When file encoding doesn't match specified encoding
- PermissionError: When file access is denied
- ValueError: When invalid transformation mode is specified
All errors are reported to stderr with descriptive messages.
#!/usr/bin/env python3
"""
Sample Text Processor - Basic text analysis and transformation tool
This script demonstrates the basic structure and functionality expected in
BASIC tier skills. It provides text processing capabilities with proper
argument parsing, error handling, and dual output formats.
Usage:
python text_processor.py analyze <file> [options]
python text_processor.py transform <file> --mode <mode> [options]
python text_processor.py batch <directory> [options]
Author: Claude Skills Engineering Team
Version: 1.0.0
Dependencies: Python Standard Library Only
"""
import argparse
import json
import os
import sys
from collections import Counter
from pathlib import Path
from typing import Dict, List, Any, Optional
class TextProcessor:
"""Core text processing functionality"""
def __init__(self, encoding: str = 'utf-8'):
self.encoding = encoding
def analyze_text(self, text: str) -> Dict[str, Any]:
"""Analyze text and return statistics"""
lines = text.split('\n')
words = text.lower().split()
# Calculate basic statistics
stats = {
'total_words': len(words),
'unique_words': len(set(words)),
'total_characters': len(text),
'lines': len(lines),
'average_word_length': sum(len(word) for word in words) / len(words) if words else 0
}
# Find most frequent word
if words:
word_counts = Counter(words)
most_common = word_counts.most_common(1)[0]
stats['most_frequent'] = {
'word': most_common[0],
'count': most_common[1]
}
else:
stats['most_frequent'] = {'word': '', 'count': 0}
return stats
def transform_text(self, text: str, mode: str) -> str:
"""Transform text according to specified mode"""
if mode == 'upper':
return text.upper()
elif mode == 'lower':
return text.lower()
elif mode == 'title':
return text.title()
elif mode == 'reverse':
return text[::-1]
else:
raise ValueError(f"Unknown transformation mode: {mode}")
def process_file(self, file_path: str) -> Dict[str, Any]:
"""Process a single text file"""
try:
with open(file_path, 'r', encoding=self.encoding) as file:
content = file.read()
stats = self.analyze_text(content)
stats['file'] = file_path
stats['file_size'] = os.path.getsize(file_path)
return stats
except FileNotFoundError:
raise FileNotFoundError(f"File not found: {file_path}")
except UnicodeDecodeError:
raise UnicodeDecodeError(f"Cannot decode file with {self.encoding} encoding: {file_path}")
except PermissionError:
raise PermissionError(f"Permission denied accessing file: {file_path}")
class OutputFormatter:
"""Handles dual output format generation"""
@staticmethod
def format_json(data: Dict[str, Any]) -> str:
"""Format data as JSON"""
return json.dumps(data, indent=2, ensure_ascii=False)
@staticmethod
def format_human_readable(data: Dict[str, Any]) -> str:
"""Format data as human-readable text"""
lines = []
lines.append("=== TEXT ANALYSIS RESULTS ===")
lines.append(f"File: {data.get('file', 'Unknown')}")
lines.append(f"File size: {data.get('file_size', 0)} bytes")
lines.append(f"Total words: {data.get('total_words', 0)}")
lines.append(f"Unique words: {data.get('unique_words', 0)}")
lines.append(f"Total characters: {data.get('total_characters', 0)}")
lines.append(f"Lines: {data.get('lines', 0)}")
lines.append(f"Average word length: {data.get('average_word_length', 0):.1f}")
most_frequent = data.get('most_frequent', {})
lines.append(f"Most frequent word: \"{most_frequent.get('word', '')}\" ({most_frequent.get('count', 0)} occurrences)")
return "\n".join(lines)
class FileManager:
"""Manages file I/O operations and batch processing"""
def __init__(self, verbose: bool = False):
self.verbose = verbose
def log_verbose(self, message: str):
"""Log verbose message if verbose mode enabled"""
if self.verbose:
print(f"[INFO] {message}", file=sys.stderr)
def find_text_files(self, directory: str) -> List[str]:
"""Find all text files in directory"""
text_extensions = {'.txt', '.md', '.rst', '.csv', '.log'}
text_files = []
try:
for file_path in Path(directory).rglob('*'):
if file_path.is_file() and file_path.suffix.lower() in text_extensions:
text_files.append(str(file_path))
except PermissionError:
raise PermissionError(f"Permission denied accessing directory: {directory}")
return text_files
def write_output(self, content: str, output_path: Optional[str] = None):
"""Write content to file or stdout"""
if output_path:
try:
# Create directory if needed
output_dir = os.path.dirname(output_path)
if output_dir and not os.path.exists(output_dir):
os.makedirs(output_dir)
with open(output_path, 'w', encoding='utf-8') as file:
file.write(content)
self.log_verbose(f"Output written to: {output_path}")
except PermissionError:
raise PermissionError(f"Permission denied writing to: {output_path}")
else:
print(content)
def analyze_command(args: argparse.Namespace) -> int:
"""Handle analyze command"""
try:
processor = TextProcessor(args.encoding)
file_manager = FileManager(args.verbose)
file_manager.log_verbose(f"Analyzing file: {args.file}")
# Process the file
results = processor.process_file(args.file)
# Format output
if args.format == 'json':
output = OutputFormatter.format_json(results)
else:
output = OutputFormatter.format_human_readable(results)
# Write output
file_manager.write_output(output, args.output)
return 0
except FileNotFoundError as e:
print(f"Error: {e}", file=sys.stderr)
return 1
except UnicodeDecodeError as e:
print(f"Error: {e}", file=sys.stderr)
print(f"Try using --encoding option with different encoding", file=sys.stderr)
return 1
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
return 1
def transform_command(args: argparse.Namespace) -> int:
"""Handle transform command"""
try:
processor = TextProcessor(args.encoding)
file_manager = FileManager(args.verbose)
file_manager.log_verbose(f"Transforming file: {args.file}")
# Read and transform the file
with open(args.file, 'r', encoding=args.encoding) as file:
content = file.read()
transformed = processor.transform_text(content, args.mode)
# Write transformed content
file_manager.write_output(transformed, args.output)
return 0
except FileNotFoundError as e:
print(f"Error: {e}", file=sys.stderr)
return 1
except ValueError as e:
print(f"Error: {e}", file=sys.stderr)
return 1
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
return 1
def batch_command(args: argparse.Namespace) -> int:
"""Handle batch command"""
try:
processor = TextProcessor(args.encoding)
file_manager = FileManager(args.verbose)
file_manager.log_verbose(f"Finding text files in: {args.directory}")
# Find all text files
text_files = file_manager.find_text_files(args.directory)
if not text_files:
print(f"No text files found in directory: {args.directory}", file=sys.stderr)
return 1
file_manager.log_verbose(f"Found {len(text_files)} text files")
# Process all files
all_results = []
for i, file_path in enumerate(text_files, 1):
try:
file_manager.log_verbose(f"Processing {i}/{len(text_files)}: {file_path}")
results = processor.process_file(file_path)
all_results.append(results)
except Exception as e:
print(f"Warning: Failed to process {file_path}: {e}", file=sys.stderr)
continue
if not all_results:
print("Error: No files could be processed successfully", file=sys.stderr)
return 1
# Format batch results
batch_summary = {
'total_files': len(all_results),
'total_words': sum(r.get('total_words', 0) for r in all_results),
'total_characters': sum(r.get('total_characters', 0) for r in all_results),
'files': all_results
}
if args.format == 'json':
output = OutputFormatter.format_json(batch_summary)
else:
lines = []
lines.append("=== BATCH PROCESSING RESULTS ===")
lines.append(f"Total files processed: {batch_summary['total_files']}")
lines.append(f"Total words across all files: {batch_summary['total_words']}")
lines.append(f"Total characters across all files: {batch_summary['total_characters']}")
lines.append("")
lines.append("Individual file results:")
for result in all_results:
lines.append(f" {result['file']}: {result['total_words']} words")
output = "\n".join(lines)
# Write output
file_manager.write_output(output, args.output)
return 0
except PermissionError as e:
print(f"Error: {e}", file=sys.stderr)
return 1
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
return 1
def main():
"""Main entry point with argument parsing"""
parser = argparse.ArgumentParser(
description="Sample Text Processor - Basic text analysis and transformation",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
Analysis:
python text_processor.py analyze document.txt
python text_processor.py analyze document.txt --format json --output results.json
Transformation:
python text_processor.py transform document.txt --mode upper
python text_processor.py transform document.txt --mode title --output transformed.txt
Batch processing:
python text_processor.py batch text_files/ --verbose
python text_processor.py batch text_files/ --format json --output batch_results.json
Transformation modes:
upper - Convert to uppercase
lower - Convert to lowercase
title - Convert to title case
reverse - Reverse the text
"""
)
parser.add_argument('--format',
choices=['json', 'text'],
default='text',
help='Output format (default: text)')
parser.add_argument('--output',
help='Output file path (default: stdout)')
parser.add_argument('--encoding',
default='utf-8',
help='Text file encoding (default: utf-8)')
parser.add_argument('--verbose',
action='store_true',
help='Enable verbose output')
subparsers = parser.add_subparsers(dest='command', help='Available commands')
# Analyze subcommand
analyze_parser = subparsers.add_parser('analyze', help='Analyze text file statistics')
analyze_parser.add_argument('file', help='Text file to analyze')
# Transform subcommand
transform_parser = subparsers.add_parser('transform', help='Transform text file')
transform_parser.add_argument('file', help='Text file to transform')
transform_parser.add_argument('--mode',
required=True,
choices=['upper', 'lower', 'title', 'reverse'],
help='Transformation mode')
# Batch subcommand
batch_parser = subparsers.add_parser('batch', help='Process multiple files')
batch_parser.add_argument('directory', help='Directory containing text files')
args = parser.parse_args()
if not args.command:
parser.print_help()
return 1
try:
if args.command == 'analyze':
return analyze_command(args)
elif args.command == 'transform':
return transform_command(args)
elif args.command == 'batch':
return batch_command(args)
else:
print(f"Unknown command: {args.command}", file=sys.stderr)
return 1
except KeyboardInterrupt:
print("\nOperation interrupted by user", file=sys.stderr)
return 130
except Exception as e:
print(f"Unexpected error: {e}", file=sys.stderr)
return 1
if __name__ == "__main__":
sys.exit(main()){
"skill_path": "assets/sample-skill",
"timestamp": "2026-02-16T16:41:00Z",
"overall_score": 85.0,
"compliance_level": "GOOD",
"checks": {
"skill_md_exists": {
"passed": true,
"message": "SKILL.md found",
"score": 1.0
},
"readme_exists": {
"passed": true,
"message": "README.md found",
"score": 1.0
},
"skill_md_length": {
"passed": true,
"message": "SKILL.md has 145 lines (≥100)",
"score": 1.0
},
"frontmatter_complete": {
"passed": true,
"message": "All required frontmatter fields present",
"score": 1.0
},
"required_sections": {
"passed": true,
"message": "All required sections present",
"score": 1.0
},
"dir_scripts_exists": {
"passed": true,
"message": "scripts/ directory found",
"score": 1.0
},
"min_scripts_count": {
"passed": true,
"message": "Found 1 Python scripts (≥1)",
"score": 1.0
},
"script_syntax_text_processor.py": {
"passed": true,
"message": "text_processor.py has valid Python syntax",
"score": 1.0
},
"script_argparse_text_processor.py": {
"passed": true,
"message": "Uses argparse in text_processor.py",
"score": 1.0
},
"script_main_guard_text_processor.py": {
"passed": true,
"message": "Has main guard in text_processor.py",
"score": 1.0
},
"tier_compliance": {
"passed": true,
"message": "Meets BASIC tier requirements",
"score": 1.0
}
},
"warnings": [],
"errors": [],
"suggestions": [
"Consider adding optional directories: references, expected_outputs"
]
}Skill Tester - Quality Assurance Meta-Skill
A POWERFUL-tier skill that provides comprehensive validation, testing, and quality scoring for skills in the claude-skills ecosystem.
Overview
The Skill Tester is a meta-skill that ensures quality and consistency across all skills in the repository through:
- Structure Validation - Verifies directory structure, file presence, and documentation standards
- Script Testing - Tests Python scripts for syntax, functionality, and compliance
- Quality Scoring - Provides comprehensive quality assessment across multiple dimensions
Quick Start
Validate a Skill
# Basic validation
python scripts/skill_validator.py engineering/my-skill
# Validate against specific tier
python scripts/skill_validator.py engineering/my-skill --tier POWERFUL --jsonTest Scripts
# Test all scripts in a skill
python scripts/script_tester.py engineering/my-skill
# Test with custom timeout
python scripts/script_tester.py engineering/my-skill --timeout 60 --jsonScore Quality
# Get quality assessment
python scripts/quality_scorer.py engineering/my-skill
# Detailed scoring with improvement suggestions
python scripts/quality_scorer.py engineering/my-skill --detailed --jsonComponents
Scripts
- skill_validator.py (700+ LOC) - Validates skill structure and compliance
- script_tester.py (800+ LOC) - Tests script functionality and quality
- quality_scorer.py (1100+ LOC) - Multi-dimensional quality assessment
Reference Documentation
- skill-structure-specification.md - Complete structural requirements
- tier-requirements-matrix.md - Tier-specific quality standards
- quality-scoring-rubric.md - Detailed scoring methodology
Sample Assets
- sample-skill/ - Complete sample skill for testing the tester itself
Features
Validation Capabilities
- SKILL.md format and content validation
- Directory structure compliance checking
- Python script syntax and import validation
- Argparse implementation verification
- Tier-specific requirement enforcement
Testing Framework
- Syntax validation using AST parsing
- Import analysis for external dependencies
- Runtime execution testing with timeout protection
- Help functionality verification
- Sample data processing validation
- Output format compliance checking
Quality Assessment
- Documentation quality scoring (25%)
- Code quality evaluation (25%)
- Completeness assessment (25%)
- Usability analysis (25%)
- Letter grade assignment (A+ to F)
- Tier recommendation generation
- Improvement roadmap creation
CI/CD Integration
GitHub Actions Example
name: Skill Quality Gate
on:
pull_request:
paths: ['engineering/**']
jobs:
validate-skills:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Validate Skills
run: |
for skill in $(git diff --name-only ${{ github.event.before }} | grep -E '^engineering/[^/]+/' | cut -d'/' -f1-2 | sort -u); do
python engineering/skill-tester/scripts/skill_validator.py $skill --json
python engineering/skill-tester/scripts/script_tester.py $skill
python engineering/skill-tester/scripts/quality_scorer.py $skill --minimum-score 75
donePre-commit Hook
#!/bin/bash
# .git/hooks/pre-commit
python engineering/skill-tester/scripts/skill_validator.py engineering/my-skill --tier STANDARD
if [ $? -ne 0 ]; then
echo "Skill validation failed. Commit blocked."
exit 1
fiQuality Standards
All Scripts
- Zero External Dependencies - Python standard library only
- Comprehensive Error Handling - Meaningful error messages and recovery
- Dual Output Support - Both JSON and human-readable formats
- Proper Documentation - Comprehensive docstrings and comments
- CLI Best Practices - Full argparse implementation with help text
Validation Accuracy
- Structure Checks - 100% accurate directory and file validation
- Content Analysis - Deep parsing of SKILL.md and documentation
- Code Analysis - AST-based Python code validation
- Compliance Scoring - Objective, repeatable quality assessment
Self-Testing
The skill-tester can validate itself:
# Validate the skill-tester structure
python scripts/skill_validator.py . --tier POWERFUL
# Test the skill-tester scripts
python scripts/script_tester.py .
# Score the skill-tester quality
python scripts/quality_scorer.py . --detailedAdvanced Usage
Batch Validation
# Validate all skills in repository
find engineering/ -maxdepth 1 -type d | while read skill; do
echo "Validating $skill..."
python engineering/skill-tester/scripts/skill_validator.py "$skill"
doneQuality Monitoring
# Generate quality report for all skills
python engineering/skill-tester/scripts/quality_scorer.py engineering/ \
--batch --json > quality_report.jsonCustom Scoring Thresholds
# Enforce minimum quality scores
python scripts/quality_scorer.py engineering/my-skill --minimum-score 80
# Exit code 0 = passed, 1 = failed, 2 = needs improvementError Handling
All scripts provide comprehensive error handling:
- File System Errors - Missing files, permission issues, invalid paths
- Content Errors - Malformed YAML, invalid JSON, encoding issues
- Execution Errors - Script timeouts, runtime failures, import errors
- Validation Errors - Standards violations, compliance failures
Output Formats
Human-Readable
=== SKILL VALIDATION REPORT ===
Skill: engineering/my-skill
Overall Score: 85.2/100 (B+)
Tier Recommendation: STANDARD
STRUCTURE VALIDATION:
✓ PASS: SKILL.md found
✓ PASS: README.md found
✓ PASS: scripts/ directory found
SUGGESTIONS:
• Add references/ directory
• Improve error handling in main.pyJSON Format
{
"skill_path": "engineering/my-skill",
"overall_score": 85.2,
"letter_grade": "B+",
"tier_recommendation": "STANDARD",
"dimensions": {
"Documentation": {"score": 88.5, "weight": 0.25},
"Code Quality": {"score": 82.0, "weight": 0.25},
"Completeness": {"score": 85.5, "weight": 0.25},
"Usability": {"score": 84.8, "weight": 0.25}
}
}Requirements
- Python 3.7+ - No external dependencies required
- File System Access - Read access to skill directories
- Execution Permissions - Ability to run Python scripts for testing
Contributing
See SKILL.md for comprehensive documentation and contribution guidelines.
The skill-tester itself serves as a reference implementation of POWERFUL-tier quality standards.
Quality Scoring Rubric
Version: 1.0.0 Last Updated: 2026-02-16 Authority: Claude Skills Engineering Team
Overview
This document defines the comprehensive quality scoring methodology used to assess skills within the claude-skills ecosystem. The scoring system evaluates four key dimensions, each weighted equally at 25%, to provide an objective and consistent measure of skill quality.
Scoring Framework
Overall Scoring Scale
- A+ (95-100): Exceptional quality, exceeds all standards
- A (90-94): Excellent quality, meets highest standards consistently
- A- (85-89): Very good quality, minor areas for improvement
- B+ (80-84): Good quality, meets most standards well
- B (75-79): Satisfactory quality, meets standards adequately
- B- (70-74): Below average, several areas need improvement
- C+ (65-69): Poor quality, significant improvements needed
- C (60-64): Minimal acceptable quality, major improvements required
- C- (55-59): Unacceptable quality, extensive rework needed
- D (50-54): Very poor quality, fundamental issues present
- F (0-49): Failing quality, does not meet basic standards
Dimension Weights
Each dimension contributes equally to the overall score:
- Documentation Quality: 25%
- Code Quality: 25%
- Completeness: 25%
- Usability: 25%
Documentation Quality (25% Weight)
Scoring Components
SKILL.md Quality (40% of Documentation Score)
Component Breakdown:
- Length and Depth (25%): Line count and content substance
- Frontmatter Quality (25%): Completeness and accuracy of YAML metadata
- Section Coverage (25%): Required and recommended section presence
- Content Depth (25%): Technical detail and comprehensiveness
Scoring Criteria:
| Score Range | Length | Frontmatter | Sections | Depth |
|---|---|---|---|---|
| 90-100 | 400+ lines | All fields complete + extras | All required + 4+ recommended | Rich technical detail, examples |
| 80-89 | 300-399 lines | All required fields complete | All required + 2-3 recommended | Good technical coverage |
| 70-79 | 200-299 lines | Most required fields | All required + 1 recommended | Adequate technical content |
| 60-69 | 150-199 lines | Some required fields | Most required sections | Basic technical information |
| 50-59 | 100-149 lines | Minimal frontmatter | Some required sections | Limited technical detail |
| Below 50 | <100 lines | Missing/invalid frontmatter | Few/no required sections | Insufficient content |
README.md Quality (25% of Documentation Score)
Scoring Criteria:
- Excellent (90-100): 1000+ chars, comprehensive usage guide, examples, troubleshooting
- Good (75-89): 500-999 chars, clear usage instructions, basic examples
- Satisfactory (60-74): 200-499 chars, minimal usage information
- Poor (40-59): <200 chars or confusing content
- Failing (0-39): Missing or completely inadequate
Reference Documentation (20% of Documentation Score)
Scoring Criteria:
- Excellent (90-100): Multiple comprehensive reference docs (2000+ chars total)
- Good (75-89): 2-3 reference files with substantial content
- Satisfactory (60-74): 1-2 reference files with adequate content
- Poor (40-59): Minimal reference content or poor quality
- Failing (0-39): No reference documentation
Examples and Usage Clarity (15% of Documentation Score)
Scoring Criteria:
- Excellent (90-100): 5+ diverse examples, clear usage patterns
- Good (75-89): 3-4 examples covering different scenarios
- Satisfactory (60-74): 2-3 basic examples
- Poor (40-59): 1-2 minimal examples
- Failing (0-39): No examples or unclear usage
Code Quality (25% Weight)
Scoring Components
Script Complexity and Architecture (25% of Code Score)
Evaluation Criteria:
- Lines of code per script relative to tier requirements
- Function and class organization
- Code modularity and reusability
- Algorithm sophistication
Scoring Matrix:
| Tier | Excellent (90-100) | Good (75-89) | Satisfactory (60-74) | Poor (Below 60) |
|---|---|---|---|---|
| BASIC | 200-300 LOC, well-structured | 150-199 LOC, organized | 100-149 LOC, basic | <100 LOC, minimal |
| STANDARD | 400-500 LOC, modular | 350-399 LOC, structured | 300-349 LOC, adequate | <300 LOC, basic |
| POWERFUL | 600-800 LOC, sophisticated | 550-599 LOC, advanced | 500-549 LOC, solid | <500 LOC, simple |
Error Handling Quality (25% of Code Score)
Scoring Criteria:
- Excellent (90-100): Comprehensive exception handling, specific error types, recovery mechanisms
- Good (75-89): Good exception handling, meaningful error messages, logging
- Satisfactory (60-74): Basic try/except blocks, simple error messages
- Poor (40-59): Minimal error handling, generic exceptions
- Failing (0-39): No error handling or inappropriate handling
Error Handling Checklist:
- [ ] Try/except blocks for risky operations
- [ ] Specific exception types (not just Exception)
- [ ] Meaningful error messages for users
- [ ] Proper error logging or reporting
- [ ] Graceful degradation where possible
- [ ] Input validation and sanitization
Code Structure and Organization (25% of Code Score)
Evaluation Elements:
- Function decomposition and single responsibility
- Class design and inheritance patterns
- Import organization and dependency management
- Documentation and comments quality
- Consistent naming conventions
- PEP 8 compliance
Scoring Guidelines:
- Excellent (90-100): Exemplary structure, comprehensive docstrings, perfect style
- Good (75-89): Well-organized, good documentation, minor style issues
- Satisfactory (60-74): Adequate structure, basic documentation, some style issues
- Poor (40-59): Poor organization, minimal documentation, style problems
- Failing (0-39): No clear structure, no documentation, major style violations
Output Format Support (25% of Code Score)
Required Capabilities:
- JSON output format support
- Human-readable output format
- Proper data serialization
- Consistent output structure
- Error output handling
Scoring Criteria:
- Excellent (90-100): Dual format + custom formats, perfect serialization
- Good (75-89): Dual format support, good serialization
- Satisfactory (60-74): Single format well-implemented
- Poor (40-59): Basic output, formatting issues
- Failing (0-39): Poor or no structured output
Completeness (25% Weight)
Scoring Components
Directory Structure Compliance (25% of Completeness Score)
Required Directories by Tier:
- BASIC: scripts/ (required), assets/ + references/ (recommended)
- STANDARD: scripts/ + assets/ + references/ (required), expected_outputs/ (recommended)
- POWERFUL: scripts/ + assets/ + references/ + expected_outputs/ (all required)
Scoring Calculation:
Structure Score = (Required Present / Required Total) * 0.6 +
(Recommended Present / Recommended Total) * 0.4Asset Availability and Quality (25% of Completeness Score)
Scoring Criteria:
- Excellent (90-100): 5+ diverse assets, multiple file types, realistic data
- Good (75-89): 3-4 assets, some diversity, good quality
- Satisfactory (60-74): 2-3 assets, basic variety
- Poor (40-59): 1-2 minimal assets
- Failing (0-39): No assets or unusable assets
Asset Quality Factors:
- File diversity (JSON, CSV, YAML, etc.)
- Data realism and complexity
- Coverage of use cases
- File size appropriateness
- Documentation of asset purpose
Expected Output Coverage (25% of Completeness Score)
Evaluation Criteria:
- Correspondence with asset files
- Coverage of success and error scenarios
- Output format variety
- Reproducibility and accuracy
Scoring Matrix:
- Excellent (90-100): Complete output coverage, all scenarios, verified accuracy
- Good (75-89): Good coverage, most scenarios, mostly accurate
- Satisfactory (60-74): Basic coverage, main scenarios
- Poor (40-59): Minimal coverage, some inaccuracies
- Failing (0-39): No expected outputs or completely inaccurate
Test Coverage and Validation (25% of Completeness Score)
Assessment Areas:
- Sample data processing capability
- Output verification mechanisms
- Edge case handling
- Error condition testing
- Integration test scenarios
Scoring Guidelines:
- Excellent (90-100): Comprehensive test coverage, automated validation
- Good (75-89): Good test coverage, manual validation possible
- Satisfactory (60-74): Basic testing capability
- Poor (40-59): Minimal testing support
- Failing (0-39): No testing or validation capability
Usability (25% Weight)
Scoring Components
Installation and Setup Simplicity (25% of Usability Score)
Evaluation Factors:
- Dependency requirements (Python stdlib preferred)
- Setup complexity
- Environment requirements
- Installation documentation clarity
Scoring Criteria:
- Excellent (90-100): Zero external dependencies, single-file execution
- Good (75-89): Minimal dependencies, simple setup
- Satisfactory (60-74): Some dependencies, documented setup
- Poor (40-59): Complex dependencies, unclear setup
- Failing (0-39): Unable to install or excessive complexity
Usage Clarity and Help Quality (25% of Usability Score)
Assessment Elements:
- Command-line help comprehensiveness
- Usage example clarity
- Parameter documentation quality
- Error message helpfulness
Help Quality Checklist:
- [ ] Comprehensive --help output
- [ ] Clear parameter descriptions
- [ ] Usage examples included
- [ ] Error messages are actionable
- [ ] Progress indicators where appropriate
Scoring Matrix:
- Excellent (90-100): Exemplary help, multiple examples, perfect error messages
- Good (75-89): Good help quality, clear examples, helpful errors
- Satisfactory (60-74): Adequate help, basic examples
- Poor (40-59): Minimal help, confusing interface
- Failing (0-39): No help or completely unclear interface
Documentation Accessibility (25% of Usability Score)
Evaluation Criteria:
- README quick start effectiveness
- SKILL.md navigation and structure
- Reference material organization
- Learning curve considerations
Accessibility Factors:
- Information hierarchy clarity
- Cross-reference quality
- Beginner-friendly explanations
- Advanced user shortcuts
- Troubleshooting guidance
Practical Example Quality (25% of Usability Score)
Assessment Areas:
- Example realism and relevance
- Complexity progression (simple to advanced)
- Output demonstration
- Common use case coverage
- Integration scenarios
Scoring Guidelines:
- Excellent (90-100): 5+ examples, perfect progression, real-world scenarios
- Good (75-89): 3-4 examples, good variety, practical scenarios
- Satisfactory (60-74): 2-3 examples, adequate coverage
- Poor (40-59): 1-2 examples, limited practical value
- Failing (0-39): No examples or completely impractical
Scoring Calculations
Dimension Score Calculation
Each dimension score is calculated as a weighted average of its components:
def calculate_dimension_score(components):
total_weighted_score = 0
total_weight = 0
for component_name, component_data in components.items():
score = component_data['score']
weight = component_data['weight']
total_weighted_score += score * weight
total_weight += weight
return total_weighted_score / total_weight if total_weight > 0 else 0Overall Score Calculation
The overall score combines all dimensions with equal weighting:
def calculate_overall_score(dimensions):
return sum(dimension.score * 0.25 for dimension in dimensions.values())Letter Grade Assignment
def assign_letter_grade(overall_score):
if overall_score >= 95: return "A+"
elif overall_score >= 90: return "A"
elif overall_score >= 85: return "A-"
elif overall_score >= 80: return "B+"
elif overall_score >= 75: return "B"
elif overall_score >= 70: return "B-"
elif overall_score >= 65: return "C+"
elif overall_score >= 60: return "C"
elif overall_score >= 55: return "C-"
elif overall_score >= 50: return "D"
else: return "F"Quality Improvement Recommendations
Score-Based Recommendations
For Scores Below 60 (C- or Lower)
Priority Actions: 1. Address fundamental structural issues 2. Implement basic error handling 3. Add essential documentation sections 4. Create minimal viable examples 5. Fix critical functionality issues
For Scores 60-74 (C+ to B-)
Improvement Areas: 1. Expand documentation comprehensiveness 2. Enhance error handling sophistication 3. Add more diverse examples and use cases 4. Improve code organization and structure 5. Increase test coverage and validation
For Scores 75-84 (B to B+)
Enhancement Opportunities: 1. Refine documentation for expert-level quality 2. Implement advanced error recovery mechanisms 3. Add comprehensive reference materials 4. Optimize code architecture and performance 5. Develop extensive example library
For Scores 85+ (A- or Higher)
Excellence Maintenance: 1. Regular quality audits and updates 2. Community feedback integration 3. Best practice evolution tracking 4. Mentoring lower-quality skills 5. Innovation and cutting-edge feature adoption
Dimension-Specific Improvement Strategies
Low Documentation Scores
- Expand SKILL.md with technical details
- Add comprehensive API reference
- Include architecture diagrams and explanations
- Develop troubleshooting guides
- Create contributor documentation
Low Code Quality Scores
- Refactor for better modularity
- Implement comprehensive error handling
- Add extensive code documentation
- Apply advanced design patterns
- Optimize performance and efficiency
Low Completeness Scores
- Add missing directories and files
- Develop comprehensive sample datasets
- Create expected output libraries
- Implement automated testing
- Add integration examples
Low Usability Scores
- Simplify installation process
- Improve command-line interface design
- Enhance help text and documentation
- Create beginner-friendly tutorials
- Add interactive examples
Quality Assurance Process
Automated Scoring
The quality scorer runs automated assessments based on this rubric: 1. File system analysis for structure compliance 2. Content analysis for documentation quality 3. Code analysis for quality metrics 4. Asset inventory and quality assessment
Manual Review Process
Human reviewers validate automated scores and provide qualitative insights: 1. Content quality assessment beyond automated metrics 2. Usability testing with real-world scenarios 3. Technical accuracy verification 4. Community value assessment
Continuous Improvement
The scoring rubric evolves based on:
- Community feedback and usage patterns
- Industry best practice changes
- Tool capability enhancements
- Quality trend analysis
This quality scoring rubric ensures consistent, objective, and comprehensive assessment of all skills within the claude-skills ecosystem while providing clear guidance for quality improvement.
Skill Structure Specification
Version: 1.0.0 Last Updated: 2026-02-16 Authority: Claude Skills Engineering Team
Overview
This document defines the mandatory and optional components that constitute a well-formed skill within the claude-skills ecosystem. All skills must adhere to these structural requirements to ensure consistency, maintainability, and quality across the repository.
Directory Structure
Mandatory Components
skill-name/
├── SKILL.md # Primary skill documentation (REQUIRED)
├── README.md # Usage instructions and quick start (REQUIRED)
└── scripts/ # Python implementation scripts (REQUIRED)
└── *.py # At least one Python scriptRecommended Components
skill-name/
├── SKILL.md
├── README.md
├── scripts/
│ └── *.py
├── assets/ # Sample data and input files (RECOMMENDED)
│ ├── samples/
│ ├── examples/
│ └── data/
├── references/ # Reference documentation (RECOMMENDED)
│ ├── api-reference.md
│ ├── specifications.md
│ └── external-links.md
└── expected_outputs/ # Expected results for testing (RECOMMENDED)
├── sample_output.json
├── example_results.txt
└── test_cases/Optional Components
skill-name/
├── [mandatory and recommended components]
├── tests/ # Unit tests and validation scripts
├── examples/ # Extended examples and tutorials
├── docs/ # Additional documentation
├── config/ # Configuration files
└── templates/ # Template files for code generationFile Requirements
SKILL.md Requirements
The SKILL.md file serves as the primary documentation for the skill and must contain:
Mandatory YAML Frontmatter
---
Name: skill-name
Tier: [BASIC|STANDARD|POWERFUL]
Category: [Category Name]
Dependencies: [None|List of dependencies]
Author: [Author Name]
Version: [Semantic Version]
Last Updated: [YYYY-MM-DD]
---Required Sections
- Description: Comprehensive overview of the skill's purpose and capabilities
- Features: Detailed list of key features and functionality
- Usage: Instructions for using the skill and its components
- Examples: Practical usage examples with expected outcomes
Recommended Sections
- Architecture: Technical architecture and design decisions
- Installation: Setup and installation instructions
- Configuration: Configuration options and parameters
- Troubleshooting: Common issues and solutions
- Contributing: Guidelines for contributors
- Changelog: Version history and changes
Content Requirements by Tier
- BASIC: Minimum 100 lines of substantial content
- STANDARD: Minimum 200 lines of substantial content
- POWERFUL: Minimum 300 lines of substantial content
README.md Requirements
The README.md file provides quick start instructions and must include:
Mandatory Content
- Brief description of the skill
- Quick start instructions
- Basic usage examples
- Link to full SKILL.md documentation
Recommended Content
- Installation instructions
- Prerequisites and dependencies
- Command-line usage examples
- Troubleshooting section
- Contributing guidelines
Length Requirements
- Minimum 200 characters of substantial content
- Recommended 500+ characters for comprehensive coverage
Scripts Directory Requirements
The scripts/ directory contains all Python implementation files:
Mandatory Requirements
- At least one Python (.py) file
- All scripts must be executable Python 3.7+
- No external dependencies outside Python standard library
- Proper file naming conventions (lowercase, hyphens for separation)
Script Content Requirements
- Shebang line:
#!/usr/bin/env python3 - Module docstring: Comprehensive description of script purpose
- Argparse implementation: Command-line argument parsing
- Main guard:
if __name__ == "__main__":protection - Error handling: Appropriate exception handling and user feedback
- Dual output: Support for both JSON and human-readable output formats
Script Size Requirements by Tier
- BASIC: 100-300 lines of code per script
- STANDARD: 300-500 lines of code per script
- POWERFUL: 500-800 lines of code per script
Assets Directory Structure
The assets/ directory contains sample data and supporting files:
assets/
├── samples/ # Sample input data
│ ├── simple_example.json
│ ├── complex_dataset.csv
│ └── test_configuration.yaml
├── examples/ # Example files demonstrating usage
│ ├── basic_workflow.py
│ ├── advanced_usage.sh
│ └── integration_example.md
└── data/ # Static data files
├── reference_data.json
├── lookup_tables.csv
└── configuration_templates/Content Requirements
- At least 2 sample files demonstrating different use cases
- Files should represent realistic usage scenarios
- Include both simple and complex examples where applicable
- Provide diverse file formats (JSON, CSV, YAML, etc.)
References Directory Structure
The references/ directory contains detailed reference documentation:
references/
├── api-reference.md # Complete API documentation
├── specifications.md # Technical specifications and requirements
├── external-links.md # Links to related resources
├── algorithms.md # Algorithm descriptions and implementations
└── best-practices.md # Usage best practices and patternsContent Requirements
- Each file should contain substantial technical content (500+ words)
- Include code examples and technical specifications
- Provide external references and links where appropriate
- Maintain consistent documentation format and style
Expected Outputs Directory Structure
The expected_outputs/ directory contains reference outputs for testing:
expected_outputs/
├── basic_example_output.json
├── complex_scenario_result.txt
├── error_cases/
│ ├── invalid_input_error.json
│ └── timeout_error.txt
└── test_cases/
├── unit_test_outputs/
└── integration_test_results/Content Requirements
- Outputs correspond to sample inputs in assets/ directory
- Include both successful and error case examples
- Provide outputs in multiple formats (JSON, text, CSV)
- Ensure outputs are reproducible and verifiable
Naming Conventions
Directory Names
- Use lowercase letters only
- Use hyphens (-) to separate words
- Keep names concise but descriptive
- Avoid special characters and spaces
Examples: data-processor, api-client, ml-trainer
File Names
- Use lowercase letters for Python scripts
- Use hyphens (-) to separate words in script names
- Use underscores (_) only when required by Python conventions
- Use descriptive names that indicate purpose
Examples: data-processor.py, api-client.py, quality_scorer.py
Script Internal Naming
- Use PascalCase for class names
- Use snake_case for function and variable names
- Use UPPER_CASE for constants
- Use descriptive names that indicate purpose
Quality Standards
Documentation Standards
- All documentation must be written in clear, professional English
- Use proper Markdown formatting and structure
- Include code examples with syntax highlighting
- Provide comprehensive coverage of all features
- Maintain consistent terminology throughout
Code Standards
- Follow PEP 8 Python style guidelines
- Include comprehensive docstrings for all functions and classes
- Implement proper error handling with meaningful error messages
- Use type hints where appropriate
- Maintain reasonable code complexity and readability
Testing Standards
- Provide sample data that exercises all major functionality
- Include expected outputs for verification
- Cover both successful and error scenarios
- Ensure reproducible results across different environments
Validation Criteria
Skills are validated against the following criteria:
Structural Validation
- All mandatory files and directories present
- Proper file naming conventions followed
- Directory structure matches specification
- File permissions and accessibility correct
Content Validation
- SKILL.md meets minimum length and section requirements
- README.md provides adequate quick start information
- Scripts contain required components (argparse, main guard, etc.)
- Sample data and expected outputs are complete and realistic
Quality Validation
- Documentation is comprehensive and accurate
- Code follows established style and quality guidelines
- Examples are practical and demonstrate real usage
- Error handling is appropriate and user-friendly
Compliance Levels
Full Compliance
- All mandatory components present and complete
- All recommended components present with substantial content
- Exceeds minimum quality thresholds for tier
- Demonstrates best practices throughout
Partial Compliance
- All mandatory components present
- Most recommended components present
- Meets minimum quality thresholds for tier
- Generally follows established patterns
Non-Compliance
- Missing mandatory components
- Inadequate content quality or length
- Does not meet minimum tier requirements
- Significant deviations from established standards
Migration and Updates
Existing Skills
Skills created before this specification should be updated to comply within:
- POWERFUL tier: 30 days
- STANDARD tier: 60 days
- BASIC tier: 90 days
Specification Updates
- Changes to this specification require team consensus
- Breaking changes must provide 90-day migration period
- All changes must be documented with rationale and examples
- Automated validation tools must be updated accordingly
Tools and Automation
Validation Tools
skill_validator.py- Validates structure and content compliancescript_tester.py- Tests script functionality and qualityquality_scorer.py- Provides comprehensive quality assessment
Integration Points
- Pre-commit hooks for basic validation
- CI/CD pipeline integration for pull request validation
- Automated quality reporting and tracking
- Integration with code review processes
Examples and Templates
Minimal BASIC Tier Example
basic-skill/
├── SKILL.md # 100+ lines
├── README.md # Basic usage instructions
└── scripts/
└── main.py # 100-300 lines with argparseComplete POWERFUL Tier Example
powerful-skill/
├── SKILL.md # 300+ lines with comprehensive sections
├── README.md # Detailed usage and setup
├── scripts/ # Multiple sophisticated scripts
│ ├── main_processor.py # 500-800 lines
│ ├── data_analyzer.py # 500-800 lines
│ └── report_generator.py # 500-800 lines
├── assets/ # Diverse sample data
│ ├── samples/
│ ├── examples/
│ └── data/
├── references/ # Comprehensive documentation
│ ├── api-reference.md
│ ├── specifications.md
│ └── best-practices.md
└── expected_outputs/ # Complete test outputs
├── json_outputs/
├── text_reports/
└── error_cases/This specification serves as the authoritative guide for skill structure within the claude-skills ecosystem. Adherence to these standards ensures consistency, quality, and maintainability across all skills in the repository.
Tier Requirements Matrix
Version: 1.0.0 Last Updated: 2026-02-16 Authority: Claude Skills Engineering Team
Overview
This document provides a comprehensive matrix of requirements for each skill tier within the claude-skills ecosystem. Skills are classified into three tiers based on complexity, functionality, and comprehensiveness: BASIC, STANDARD, and POWERFUL.
Tier Classification Philosophy
BASIC Tier
Entry-level skills that provide fundamental functionality with minimal complexity. Suitable for simple automation tasks, basic data processing, or straightforward utilities.
STANDARD Tier
Intermediate skills that offer enhanced functionality with moderate complexity. Suitable for business processes, advanced data manipulation, or multi-step workflows.
POWERFUL Tier
Advanced skills that provide comprehensive functionality with sophisticated implementation. Suitable for complex systems, enterprise-grade tools, or mission-critical applications.
Requirements Matrix
| Component | BASIC | STANDARD | POWERFUL |
|---|---|---|---|
| SKILL.md Lines | ≥100 | ≥200 | ≥300 |
| Scripts Count | ≥1 | ≥1 | ≥2 |
| Script Size (LOC) | 100-300 | 300-500 | 500-800 |
| Required Directories | scripts | scripts, assets, references | scripts, assets, references, expected_outputs |
| Argparse Implementation | Basic | Advanced | Complex with subcommands |
| Output Formats | Human-readable | JSON + Human-readable | JSON + Human-readable + Custom |
| Error Handling | Basic | Comprehensive | Advanced with recovery |
| Documentation Depth | Functional | Comprehensive | Expert-level |
| Examples Provided | ≥1 | ≥3 | ≥5 |
| Test Coverage | Basic validation | Sample data testing | Comprehensive test suite |
Detailed Requirements by Tier
BASIC Tier Requirements
Documentation Requirements
- SKILL.md: Minimum 100 lines of substantial content
- Required Sections: Name, Description, Features, Usage, Examples
- README.md: Basic usage instructions (200+ characters)
- Content Quality: Clear and functional documentation
- Examples: At least 1 practical usage example
Code Requirements
- Scripts: Minimum 1 Python script (100-300 LOC)
- Argparse: Basic command-line argument parsing
- Main Guard:
if __name__ == "__main__":protection - Dependencies: Python standard library only
- Output: Human-readable format with clear messaging
- Error Handling: Basic exception handling with user-friendly messages
Structure Requirements
- Mandatory Directories:
scripts/ - Recommended Directories:
assets/,references/ - File Organization: Logical file naming and structure
- Assets: Optional sample data files
Quality Standards
- Code Style: Follows basic Python conventions
- Documentation: Adequate coverage of functionality
- Usability: Clear usage instructions and examples
- Completeness: All essential components present
STANDARD Tier Requirements
Documentation Requirements
- SKILL.md: Minimum 200 lines with comprehensive coverage
- Required Sections: All BASIC sections plus Architecture, Installation
- README.md: Detailed usage instructions (500+ characters)
- References: Technical documentation in
references/directory - Content Quality: Professional-grade documentation with technical depth
- Examples: At least 3 diverse usage examples
Code Requirements
- Scripts: 1-2 Python scripts (300-500 LOC each)
- Argparse: Advanced argument parsing with subcommands and validation
- Output Formats: Both JSON and human-readable output support
- Error Handling: Comprehensive exception handling with specific error types
- Code Structure: Well-organized classes and functions
- Documentation: Comprehensive docstrings for all functions
Structure Requirements
- Mandatory Directories:
scripts/,assets/,references/ - Recommended Directories:
expected_outputs/ - Assets: Multiple sample files demonstrating different use cases
- References: Technical specifications and API documentation
- Expected Outputs: Sample results for validation
Quality Standards
- Code Quality: Advanced Python patterns and best practices
- Documentation: Expert-level technical documentation
- Testing: Sample data processing with validation
- Integration: Consideration for CI/CD and automation use
POWERFUL Tier Requirements
Documentation Requirements
- SKILL.md: Minimum 300 lines with expert-level comprehensiveness
- Required Sections: All STANDARD sections plus Troubleshooting, Contributing, Advanced Usage
- README.md: Comprehensive guide with installation and setup (1000+ characters)
- References: Multiple technical documents with specifications
- Content Quality: Publication-ready documentation with architectural details
- Examples: At least 5 examples covering simple to complex scenarios
Code Requirements
- Scripts: 2-3 Python scripts (500-800 LOC each)
- Argparse: Complex argument parsing with multiple modes and configurations
- Output Formats: JSON, human-readable, and custom format support
- Error Handling: Advanced error handling with recovery mechanisms
- Code Architecture: Sophisticated design patterns and modular structure
- Performance: Optimized for efficiency and scalability
Structure Requirements
- Mandatory Directories:
scripts/,assets/,references/,expected_outputs/ - Optional Directories:
tests/,examples/,docs/ - Assets: Comprehensive sample data covering edge cases
- References: Complete technical specification suite
- Expected Outputs: Full test result coverage including error cases
- Testing: Comprehensive validation and test coverage
Quality Standards
- Enterprise Grade: Production-ready code with enterprise patterns
- Documentation: Comprehensive technical documentation suitable for technical teams
- Integration: Full CI/CD integration capabilities
- Maintainability: Designed for long-term maintenance and extension
Tier Assessment Criteria
Automatic Tier Classification
Skills are automatically classified based on quantitative metrics:
def classify_tier(skill_metrics):
if (skill_metrics['skill_md_lines'] >= 300 and
skill_metrics['script_count'] >= 2 and
skill_metrics['min_script_size'] >= 500 and
all_required_dirs_present(['scripts', 'assets', 'references', 'expected_outputs'])):
return 'POWERFUL'
elif (skill_metrics['skill_md_lines'] >= 200 and
skill_metrics['script_count'] >= 1 and
skill_metrics['min_script_size'] >= 300 and
all_required_dirs_present(['scripts', 'assets', 'references'])):
return 'STANDARD'
else:
return 'BASIC'Manual Tier Override
Manual tier assignment may be considered when:
- Skill provides exceptional value despite not meeting all quantitative requirements
- Skill addresses critical infrastructure or security needs
- Skill demonstrates innovative approaches or cutting-edge techniques
- Skill provides essential integration or compatibility functions
Tier Promotion Criteria
Skills may be promoted to higher tiers when:
- All quantitative requirements for higher tier are met
- Quality assessment scores exceed tier thresholds
- Community usage and feedback indicate higher value
- Continuous integration and maintenance demonstrate reliability
Tier Demotion Criteria
Skills may be demoted to lower tiers when:
- Quality degradation below tier standards
- Lack of maintenance or updates
- Compatibility issues or security vulnerabilities
- Community feedback indicates reduced value
Implementation Guidelines by Tier
BASIC Tier Implementation
# Example argparse implementation for BASIC tier
parser = argparse.ArgumentParser(description="Basic skill functionality")
parser.add_argument("input", help="Input file or parameter")
parser.add_argument("--output", help="Output destination")
parser.add_argument("--verbose", action="store_true", help="Verbose output")
# Basic error handling
try:
result = process_input(args.input)
print(f"Processing completed: {result}")
except FileNotFoundError:
print("Error: Input file not found")
sys.exit(1)
except Exception as e:
print(f"Error: {str(e)}")
sys.exit(1)STANDARD Tier Implementation
# Example argparse implementation for STANDARD tier
parser = argparse.ArgumentParser(
description="Standard skill with advanced functionality",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="Examples:\n python script.py input.json --format json\n python script.py data/ --batch --output results/"
)
parser.add_argument("input", help="Input file or directory")
parser.add_argument("--format", choices=["json", "text"], default="json", help="Output format")
parser.add_argument("--batch", action="store_true", help="Process multiple files")
parser.add_argument("--output", help="Output destination")
# Advanced error handling with specific exception types
try:
if args.batch:
results = batch_process(args.input)
else:
results = single_process(args.input)
if args.format == "json":
print(json.dumps(results, indent=2))
else:
print_human_readable(results)
except FileNotFoundError as e:
logging.error(f"File not found: {e}")
sys.exit(1)
except ValueError as e:
logging.error(f"Invalid input: {e}")
sys.exit(2)
except Exception as e:
logging.error(f"Unexpected error: {e}")
sys.exit(1)POWERFUL Tier Implementation
# Example argparse implementation for POWERFUL tier
parser = argparse.ArgumentParser(
description="Powerful skill with comprehensive functionality",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
Basic usage:
python script.py process input.json --output results/
Advanced batch processing:
python script.py batch data/ --format json --parallel 4 --filter "*.csv"
Custom configuration:
python script.py process input.json --config custom.yaml --dry-run
"""
)
subparsers = parser.add_subparsers(dest="command", help="Available commands")
# Process subcommand
process_parser = subparsers.add_parser("process", help="Process single file")
process_parser.add_argument("input", help="Input file path")
process_parser.add_argument("--config", help="Configuration file")
process_parser.add_argument("--dry-run", action="store_true", help="Show what would be done")
# Batch subcommand
batch_parser = subparsers.add_parser("batch", help="Process multiple files")
batch_parser.add_argument("directory", help="Input directory")
batch_parser.add_argument("--parallel", type=int, default=1, help="Number of parallel processes")
batch_parser.add_argument("--filter", help="File filter pattern")
# Comprehensive error handling with recovery
try:
if args.command == "process":
result = process_with_recovery(args.input, args.config, args.dry_run)
elif args.command == "batch":
result = batch_process_with_monitoring(args.directory, args.parallel, args.filter)
else:
parser.print_help()
sys.exit(1)
# Multiple output format support
output_formatter = OutputFormatter(args.format)
output_formatter.write(result, args.output)
except KeyboardInterrupt:
logging.info("Processing interrupted by user")
sys.exit(130)
except ProcessingError as e:
logging.error(f"Processing failed: {e}")
if e.recoverable:
logging.info("Attempting recovery...")
# Recovery logic here
sys.exit(1)
except ValidationError as e:
logging.error(f"Validation failed: {e}")
logging.info("Check input format and try again")
sys.exit(2)
except Exception as e:
logging.critical(f"Critical error: {e}")
logging.info("Please report this issue")
sys.exit(1)Quality Scoring by Tier
Scoring Thresholds
- POWERFUL Tier: Overall score ≥80, all dimensions ≥75
- STANDARD Tier: Overall score ≥70, 3+ dimensions ≥65
- BASIC Tier: Overall score ≥60, meets minimum requirements
Dimension Weights (All Tiers)
- Documentation: 25%
- Code Quality: 25%
- Completeness: 25%
- Usability: 25%
Tier-Specific Quality Expectations
BASIC Tier Quality Profile
- Documentation: Functional and clear (60+ points expected)
- Code Quality: Clean and maintainable (60+ points expected)
- Completeness: Essential components present (60+ points expected)
- Usability: Easy to understand and use (60+ points expected)
STANDARD Tier Quality Profile
- Documentation: Professional and comprehensive (70+ points expected)
- Code Quality: Advanced patterns and best practices (70+ points expected)
- Completeness: All recommended components (70+ points expected)
- Usability: Well-designed user experience (70+ points expected)
POWERFUL Tier Quality Profile
- Documentation: Expert-level and publication-ready (80+ points expected)
- Code Quality: Enterprise-grade implementation (80+ points expected)
- Completeness: Comprehensive test and validation coverage (80+ points expected)
- Usability: Exceptional user experience with extensive help (80+ points expected)
Tier Migration Process
Promotion Process
1. Assessment: Quality scorer evaluates skill against higher tier requirements 2. Review: Engineering team reviews assessment and implementation 3. Testing: Comprehensive testing against higher tier standards 4. Approval: Team consensus on tier promotion 5. Update: Skill metadata and documentation updated to reflect new tier
Demotion Process
1. Issue Identification: Quality degradation or standards violation identified 2. Assessment: Current quality evaluated against tier requirements 3. Notice: Skill maintainer notified of potential demotion 4. Grace Period: 30-day period for remediation 5. Final Review: Re-assessment after grace period 6. Action: Tier adjustment or removal if standards not met
Tier Change Communication
- All tier changes logged in skill CHANGELOG.md
- Repository-level tier change notifications
- Integration with CI/CD systems for automated handling
- Community notifications for significant changes
Compliance Monitoring
Automated Monitoring
- Daily quality assessment scans
- Tier compliance validation in CI/CD
- Automated reporting of tier violations
- Integration with code review processes
Manual Review Process
- Quarterly tier review cycles
- Community feedback integration
- Expert panel reviews for complex cases
- Appeals process for tier disputes
Enforcement Actions
- Warning: First violation or minor issues
- Probation: Repeated violations or moderate issues
- Demotion: Serious violations or quality degradation
- Removal: Critical violations or abandonment
This tier requirements matrix serves as the definitive guide for skill classification and quality standards within the claude-skills ecosystem. Regular updates ensure alignment with evolving best practices and community needs.
#!/usr/bin/env python3
"""
Quality Scorer - Scores skills across multiple quality dimensions
This script provides comprehensive quality assessment for skills in the claude-skills
ecosystem by evaluating documentation, code quality, completeness, and usability.
Generates letter grades, tier recommendations, and improvement roadmaps.
Usage:
python quality_scorer.py <skill_path> [--detailed] [--minimum-score SCORE] [--json]
Author: Claude Skills Engineering Team
Version: 1.0.0
Dependencies: Python Standard Library Only
"""
import argparse
import ast
import json
import os
import re
import sys
from datetime import datetime
from pathlib import Path
from typing import Dict, List, Any, Optional, Tuple
import yaml
class QualityDimension:
"""Represents a quality scoring dimension"""
def __init__(self, name: str, weight: float, description: str):
self.name = name
self.weight = weight
self.description = description
self.score = 0.0
self.max_score = 100.0
self.details = {}
self.suggestions = []
def add_score(self, component: str, score: float, max_score: float, details: str = ""):
"""Add a component score"""
self.details[component] = {
"score": score,
"max_score": max_score,
"percentage": (score / max_score * 100) if max_score > 0 else 0,
"details": details
}
def calculate_final_score(self):
"""Calculate the final weighted score for this dimension"""
if not self.details:
self.score = 0.0
return
total_score = sum(detail["score"] for detail in self.details.values())
total_max = sum(detail["max_score"] for detail in self.details.values())
self.score = (total_score / total_max * 100) if total_max > 0 else 0.0
def add_suggestion(self, suggestion: str):
"""Add an improvement suggestion"""
self.suggestions.append(suggestion)
class QualityReport:
"""Container for quality assessment results"""
def __init__(self, skill_path: str):
self.skill_path = skill_path
self.timestamp = datetime.utcnow().isoformat() + "Z"
self.dimensions = {}
self.overall_score = 0.0
self.letter_grade = "F"
self.tier_recommendation = "BASIC"
self.improvement_roadmap = []
self.summary_stats = {}
def add_dimension(self, dimension: QualityDimension):
"""Add a quality dimension"""
self.dimensions[dimension.name] = dimension
def calculate_overall_score(self):
"""Calculate overall weighted score"""
if not self.dimensions:
return
total_weighted_score = 0.0
total_weight = 0.0
for dimension in self.dimensions.values():
total_weighted_score += dimension.score * dimension.weight
total_weight += dimension.weight
self.overall_score = total_weighted_score / total_weight if total_weight > 0 else 0.0
# Calculate letter grade
if self.overall_score >= 95:
self.letter_grade = "A+"
elif self.overall_score >= 90:
self.letter_grade = "A"
elif self.overall_score >= 85:
self.letter_grade = "A-"
elif self.overall_score >= 80:
self.letter_grade = "B+"
elif self.overall_score >= 75:
self.letter_grade = "B"
elif self.overall_score >= 70:
self.letter_grade = "B-"
elif self.overall_score >= 65:
self.letter_grade = "C+"
elif self.overall_score >= 60:
self.letter_grade = "C"
elif self.overall_score >= 55:
self.letter_grade = "C-"
elif self.overall_score >= 50:
self.letter_grade = "D"
else:
self.letter_grade = "F"
# Recommend tier based on overall score and specific criteria
self._calculate_tier_recommendation()
# Generate improvement roadmap
self._generate_improvement_roadmap()
# Calculate summary statistics
self._calculate_summary_stats()
def _calculate_tier_recommendation(self):
"""Calculate recommended tier based on quality scores"""
doc_score = self.dimensions.get("Documentation", QualityDimension("", 0, "")).score
code_score = self.dimensions.get("Code Quality", QualityDimension("", 0, "")).score
completeness_score = self.dimensions.get("Completeness", QualityDimension("", 0, "")).score
usability_score = self.dimensions.get("Usability", QualityDimension("", 0, "")).score
# POWERFUL tier requirements (all dimensions must be strong)
if (self.overall_score >= 80 and
all(score >= 75 for score in [doc_score, code_score, completeness_score, usability_score])):
self.tier_recommendation = "POWERFUL"
# STANDARD tier requirements (most dimensions good)
elif (self.overall_score >= 70 and
sum(1 for score in [doc_score, code_score, completeness_score, usability_score] if score >= 65) >= 3):
self.tier_recommendation = "STANDARD"
# BASIC tier (minimum viable quality)
else:
self.tier_recommendation = "BASIC"
def _generate_improvement_roadmap(self):
"""Generate prioritized improvement suggestions"""
all_suggestions = []
# Collect suggestions from all dimensions with scores
for dim_name, dimension in self.dimensions.items():
for suggestion in dimension.suggestions:
priority = "HIGH" if dimension.score < 60 else "MEDIUM" if dimension.score < 75 else "LOW"
all_suggestions.append({
"priority": priority,
"dimension": dim_name,
"suggestion": suggestion,
"current_score": dimension.score
})
# Sort by priority and score
priority_order = {"HIGH": 0, "MEDIUM": 1, "LOW": 2}
all_suggestions.sort(key=lambda x: (priority_order[x["priority"]], x["current_score"]))
self.improvement_roadmap = all_suggestions[:10] # Top 10 suggestions
def _calculate_summary_stats(self):
"""Calculate summary statistics"""
scores = [dim.score for dim in self.dimensions.values()]
self.summary_stats = {
"highest_dimension": max(self.dimensions.items(), key=lambda x: x[1].score)[0] if scores else "None",
"lowest_dimension": min(self.dimensions.items(), key=lambda x: x[1].score)[0] if scores else "None",
"score_variance": sum((score - self.overall_score) ** 2 for score in scores) / len(scores) if scores else 0,
"dimensions_above_70": sum(1 for score in scores if score >= 70),
"dimensions_below_50": sum(1 for score in scores if score < 50)
}
class QualityScorer:
"""Main quality scoring engine"""
def __init__(self, skill_path: str, detailed: bool = False, verbose: bool = False):
self.skill_path = Path(skill_path).resolve()
self.detailed = detailed
self.verbose = verbose
self.report = QualityReport(str(self.skill_path))
def log_verbose(self, message: str):
"""Log verbose message if verbose mode enabled"""
if self.verbose:
print(f"[VERBOSE] {message}", file=sys.stderr)
def assess_quality(self) -> QualityReport:
"""Main quality assessment entry point"""
try:
self.log_verbose(f"Starting quality assessment for {self.skill_path}")
# Check if skill path exists
if not self.skill_path.exists():
raise ValueError(f"Skill path does not exist: {self.skill_path}")
# Score each dimension
self._score_documentation()
self._score_code_quality()
self._score_completeness()
self._score_usability()
# Calculate overall metrics
self.report.calculate_overall_score()
self.log_verbose(f"Quality assessment completed. Overall score: {self.report.overall_score:.1f}")
except Exception as e:
print(f"Quality assessment failed: {str(e)}", file=sys.stderr)
raise
return self.report
def _score_documentation(self):
"""Score documentation quality (25% weight)"""
self.log_verbose("Scoring documentation quality...")
dimension = QualityDimension("Documentation", 0.25, "Quality of documentation and written materials")
# Score SKILL.md
self._score_skill_md(dimension)
# Score README.md
self._score_readme(dimension)
# Score reference documentation
self._score_references(dimension)
# Score examples and usage clarity
self._score_examples(dimension)
dimension.calculate_final_score()
self.report.add_dimension(dimension)
def _score_skill_md(self, dimension: QualityDimension):
"""Score SKILL.md quality"""
skill_md_path = self.skill_path / "SKILL.md"
if not skill_md_path.exists():
dimension.add_score("skill_md_existence", 0, 25, "SKILL.md does not exist")
dimension.add_suggestion("Create comprehensive SKILL.md file")
return
try:
content = skill_md_path.read_text(encoding='utf-8')
lines = [line for line in content.split('\n') if line.strip()]
# Score based on length and depth
line_count = len(lines)
if line_count >= 400:
length_score = 25
elif line_count >= 300:
length_score = 20
elif line_count >= 200:
length_score = 15
elif line_count >= 100:
length_score = 10
else:
length_score = 5
dimension.add_score("skill_md_length", length_score, 25,
f"SKILL.md has {line_count} lines")
if line_count < 300:
dimension.add_suggestion("Expand SKILL.md with more detailed sections")
# Score frontmatter quality
frontmatter_score = self._score_frontmatter(content)
dimension.add_score("skill_md_frontmatter", frontmatter_score, 25,
"Frontmatter completeness and accuracy")
# Score section completeness
section_score = self._score_sections(content)
dimension.add_score("skill_md_sections", section_score, 25,
"Required and recommended section coverage")
# Score content depth
depth_score = self._score_content_depth(content)
dimension.add_score("skill_md_depth", depth_score, 25,
"Content depth and technical detail")
except Exception as e:
dimension.add_score("skill_md_readable", 0, 25, f"Error reading SKILL.md: {str(e)}")
dimension.add_suggestion("Fix SKILL.md file encoding or format issues")
def _score_frontmatter(self, content: str) -> float:
"""Score SKILL.md frontmatter quality"""
required_fields = ["Name", "Tier", "Category", "Dependencies", "Author", "Version"]
recommended_fields = ["Last Updated", "Description"]
try:
if not content.startswith('---'):
return 5 # Partial credit for having some structure
end_marker = content.find('---', 3)
if end_marker == -1:
return 5
frontmatter_text = content[3:end_marker].strip()
frontmatter = yaml.safe_load(frontmatter_text)
if not isinstance(frontmatter, dict):
return 5
score = 0
# Required fields (15 points)
present_required = sum(1 for field in required_fields if field in frontmatter)
score += (present_required / len(required_fields)) * 15
# Recommended fields (5 points)
present_recommended = sum(1 for field in recommended_fields if field in frontmatter)
score += (present_recommended / len(recommended_fields)) * 5
# Quality of field values (5 points)
quality_bonus = 0
for field, value in frontmatter.items():
if isinstance(value, str) and len(value.strip()) > 3:
quality_bonus += 0.5
score += min(quality_bonus, 5)
return min(score, 25)
except yaml.YAMLError:
return 5 # Some credit for attempting frontmatter
def _score_sections(self, content: str) -> float:
"""Score section completeness"""
required_sections = ["Description", "Features", "Usage", "Examples"]
recommended_sections = ["Architecture", "Installation", "Troubleshooting", "Contributing"]
score = 0
# Required sections (15 points)
present_required = 0
for section in required_sections:
if re.search(rf'^#+\s*{re.escape(section)}\s*$', content, re.MULTILINE | re.IGNORECASE):
present_required += 1
score += (present_required / len(required_sections)) * 15
# Recommended sections (10 points)
present_recommended = 0
for section in recommended_sections:
if re.search(rf'^#+\s*{re.escape(section)}\s*$', content, re.MULTILINE | re.IGNORECASE):
present_recommended += 1
score += (present_recommended / len(recommended_sections)) * 10
return score
def _score_content_depth(self, content: str) -> float:
"""Score content depth and technical detail"""
score = 0
# Code examples (8 points)
code_blocks = len(re.findall(r'```[\w]*\n.*?\n```', content, re.DOTALL))
score += min(code_blocks * 2, 8)
# Technical depth indicators (8 points)
depth_indicators = ['API', 'algorithm', 'architecture', 'implementation', 'performance',
'scalability', 'security', 'integration', 'configuration', 'parameters']
depth_score = sum(1 for indicator in depth_indicators if indicator.lower() in content.lower())
score += min(depth_score * 0.8, 8)
# Usage examples (9 points)
example_patterns = [r'Example:', r'Usage:', r'```bash', r'```python', r'```yaml']
example_count = sum(len(re.findall(pattern, content, re.IGNORECASE)) for pattern in example_patterns)
score += min(example_count * 1.5, 9)
return score
def _score_readme(self, dimension: QualityDimension):
"""Score README.md quality"""
readme_path = self.skill_path / "README.md"
if not readme_path.exists():
dimension.add_score("readme_existence", 10, 25, "README.md exists (partial credit)")
dimension.add_suggestion("Create README.md with usage instructions")
return
try:
content = readme_path.read_text(encoding='utf-8')
# Length and substance
if len(content.strip()) >= 1000:
length_score = 25
elif len(content.strip()) >= 500:
length_score = 20
elif len(content.strip()) >= 200:
length_score = 15
else:
length_score = 10
dimension.add_score("readme_quality", length_score, 25,
f"README.md content quality ({len(content)} characters)")
if len(content.strip()) < 500:
dimension.add_suggestion("Expand README.md with more detailed usage examples")
except Exception:
dimension.add_score("readme_readable", 5, 25, "README.md exists but has issues")
def _score_references(self, dimension: QualityDimension):
"""Score reference documentation quality"""
references_dir = self.skill_path / "references"
if not references_dir.exists():
dimension.add_score("references_existence", 0, 25, "No references directory")
dimension.add_suggestion("Add references directory with documentation")
return
ref_files = list(references_dir.glob("*.md")) + list(references_dir.glob("*.txt"))
if not ref_files:
dimension.add_score("references_content", 5, 25, "References directory empty")
dimension.add_suggestion("Add reference documentation files")
return
# Score based on number and quality of reference files
score = min(len(ref_files) * 5, 20) # Up to 20 points for multiple files
# Bonus for substantial content
total_content = 0
for ref_file in ref_files:
try:
content = ref_file.read_text(encoding='utf-8')
total_content += len(content.strip())
except:
continue
if total_content >= 2000:
score += 5 # Bonus for substantial reference content
dimension.add_score("references_quality", score, 25,
f"References: {len(ref_files)} files, {total_content} chars")
def _score_examples(self, dimension: QualityDimension):
"""Score examples and usage clarity"""
score = 0
# Look for example files in various locations
example_locations = ["examples", "assets", "scripts"]
example_files = []
for location in example_locations:
location_path = self.skill_path / location
if location_path.exists():
example_files.extend(location_path.glob("*example*"))
example_files.extend(location_path.glob("*sample*"))
example_files.extend(location_path.glob("*demo*"))
# Score based on example availability
if len(example_files) >= 3:
score = 25
elif len(example_files) >= 2:
score = 20
elif len(example_files) >= 1:
score = 15
else:
score = 10
dimension.add_suggestion("Add more usage examples and sample files")
dimension.add_score("examples_availability", score, 25,
f"Found {len(example_files)} example/sample files")
def _score_code_quality(self):
"""Score code quality (25% weight)"""
self.log_verbose("Scoring code quality...")
dimension = QualityDimension("Code Quality", 0.25, "Quality of Python scripts and implementation")
scripts_dir = self.skill_path / "scripts"
if not scripts_dir.exists():
dimension.add_score("scripts_existence", 0, 100, "No scripts directory")
dimension.add_suggestion("Create scripts directory with Python files")
dimension.calculate_final_score()
self.report.add_dimension(dimension)
return
python_files = list(scripts_dir.glob("*.py"))
if not python_files:
dimension.add_score("python_scripts", 0, 100, "No Python scripts found")
dimension.add_suggestion("Add Python scripts to scripts directory")
dimension.calculate_final_score()
self.report.add_dimension(dimension)
return
# Score script complexity and quality
self._score_script_complexity(python_files, dimension)
# Score error handling
self._score_error_handling(python_files, dimension)
# Score code structure
self._score_code_structure(python_files, dimension)
# Score output format support
self._score_output_support(python_files, dimension)
dimension.calculate_final_score()
self.report.add_dimension(dimension)
def _score_script_complexity(self, python_files: List[Path], dimension: QualityDimension):
"""Score script complexity and sophistication"""
total_complexity = 0
script_count = len(python_files)
for script_path in python_files:
try:
content = script_path.read_text(encoding='utf-8')
# Count lines of code (excluding empty lines and comments)
lines = content.split('\n')
loc = len([line for line in lines if line.strip() and not line.strip().startswith('#')])
# Score based on LOC
if loc >= 800:
complexity_score = 25
elif loc >= 500:
complexity_score = 20
elif loc >= 300:
complexity_score = 15
elif loc >= 100:
complexity_score = 10
else:
complexity_score = 5
total_complexity += complexity_score
except Exception:
continue
avg_complexity = total_complexity / script_count if script_count > 0 else 0
dimension.add_score("script_complexity", avg_complexity, 25,
f"Average script complexity across {script_count} scripts")
if avg_complexity < 15:
dimension.add_suggestion("Consider expanding scripts with more functionality")
def _score_error_handling(self, python_files: List[Path], dimension: QualityDimension):
"""Score error handling quality"""
total_error_score = 0
script_count = len(python_files)
for script_path in python_files:
try:
content = script_path.read_text(encoding='utf-8')
error_score = 0
# Check for try/except blocks
try_count = content.count('try:')
error_score += min(try_count * 5, 15) # Up to 15 points for try/except
# Check for specific exception handling
exception_types = ['Exception', 'ValueError', 'FileNotFoundError', 'KeyError', 'TypeError']
for exc_type in exception_types:
if exc_type in content:
error_score += 2 # 2 points per specific exception type
# Check for logging or error reporting
if any(indicator in content for indicator in ['print(', 'logging.', 'sys.stderr']):
error_score += 5 # 5 points for error reporting
total_error_score += min(error_score, 25) # Cap at 25 per script
except Exception:
continue
avg_error_score = total_error_score / script_count if script_count > 0 else 0
dimension.add_score("error_handling", avg_error_score, 25,
f"Error handling quality across {script_count} scripts")
if avg_error_score < 15:
dimension.add_suggestion("Improve error handling with try/except blocks and meaningful error messages")
def _score_code_structure(self, python_files: List[Path], dimension: QualityDimension):
"""Score code structure and organization"""
total_structure_score = 0
script_count = len(python_files)
for script_path in python_files:
try:
content = script_path.read_text(encoding='utf-8')
structure_score = 0
# Check for functions and classes
function_count = content.count('def ')
class_count = content.count('class ')
structure_score += min(function_count * 2, 10) # Up to 10 points for functions
structure_score += min(class_count * 3, 9) # Up to 9 points for classes
# Check for docstrings
docstring_patterns = ['"""', "'''", 'def.*:\n.*"""', 'class.*:\n.*"""']
for pattern in docstring_patterns:
if re.search(pattern, content):
structure_score += 1 # 1 point per docstring indicator
# Check for if __name__ == "__main__"
if 'if __name__ == "__main__"' in content:
structure_score += 3
# Check for imports organization
if content.lstrip().startswith(('import ', 'from ')):
structure_score += 2 # Imports at top
total_structure_score += min(structure_score, 25)
except Exception:
continue
avg_structure_score = total_structure_score / script_count if script_count > 0 else 0
dimension.add_score("code_structure", avg_structure_score, 25,
f"Code structure quality across {script_count} scripts")
if avg_structure_score < 15:
dimension.add_suggestion("Improve code structure with more functions, classes, and documentation")
def _score_output_support(self, python_files: List[Path], dimension: QualityDimension):
"""Score output format support"""
total_output_score = 0
script_count = len(python_files)
for script_path in python_files:
try:
content = script_path.read_text(encoding='utf-8')
output_score = 0
# Check for JSON support
if any(indicator in content for indicator in ['json.dump', 'json.load', '--json']):
output_score += 12 # JSON support
# Check for formatted output
if any(indicator in content for indicator in ['print(f"', 'print("', '.format(', 'f"']):
output_score += 8 # Human-readable output
# Check for argparse help
if '--help' in content or 'add_help=' in content:
output_score += 5 # Help functionality
total_output_score += min(output_score, 25)
except Exception:
continue
avg_output_score = total_output_score / script_count if script_count > 0 else 0
dimension.add_score("output_support", avg_output_score, 25,
f"Output format support across {script_count} scripts")
if avg_output_score < 15:
dimension.add_suggestion("Add support for both JSON and human-readable output formats")
def _score_completeness(self):
"""Score completeness (25% weight)"""
self.log_verbose("Scoring completeness...")
dimension = QualityDimension("Completeness", 0.25, "Completeness of required components and assets")
# Score directory structure
self._score_directory_structure(dimension)
# Score asset availability
self._score_assets(dimension)
# Score expected outputs
self._score_expected_outputs(dimension)
# Score test coverage
self._score_test_coverage(dimension)
dimension.calculate_final_score()
self.report.add_dimension(dimension)
def _score_directory_structure(self, dimension: QualityDimension):
"""Score directory structure completeness"""
required_dirs = ["scripts"]
recommended_dirs = ["assets", "references", "expected_outputs"]
score = 0
# Required directories (15 points)
for dir_name in required_dirs:
if (self.skill_path / dir_name).exists():
score += 15 / len(required_dirs)
# Recommended directories (10 points)
present_recommended = 0
for dir_name in recommended_dirs:
if (self.skill_path / dir_name).exists():
present_recommended += 1
score += (present_recommended / len(recommended_dirs)) * 10
dimension.add_score("directory_structure", score, 25,
f"Directory structure completeness")
missing_recommended = [d for d in recommended_dirs if not (self.skill_path / d).exists()]
if missing_recommended:
dimension.add_suggestion(f"Add recommended directories: {', '.join(missing_recommended)}")
def _score_assets(self, dimension: QualityDimension):
"""Score asset availability and quality"""
assets_dir = self.skill_path / "assets"
if not assets_dir.exists():
dimension.add_score("assets_existence", 5, 25, "Assets directory missing")
dimension.add_suggestion("Create assets directory with sample data")
return
asset_files = [f for f in assets_dir.rglob("*") if f.is_file()]
if not asset_files:
dimension.add_score("assets_content", 10, 25, "Assets directory empty")
dimension.add_suggestion("Add sample data files to assets directory")
return
# Score based on number and diversity of assets
score = min(len(asset_files) * 3, 20) # Up to 20 points for multiple assets
# Bonus for diverse file types
extensions = set(f.suffix.lower() for f in asset_files if f.suffix)
if len(extensions) >= 3:
score += 5 # Bonus for file type diversity
dimension.add_score("assets_quality", score, 25,
f"Assets: {len(asset_files)} files, {len(extensions)} types")
def _score_expected_outputs(self, dimension: QualityDimension):
"""Score expected outputs availability"""
expected_dir = self.skill_path / "expected_outputs"
if not expected_dir.exists():
dimension.add_score("expected_outputs", 10, 25, "Expected outputs directory missing")
dimension.add_suggestion("Add expected_outputs directory with sample results")
return
output_files = [f for f in expected_dir.rglob("*") if f.is_file()]
if len(output_files) >= 3:
score = 25
elif len(output_files) >= 2:
score = 20
elif len(output_files) >= 1:
score = 15
else:
score = 10
dimension.add_suggestion("Add expected output files for testing")
dimension.add_score("expected_outputs", score, 25,
f"Expected outputs: {len(output_files)} files")
def _score_test_coverage(self, dimension: QualityDimension):
"""Score test coverage and validation"""
# This is a simplified scoring - in a more sophisticated system,
# this would integrate with actual test runners
score = 15 # Base score for having a structure
# Check for test-related files
test_indicators = ["test", "spec", "check"]
test_files = []
for indicator in test_indicators:
test_files.extend(self.skill_path.rglob(f"*{indicator}*"))
if test_files:
score += 10 # Bonus for test files
dimension.add_score("test_coverage", score, 25,
f"Test coverage indicators: {len(test_files)} files")
if not test_files:
dimension.add_suggestion("Add test files or validation scripts")
def _score_usability(self):
"""Score usability (25% weight)"""
self.log_verbose("Scoring usability...")
dimension = QualityDimension("Usability", 0.25, "Ease of use and user experience")
# Score installation simplicity
self._score_installation(dimension)
# Score usage clarity
self._score_usage_clarity(dimension)
# Score help and documentation accessibility
self._score_help_accessibility(dimension)
# Score practical examples
self._score_practical_examples(dimension)
dimension.calculate_final_score()
self.report.add_dimension(dimension)
def _score_installation(self, dimension: QualityDimension):
"""Score installation simplicity"""
# Check for installation complexity indicators
score = 25 # Start with full points for standard library only approach
# Check for requirements.txt or setup.py (would reduce score)
if (self.skill_path / "requirements.txt").exists():
score -= 5 # Minor penalty for external dependencies
dimension.add_suggestion("Consider removing external dependencies for easier installation")
if (self.skill_path / "setup.py").exists():
score -= 3 # Minor penalty for complex setup
dimension.add_score("installation_simplicity", max(score, 15), 25,
"Installation complexity assessment")
def _score_usage_clarity(self, dimension: QualityDimension):
"""Score usage clarity"""
score = 0
# Check README for usage instructions
readme_path = self.skill_path / "README.md"
if readme_path.exists():
try:
content = readme_path.read_text(encoding='utf-8').lower()
if 'usage' in content or 'how to' in content:
score += 10
if 'example' in content:
score += 5
except:
pass
# Check scripts for help text quality
scripts_dir = self.skill_path / "scripts"
if scripts_dir.exists():
python_files = list(scripts_dir.glob("*.py"))
help_quality = 0
for script_path in python_files:
try:
content = script_path.read_text(encoding='utf-8')
if 'argparse' in content and 'help=' in content:
help_quality += 2
except:
continue
score += min(help_quality, 10) # Up to 10 points for help text
dimension.add_score("usage_clarity", score, 25, "Usage instructions and help quality")
if score < 15:
dimension.add_suggestion("Improve usage documentation and help text")
def _score_help_accessibility(self, dimension: QualityDimension):
"""Score help and documentation accessibility"""
score = 0
# Check for comprehensive help in scripts
scripts_dir = self.skill_path / "scripts"
if scripts_dir.exists():
python_files = list(scripts_dir.glob("*.py"))
for script_path in python_files:
try:
content = script_path.read_text(encoding='utf-8')
# Check for detailed help text
if 'epilog=' in content or 'description=' in content:
score += 5 # Detailed help
# Check for examples in help
if 'examples:' in content.lower() or 'example:' in content.lower():
score += 3 # Examples in help
except:
continue
# Check for documentation files
doc_files = list(self.skill_path.glob("*.md"))
if len(doc_files) >= 2:
score += 5 # Multiple documentation files
dimension.add_score("help_accessibility", min(score, 25), 25,
"Help and documentation accessibility")
if score < 15:
dimension.add_suggestion("Add more comprehensive help text and documentation")
def _score_practical_examples(self, dimension: QualityDimension):
"""Score practical examples quality"""
score = 0
# Look for example files
example_patterns = ["*example*", "*sample*", "*demo*", "*tutorial*"]
example_files = []
for pattern in example_patterns:
example_files.extend(self.skill_path.rglob(pattern))
# Score based on example availability and quality
if len(example_files) >= 5:
score = 25
elif len(example_files) >= 3:
score = 20
elif len(example_files) >= 2:
score = 15
elif len(example_files) >= 1:
score = 10
else:
score = 5
dimension.add_suggestion("Add more practical examples and sample files")
dimension.add_score("practical_examples", score, 25,
f"Practical examples: {len(example_files)} files")
class QualityReportFormatter:
"""Formats quality reports for output"""
@staticmethod
def format_json(report: QualityReport) -> str:
"""Format report as JSON"""
return json.dumps({
"skill_path": report.skill_path,
"timestamp": report.timestamp,
"overall_score": round(report.overall_score, 1),
"letter_grade": report.letter_grade,
"tier_recommendation": report.tier_recommendation,
"summary_stats": report.summary_stats,
"dimensions": {
name: {
"name": dim.name,
"weight": dim.weight,
"score": round(dim.score, 1),
"description": dim.description,
"details": dim.details,
"suggestions": dim.suggestions
}
for name, dim in report.dimensions.items()
},
"improvement_roadmap": report.improvement_roadmap
}, indent=2)
@staticmethod
def format_human_readable(report: QualityReport, detailed: bool = False) -> str:
"""Format report as human-readable text"""
lines = []
lines.append("=" * 70)
lines.append("SKILL QUALITY ASSESSMENT REPORT")
lines.append("=" * 70)
lines.append(f"Skill: {report.skill_path}")
lines.append(f"Timestamp: {report.timestamp}")
lines.append(f"Overall Score: {report.overall_score:.1f}/100 ({report.letter_grade})")
lines.append(f"Recommended Tier: {report.tier_recommendation}")
lines.append("")
# Dimension scores
lines.append("QUALITY DIMENSIONS:")
for name, dimension in report.dimensions.items():
lines.append(f" {name}: {dimension.score:.1f}/100 ({dimension.weight * 100:.0f}% weight)")
if detailed and dimension.details:
for component, details in dimension.details.items():
lines.append(f" • {component}: {details['score']:.1f}/{details['max_score']} - {details['details']}")
lines.append("")
# Summary statistics
if report.summary_stats:
lines.append("SUMMARY STATISTICS:")
lines.append(f" Highest Dimension: {report.summary_stats['highest_dimension']}")
lines.append(f" Lowest Dimension: {report.summary_stats['lowest_dimension']}")
lines.append(f" Dimensions Above 70%: {report.summary_stats['dimensions_above_70']}")
lines.append(f" Dimensions Below 50%: {report.summary_stats['dimensions_below_50']}")
lines.append("")
# Improvement roadmap
if report.improvement_roadmap:
lines.append("IMPROVEMENT ROADMAP:")
for i, item in enumerate(report.improvement_roadmap[:5], 1):
priority_symbol = "🔴" if item["priority"] == "HIGH" else "🟡" if item["priority"] == "MEDIUM" else "🟢"
lines.append(f" {i}. {priority_symbol} [{item['dimension']}] {item['suggestion']}")
lines.append("")
return "\n".join(lines)
def main():
"""Main entry point"""
parser = argparse.ArgumentParser(
description="Score skill quality across multiple dimensions",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
python quality_scorer.py engineering/my-skill
python quality_scorer.py engineering/my-skill --detailed --json
python quality_scorer.py engineering/my-skill --minimum-score 75
Quality Dimensions (each 25%):
Documentation - SKILL.md quality, README, references, examples
Code Quality - Script complexity, error handling, structure, output
Completeness - Directory structure, assets, expected outputs, tests
Usability - Installation simplicity, usage clarity, help accessibility
Letter Grades: A+ (95+), A (90+), A- (85+), B+ (80+), B (75+), B- (70+), C+ (65+), C (60+), C- (55+), D (50+), F (<50)
"""
)
parser.add_argument("skill_path",
help="Path to the skill directory to assess")
parser.add_argument("--detailed",
action="store_true",
help="Show detailed component scores")
parser.add_argument("--minimum-score",
type=float,
default=0,
help="Minimum acceptable score (exit with error if below)")
parser.add_argument("--json",
action="store_true",
help="Output results in JSON format")
parser.add_argument("--verbose",
action="store_true",
help="Enable verbose logging")
args = parser.parse_args()
try:
# Create scorer and assess quality
scorer = QualityScorer(args.skill_path, args.detailed, args.verbose)
report = scorer.assess_quality()
# Format and output report
if args.json:
print(QualityReportFormatter.format_json(report))
else:
print(QualityReportFormatter.format_human_readable(report, args.detailed))
# Check minimum score requirement
if report.overall_score < args.minimum_score:
print(f"\nERROR: Quality score {report.overall_score:.1f} is below minimum {args.minimum_score}", file=sys.stderr)
sys.exit(1)
# Exit with different codes based on grade
if report.letter_grade in ["A+", "A", "A-"]:
sys.exit(0) # Excellent
elif report.letter_grade in ["B+", "B", "B-"]:
sys.exit(0) # Good
elif report.letter_grade in ["C+", "C", "C-"]:
sys.exit(0) # Acceptable
elif report.letter_grade == "D":
sys.exit(2) # Needs improvement
else: # F
sys.exit(1) # Poor quality
except KeyboardInterrupt:
print("\nQuality assessment interrupted by user", file=sys.stderr)
sys.exit(130)
except Exception as e:
print(f"Quality assessment failed: {str(e)}", file=sys.stderr)
if args.verbose:
import traceback
traceback.print_exc()
sys.exit(1)
if __name__ == "__main__":
main()