
Qa Test Plan
- 79 installs
- 1 repo stars
- Updated June 17, 2026
- validkeys/sherpy
Helps with testing & qa tasks.
About
qa-test-plan is a Claude Code skill for testing & qa. It helps solo builders move faster with AI-assisted coding.
- qa-test-plan
- Testing & QA
- AI-coding skill
Qa Test Plan by the numbers
- 79 all-time installs (skills.sh)
- +2 installs in the week ending Aug 5, 2026 (Skillselion tracking)
- Ranked #1,067 of 2,153 Testing & QA skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/validkeys/sherpy --skill qa-test-planAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 79 |
|---|---|
| repo stars | ★ 1 |
| Last updated | June 17, 2026 |
| Repository | validkeys/sherpy ↗ |
What it does
Helps with testing & qa tasks.
Files
QA Test Plan
Generates a structured QA test plan from completed requirements artifacts. Produces test suites that map directly to business requirements and user personas so QA teams know exactly what to test during each delivery timeline QA round.
Prerequisites
{base_directory}/requirements/business-requirements.yaml(output from/business-requirements-interview){base_directory}/requirements/technical-requirements.yaml(output from/technical-requirements-interview)
Usage
/qa-test-plan [base-directory]If no directory is provided, auto-detect by looking for requirements/business-requirements.yaml in the current directory.
If not found, prompt the user: "Where are your requirements documents located?"
Wait for the user to provide a path before proceeding. Store as base_directory.
Process
Step 1: Determine Base Directory and Load Requirements
If no directory parameter was provided, check if requirements/business-requirements.yaml exists in the current directory.
- If found, use current directory as
base_directory - If not found, prompt: "Where are your requirements documents located?" and wait for user response
Once base_directory is determined, read both requirements files from {base_directory}/requirements/. Extract:
From `business-requirements.yaml`:
- Functional requirements and user stories
- User personas and their primary use cases
- Success criteria and metrics
- Known constraints and edge conditions
From `technical-requirements.yaml`:
- API surface (endpoints, inputs, outputs)
- Authentication and authorization model
- Non-functional requirements (performance targets, uptime SLAs)
- Data model constraints (required fields, uniqueness, formats)
- Security requirements
Step 2: Identify Test Suites
Group test coverage into suites. Each suite maps to a coherent functional area (not to individual milestones). Derive suites directly from the requirements — do not invent features not present in the source documents.
Standard suite categories to consider (include only those applicable):
| Category | Driven by |
|---|---|
| Authentication | Auth model in technical requirements |
| Core User Flows | User personas + functional requirements |
| Data Validation | Data model constraints |
| API Contract | API design in technical requirements |
| Permissions & Roles | Authorization model |
| Error Handling | Edge cases in requirements + API error states |
| Performance | Non-functional performance targets |
| Security | Security requirements |
| Integration | External dependencies in technical requirements |
Step 3: Generate Test Cases per Suite
For each suite, write test cases covering:
- Positive — happy path, expected successful outcomes
- Negative — invalid inputs, unauthorized access, missing required fields
- Edge — boundary values, empty states, concurrent operations, large payloads
- Security — injection, unauthorized access escalation, token/session misuse
- Performance — response time under expected load (only when targets are specified)
Each test case must include:
id— unique, scoped to suite (e.g.tc-auth-001)name— plain-language description of what is being testedtype—positive | negative | edge | security | performancepriority—high | medium | low- high: covers a success criterion or a named persona's primary use case
- medium: covers a secondary flow or validation rule
- low: covers an edge/corner case unlikely to affect typical users
preconditions— system state required before executing the teststeps— numbered, concrete actionsexpected_result— specific, verifiable outcomerequirement_refs— IDs of the business/technical requirements this case validatestags— optional labels for filtering (e.g.[smoke, regression, auth])
Step 4: Compute Coverage
Calculate coverage metrics:
- Functional coverage — percentage of named functional requirements with at least one
high-priority test case - Persona coverage — percentage of user personas whose primary use case has a
positivetest case - Non-functional coverage — boolean per NFR category (performance, security, etc.) — at least one test case exists
Flag any functional requirement with no test case as a coverage gap.
Step 5: Generate qa-test-plan.yaml
Write {base_directory}/delivery/qa-test-plan.yaml.
Create directory if it doesn't exist:
mkdir -p {base_directory}/deliveryStep 6: Gap Analysis
After generating the file, report inline:
## QA Test Plan Gap Analysis
**Project:** [name]
**Test Suites:** [n]
**Total Test Cases:** [n] ([n] high / [n] medium / [n] low)
**Functional Coverage:** [n]% ([n]/[n] requirements have high-priority cases)
**Persona Coverage:** [n]% ([n]/[n] personas covered)
**NFR Coverage:**
[✓ / ✗] Performance tests present
[✓ / ✗] Security tests present
[✓ / ✗] Integration tests present
**Gaps:**
- [requirement or persona with no coverage, if any]
**Recommendations:** [none / list]Output Format
qa-test-plan.yaml Schema
The output document includes: version, project, generated, sources (reference file paths), summary (test counts by priority, coverage metrics), and test_suites (each with id (ts-slug), name, description, requirement_refs, and test_cases). Each test case has: id (tc-suite-nnn), name, type (positive/negative/edge/security/performance), priority (high/medium/low), preconditions, steps, expected_result, requirement_refs, and optional tags.
See [references/output-spec.md](references/output-spec.md) for the complete document specification with all fields, coverage calculation rules, and test case structure.
See [references/example.yaml](references/example.yaml) for a full example.
Example Output
See [references/example.yaml](references/example.yaml) for a complete sample QA test plan.
version: "1.0.0"
project: task-automation-cli
generated: "2026-03-26T10:00:00Z"
sources:
business_requirements: business-requirements.yaml
technical_requirements: technical-requirements.yaml
summary:
total_test_suites: 4
total_test_cases: 12
by_priority:
high: 5
medium: 5
low: 2
coverage:
functional_requirements: "88%" # 7/8 requirements have high-priority cases
personas: "100%" # all 3 personas covered
has_performance_tests: true
has_security_tests: true
test_suites:
# ── Suite 1: Workflow Loading ──────────────────────────────────────────────
- id: ts-workflow-loading
name: Workflow Loading & Validation
description: Verifies that workflow definition files are correctly parsed, validated, and rejected when malformed.
requirement_refs: [BR-001, BR-002]
test_cases:
- id: tc-wf-001
name: Valid workflow YAML loads without errors
type: positive
priority: high
preconditions:
- A syntactically valid workflow.yaml exists in the project directory
steps:
- Run `sherpy run workflow.yaml`
- Observe CLI output
expected_result: CLI confirms workflow loaded with [n] tasks listed; exit code 0
requirement_refs: [BR-001]
tags: [smoke, workflow, happy-path]
- id: tc-wf-002
name: Workflow with missing required field is rejected
type: negative
priority: high
preconditions:
- A workflow.yaml missing the `name` field exists
steps:
- Run `sherpy run workflow.yaml`
expected_result: CLI prints a descriptive validation error naming the missing field; exit code 1
requirement_refs: [BR-002]
tags: [validation, error-handling]
- id: tc-wf-003
name: Workflow referencing an unknown task type is rejected
type: negative
priority: medium
preconditions:
- "A workflow.yaml contains a task with `type: unknownType`"
steps:
- Run `sherpy run workflow.yaml`
expected_result: "CLI reports unrecognised task type with line number; exit code 1"
requirement_refs: [BR-002]
tags: [validation, error-handling]
# ── Suite 2: Task Execution ────────────────────────────────────────────────
- id: ts-task-execution
name: Task Execution Engine
description: Verifies that individual tasks execute correctly, in dependency order, and handle failures gracefully.
requirement_refs: [BR-003, BR-004]
test_cases:
- id: tc-exec-001
name: Sequential tasks execute in defined order
type: positive
priority: high
preconditions:
- A workflow with three sequential tasks (A → B → C) is defined
steps:
- Run `sherpy run workflow.yaml`
- Observe execution log
expected_result: Tasks complete in order A, B, C; each start time is after the prior task's end time
requirement_refs: [BR-003]
tags: [smoke, execution, ordering]
- id: tc-exec-002
name: Parallel tasks execute concurrently
type: positive
priority: high
preconditions:
- A workflow with two tasks sharing no dependency is defined
steps:
- Run `sherpy run workflow.yaml`
- Observe execution log timestamps
expected_result: Both tasks have overlapping execution windows; total runtime < sum of individual runtimes
requirement_refs: [BR-003]
tags: [execution, parallelism]
- id: tc-exec-003
name: "Workflow halts when a task fails and `on_failure: stop` is set"
type: negative
priority: high
preconditions:
- "A workflow where task B is configured to fail and `on_failure: stop` is set"
steps:
- Run `sherpy run workflow.yaml`
expected_result: Execution stops after task B fails; downstream tasks C and D do not run; exit code non-zero
requirement_refs: [BR-004]
tags: [error-handling, failure-modes]
- id: tc-exec-004
name: "Execution continues when a task fails and `on_failure: continue` is set"
type: edge
priority: medium
preconditions:
- "A workflow where task B fails and `on_failure: continue` is set"
steps:
- Run `sherpy run workflow.yaml`
expected_result: Tasks after B that do not depend on B still execute; summary shows B as failed
requirement_refs: [BR-004]
tags: [error-handling, edge-case]
# ── Suite 3: CLI Interface ─────────────────────────────────────────────────
- id: ts-cli
name: CLI Interface & User Experience
description: Verifies the CLI surface including help text, output formatting, and exit codes.
requirement_refs: [BR-005]
test_cases:
- id: tc-cli-001
name: --help flag prints usage information
type: positive
priority: medium
preconditions:
- CLI is installed and on PATH
steps:
- Run `sherpy --help`
expected_result: Usage text printed including all top-level commands; exit code 0
requirement_refs: [BR-005]
tags: [cli, ux, smoke]
- id: tc-cli-002
name: Running with no arguments prints help (not an error)
type: edge
priority: low
preconditions:
- CLI is installed
steps:
- Run `sherpy` with no arguments
expected_result: Help/usage text printed; exit code 0 (not an error state)
requirement_refs: [BR-005]
tags: [cli, ux, edge-case]
# ── Suite 4: Security & Performance ────────────────────────────────────────
- id: ts-security-performance
name: Security & Performance
description: Verifies non-functional requirements around execution safety and CLI responsiveness.
requirement_refs: [TR-SEC-001, TR-PERF-001]
test_cases:
- id: tc-sec-001
name: Shell task cannot access environment variables outside declared scope
type: security
priority: high
preconditions:
- A workflow with a shell task that attempts to read $SECRET_KEY
- SECRET_KEY is not in the workflow's declared env scope
steps:
- Run `sherpy run workflow.yaml`
- Inspect task output and logs
expected_result: $SECRET_KEY resolves to empty or an error is raised; the value is never printed in logs
requirement_refs: [TR-SEC-001]
tags: [security, env-isolation]
- id: tc-perf-001
name: CLI startup completes in under 500ms on cold start
type: performance
priority: medium
preconditions:
- CLI binary installed; system not under load
steps:
- Run `time sherpy --version` three times
- Record wall-clock time for each run
expected_result: All three runs complete in < 500ms
requirement_refs: [TR-PERF-001]
tags: [performance, startup]
- id: tc-perf-002
name: Workflow with 50 sequential tasks completes within acceptable time
type: performance
priority: low
preconditions:
- A generated workflow with 50 no-op shell tasks
steps:
- Run `sherpy run large-workflow.yaml`
- Record total execution time
expected_result: Execution completes in < 10 seconds; memory usage stays below 256MB
requirement_refs: [TR-PERF-001]
tags: [performance, scale]
QA Test Plan YAML Specification
Document Type: qa-test-plan.yaml Version: 1.0.0 Generated By: qa-test-plan skill Purpose: Structured test plan keyed to functional requirements and user personas, covering positive, negative, edge, security, and performance cases. Generated from business and technical requirements for use during delivery timeline QA phases.
---
Document Structure
Root Level
version: string # Semantic version (required, format: "X.Y.Z")
project: string # Project name (required)
generated: string # ISO 8601 timestamp (required)
sources:
business_requirements: string # Filename of business requirements (required)
technical_requirements: string # Filename of technical requirements (required)Validation Rules:
version: Must be semantic version format (e.g., "1.0.0")project: 3-100 characters, matches source requirementsgenerated: ISO 8601 format (e.g., "2026-04-16T10:30:00Z")sources.business_requirements: Filename (typically "business-requirements.yaml")sources.technical_requirements: Filename (typically "technical-requirements.yaml")
Summary Section
summary:
total_test_suites: integer # Number of test suites (required, ≥1)
total_test_cases: integer # Number of test cases across all suites (required, ≥1)
by_priority:
high: integer # Count of high-priority cases (required)
medium: integer # Count of medium-priority cases (required)
low: integer # Count of low-priority cases (required)
coverage:
functional_requirements: string # Percentage (required, format: "N%")
personas: string # Percentage (required, format: "N%")
has_performance_tests: boolean # Performance tests present (required)
has_security_tests: boolean # Security tests present (required)Validation Rules:
total_test_suites: Positive integer, matches test_suites array lengthtotal_test_cases: Positive integer, sum of all test cases across suitesby_priority: All values non-negative; sum equals total_test_casescoverage.functional_requirements: Format "N%" where N is 0-100coverage.personas: Format "N%" where N is 0-100has_performance_tests: true if any test case hastype: performancehas_security_tests: true if any test case hastype: security
Test Suites Array
Each test suite groups related test cases by functional area.
test_suites:
- id: string # Suite identifier (required, format: ts-[slug])
name: string # Human-readable suite name (required)
description: string # One-line scope description (required)
requirement_refs: array<string> # Requirement IDs this suite validates (required)
test_cases: array<TestCase> # Test cases for this suite (required, ≥1)Validation Rules:
id: Must match patternts-[a-z0-9-]+(e.g., "ts-auth", "ts-workflow-loading")name: 5-100 charactersdescription: 10-200 characters; one-line functional scoperequirement_refs: Non-empty array of requirement IDs (e.g., ["BR-001", "TR-SEC-001"])test_cases: Non-empty array; at least one test case per suite
Test Case Structure
Each test case within a suite validates specific behavior.
test_cases:
- id: string # Test case identifier (required, format: tc-[suite-slug]-[nnn])
name: string # Plain-language test description (required)
type: string # Test type (required, enum)
priority: string # Priority level (required, enum)
preconditions: array<string> # Required system state (required)
steps: array<string> # Numbered concrete actions (required, ≥1)
expected_result: string # Specific verifiable outcome (required)
requirement_refs: array<string> # Requirement IDs validated (required, ≥1)
tags: array<string> # Optional filter labels (optional)Validation Rules:
id: Must match patterntc-[a-z0-9-]+-\d{3}(e.g., "tc-auth-001", "tc-wf-042")name: 10-150 characters; describes what is being testedtype: Must be one of:positive,negative,edge,security,performancepriority: Must be one of:high,medium,lowpreconditions: Non-empty array; each item 10-200 characterssteps: Non-empty array; numbered concrete actionsexpected_result: 15-300 characters; specific and verifiablerequirement_refs: Non-empty array; at least one requirement referencetags: Optional array; lowercase slugs (e.g., ["smoke", "regression", "auth"])
Test Type Definitions
| Type | Purpose | When to Use |
|---|---|---|
positive | Verify expected successful outcomes | Happy path, primary user flows |
negative | Verify system rejects invalid inputs | Missing fields, unauthorized access, bad data |
edge | Verify boundary conditions | Empty states, large payloads, concurrent ops |
security | Verify security controls | Injection, escalation, token misuse |
performance | Verify non-functional targets | Response time, throughput, resource limits |
Priority Definitions
| Priority | Definition | Coverage Requirement |
|---|---|---|
high | Covers success criterion or primary persona use case | All functional requirements should have ≥1 high-priority case |
medium | Covers secondary flow or validation rule | Important but not blocking |
low | Covers edge/corner case unlikely to affect typical users | Optional coverage |
---
Coverage Metrics
Functional Requirements Coverage
Calculation: (requirements_with_high_priority_tests / total_functional_requirements) * 100
Target: ≥80% functional requirements should have at least one high-priority test case.
Persona Coverage
Calculation: (personas_with_positive_test / total_personas) * 100
Target: 100% of user personas should have at least one positive test case covering their primary use case.
Non-Functional Requirements Coverage
Boolean Checks:
has_performance_tests: true if performance targets exist in technical requirementshas_security_tests: true if security requirements exist in technical or business requirements
---
Validation Rules Summary
Document-Level Validation
1. Version must be semantic version format 2. Generated timestamp must be valid ISO 8601 3. Both source filenames must reference existing requirement documents 4. Summary counts must match actual test case counts 5. Coverage percentages must be in range 0-100%
Suite-Level Validation
1. Suite IDs must be unique within document 2. All suites must have at least one test case 3. All requirement_refs must correspond to actual requirements 4. Suite count in summary must match test_suites array length
Test Case-Level Validation
1. Test case IDs must be unique within document 2. Test case IDs must be scoped to parent suite 3. Type must be one of five defined values 4. Priority must be one of three defined values 5. Each test case must reference at least one requirement 6. Preconditions and steps must be non-empty 7. Expected result must be specific and verifiable
Coverage Validation
1. Functional coverage calculation must be accurate 2. Persona coverage calculation must be accurate 3. NFR boolean flags must reflect actual test case presence
---
Integration with Other Documents
Inputs (Required)
- business-requirements.yaml: Provides functional requirements, user personas, success criteria
- technical-requirements.yaml: Provides API surface, security requirements, performance targets, data model constraints
Used By
- timeline.yaml: QA deadline phases reference this test plan
- milestones.yaml: Definition of done may reference test plan completion
- Developer documentation: QA teams execute test cases during delivery cycles
Relationship to Other Documents
- Test suites map to functional areas, not milestones (one-to-many)
- Test cases validate requirements, not implementation tasks (requirement-driven)
- Coverage metrics ensure all business requirements have verification path
---
Example Structure
version: "1.0.0"
project: task-automation-cli
generated: "2026-04-16T10:00:00Z"
sources:
business_requirements: business-requirements.yaml
technical_requirements: technical-requirements.yaml
summary:
total_test_suites: 4
total_test_cases: 12
by_priority:
high: 5
medium: 5
low: 2
coverage:
functional_requirements: "88%"
personas: "100%"
has_performance_tests: true
has_security_tests: true
test_suites:
- id: ts-authentication
name: Authentication & Authorization
description: Verifies user login, session management, and access control.
requirement_refs: [BR-001, TR-SEC-001]
test_cases:
- id: tc-auth-001
name: Valid credentials grant access
type: positive
priority: high
preconditions:
- User account exists with valid credentials
steps:
- Navigate to login page
- Enter valid username and password
- Click "Sign In"
expected_result: User is authenticated and redirected to dashboard; session token is created
requirement_refs: [BR-001]
tags: [smoke, auth, happy-path]See example.yaml for a complete working example.
---
Generation Process
The qa-test-plan skill follows this process:
1. Load Requirements: Read business-requirements.yaml and technical-requirements.yaml 2. Identify Test Suites: Group coverage by functional area (not milestone) 3. Generate Test Cases: Create positive, negative, edge, security, and performance cases 4. Calculate Coverage: Compute functional, persona, and NFR coverage metrics 5. Validate Output: Ensure all requirements have test coverage 6. Write File: Output to {base_directory}/delivery/qa-test-plan.yaml 7. Report Gaps: Display inline gap analysis showing uncovered requirements
---
Best Practices
Test Case Writing
- Be specific: "User sees error message 'Invalid email format'" not "Error is shown"
- Use concrete actions: "Click the Submit button" not "Submit the form"
- Make results verifiable: Include expected status codes, messages, and state changes
- Reference requirements: Every test case must trace back to at least one requirement
Coverage Strategy
- High-priority coverage: Every functional requirement and success criterion needs ≥1 high-priority test
- Persona coverage: Every user persona's primary use case needs a positive test
- NFR coverage: If technical requirements specify performance or security targets, include corresponding tests
- Edge case balance: Don't over-test edge cases; focus on realistic boundary conditions
Suite Organization
- Functional grouping: Group by feature area, not by milestone or implementation order
- Cohesive suites: Each suite should have a clear functional scope
- Requirement mapping: Suite requirement_refs should cover all test cases within the suite
- Maintainability: Keep suites focused; split large suites (>15 test cases) into smaller ones
Test Maintenance
- Update test plan when requirements change
- Mark deprecated tests rather than deleting them
- Add regression tests for fixed bugs
- Keep test case IDs stable; don't renumber when adding/removing tests