
Gen Test Plan
- 52 installs
- 74 repo stars
- Updated July 21, 2026
- existential-birds/beagle
Helps with testing & qa tasks.
About
gen-test-plan is a Claude Code skill for testing & qa. It helps solo builders move faster with AI-assisted development.
- gen-test-plan
- Testing & QA
- AI-coding skill
Gen Test Plan by the numbers
- 52 all-time installs (skills.sh)
- Ranked #1,215 of 2,153 Testing & QA skills by installs in the Skillselion catalog
- Data as of Jul 28, 2026 (Skillselion catalog sync)
npx skills add https://github.com/existential-birds/beagle --skill gen-test-planAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 52 |
|---|---|
| repo stars | ★ 74 |
| Last updated | July 21, 2026 |
| Repository | existential-birds/beagle ↗ |
What it does
Helps with testing & qa tasks.
Files
Generate Test Plan
Analyze the repository's tech stack, branch changes vs default, and generate an executable YAML test plan focused on user-facing impact.
This is an E2E test plan — not an automated test wrapper. The generated plan will be executed by an autonomous agent acting exactly as a human QA tester would: launching real binaries, hitting real endpoints, interacting with real databases, and verifying real observable behavior.
Critical Rule: No Automated Test Duplication
NEVER generate test steps that re-run the project's existing automated test suite. This means:
- No
cargo test,pytest,npm test,go test,mix test, or equivalent commands as test steps - No wrapping unit/integration test modules in a test case
- No "run the tests and check they pass" — that's CI's job, not QA's
If you find yourself writing a test step that invokes the project's test runner, stop and rethink. Ask: "What would a human tester do to verify this feature works?" The answer is never "run the unit tests."
What E2E test steps look like:
- Build the binary and run it with real arguments, check stdout/stderr/exit code
- Start a server and hit it with curl
- Run a CLI command that writes to a real database, then query the database to verify
- Launch the TUI and verify it renders (via screenshot or process lifecycle)
- Chain multiple commands that exercise a full user workflow end-to-end
Hard gates
Complete these in order. Do not advance to the next gate until its Pass condition is met (each pass should leave retrievable evidence: pasted command output, a written list, or the generated file on disk). Scheduling: Gate 1 before Step 2; Gate 2 before Step 5; Gate 3 before Step 7; Gates 4–5 during Step 8 (after the Step 7 summary).
1. Diff and base pinned (after Step 1) — Resolve the base branch from --base when provided, otherwise use the repo default (main or master per Step 1). Compare HEAD to $(git merge-base HEAD origin/<base_branch>) (or equivalent if the remote ref differs). Pass: You record current_branch, base_branch, the merge-base SHA or range used, and changed_files from git diff --name-only <merge-base>..HEAD (empty list allowed if you paste or quote that output and state “no file changes vs base”).
2. Trace complete (after Step 4) — Pass: Every affected entry point you will test has a Core functionality vs Configuration/admin classification, and the Step 4 requirement holds: at least one test targets a core entry point or you document why that is impossible and flag manual review.
3. Plan file valid (after Step 6, before Step 7) — Pass: docs/testing/test-plan.yaml exists and the following command exits 0 (parses the YAML and asserts all four top-level keys are present — a single grep -E with alternations would pass on any one match, so do not substitute it):
python3 -c "import sys, yaml; d = yaml.safe_load(open('docs/testing/test-plan.yaml')) or {}; missing = [k for k in ('version', 'metadata', 'setup', 'tests') if k not in d]; sys.exit('Missing keys: ' + ', '.join(missing) if missing else 0)"4. No automated-test duplication (Step 8) — Pass: Every run: step and every services: command: is scanned for project test runners (cargo test, pytest, npm test, go test, mix test, jest, vitest, mocha, etc.); zero invocations. If any appear, remove or replace them with real E2E actions and re-run Gate 3.
5. Behavioral coverage (Step 8) — Pass: Re-read metadata.changes_summary and recent commit messages; at least one test’s context/steps exercises the primary user-visible behavior they describe. If they describe a capability (e.g., a new provider) but no step invokes it, add that test or fail verification.
Arguments
--base <branch>: Base branch to diff against (default:main)- Path: Target directory (default: current working directory)
Step 1: Gather Repository Context
# Get current branch
git rev-parse --abbrev-ref HEAD
# Resolve base branch: use --base if supplied, otherwise default (main → master)
BASE_BRANCH="${BASE_BRANCH:-$(git rev-parse --verify origin/main >/dev/null 2>&1 && echo main || echo master)}"
MERGE_BASE="$(git merge-base HEAD "origin/${BASE_BRANCH}")"
# Get changed files vs base
git diff --name-only "${MERGE_BASE}"..HEAD
# Get commit messages for context
git log --oneline "${MERGE_BASE}"..HEADCapture:
current_branch: Branch namebase_branch: Default branch to compare againstchanged_files: List of modified filescommit_messages: What the PR is about
Step 2: Detect Tech Stack
See references/stack-discovery.md for stack detection commands, entrypoint discovery, port discovery, and trace rules.
Step 3: Discover User-Facing Entry Points
A "user-facing entry point" is anything a human interacts with: CLI subcommands, HTTP endpoints, UI routes, TUI screens, gRPC services, database migrations, or configuration files that affect runtime behavior.
CLI Applications (Rust/clap, Python/argparse/click, Go/cobra)
# Rust (clap) — look for Subcommand derives and command enums
grep -rn "Subcommand\|#\[command\]" --include="*.rs" | head -20
# Python (click/typer/argparse)
grep -rn "@click.command\|@app.command\|add_parser\|add_subparser" --include="*.py" | head -20
# Go (cobra)
grep -rn "cobra.Command\|AddCommand" --include="*.go" | head -20Build a map of:
- CLI subcommands: command name + description + file:line
- Required arguments and flags per subcommand
- Environment variables the binary reads (grep for
env,std::env::var,os.Getenv,os.environ)
HTTP/API Services
Python (FastAPI/Flask):
grep -rn "@app\.\(get\|post\|put\|delete\|patch\)" --include="*.py" | head -20
grep -rn "@router\.\(get\|post\|put\|delete\|patch\)" --include="*.py" | head -20Node.js (Express/Fastify):
grep -rn "app\.\(get\|post\|put\|delete\)" --include="*.ts" --include="*.js" | head -20
grep -rn "router\.\(get\|post\|put\|delete\)" --include="*.ts" --include="*.js" | head -20Rust (axum/actix/rocket):
grep -rn "Router::new\|\.route(\|#\[get\]\|#\[post\]\|HttpServer" --include="*.rs" | head -20Go (net/http, gin, chi):
grep -rn "http.HandleFunc\|r.GET\|r.POST\|router.Get\|router.Post" --include="*.go" | head -20Elixir (Phoenix):
grep -rn "get \"/\|post \"/\|pipe_through\|live \"/\|scope \"/\"" --include="*.ex" | head -20Browser UI Routes
grep -rn "createBrowserRouter\|<Route\|path=" --include="*.tsx" --include="*.jsx" | head -20Database and Migrations
# SQL migrations
ls migrations/ db/migrate/ priv/repo/migrations/ 2>/dev/null
# Schema files
ls schema.sql schema.prisma 2>/dev/nullBuild a consolidated map of:
- CLI subcommands: name + args + file:line
- API endpoints: method + path + file:line
- UI routes: path + component + file:line
- Database migrations: filename + what they create/alter
- Configuration: env vars and config files that affect behavior
Step 4: Trace Changes to Entry Points
For each changed file, determine if it affects user-facing functionality:
1. Direct entry point change — File contains route definitions 2. Import chain analysis — Find what imports the changed file and trace up to entry points 3. Architecture-aware tracing — Read the project's CLAUDE.md, README, or architecture docs to understand data flow and module relationships, rather than relying solely on grep 4. Document the trace path in test context
Import Chain Analysis by Ecosystem
# Rust — use/mod/crate references and workspace deps
grep -rn "use.*<crate>\|mod <module>" --include="*.rs"
grep -rn "<crate-name>" --include="Cargo.toml"
# Python — from/import
grep -rn "from.*<module>\|import.*<module>" --include="*.py"
# TypeScript/JavaScript — import/require
grep -rn "from.*<module>\|require.*<module>" --include="*.ts" --include="*.tsx" --include="*.js" --include="*.jsx"
# Elixir — alias/import/use
grep -rn "alias.*<Module>\|import.*<Module>\|use.*<Module>" --include="*.ex" --include="*.exs"
# Go — package references
grep -rn "<package>\." --include="*.go"If the ecosystem is not covered above, or grep results are inconclusive, read the project's CLAUDE.md, README, or architecture docs to understand the module graph and trace the data flow from changed files to user-facing entry points.
Classify Affected Entry Points
After identifying all affected entry points, classify each one:
| Category | Description | Examples | Priority |
|---|---|---|---|
| Core functionality | Entry points where the feature does its actual work for the end user | Chat endpoint, API action, data processing pipeline, generation flow | High — test first |
| Configuration/admin | Entry points where the feature is set up, toggled, or configured | Settings page, admin dashboard, preference toggles, dropdown selections | Lower — test after core |
Classification rules:
- Ask: "If a user wanted to use this feature (not configure it), which entry point would they interact with?" — that's core functionality
- A settings page that adds a new dropdown option is configuration; the endpoint that actually uses that option is core functionality
- The same changed file (e.g., a new provider module) may affect both a settings page and a functional endpoint — both must be traced
Requirement: At least one test must target a core functionality entry point before generating configuration/admin tests. If no core functionality entry point can be identified, explicitly document why and flag this for manual review.
Output: For each affected entry point, document:
- Which changed files affect it
- The import/dependency chain
- Classification: Core functionality or Configuration/admin
- Why this entry point needs testing
Step 5: Generate Test Cases
See references/test-case-generation.md for the detailed API/browser templates, prioritization rules, and test-case guidelines.
Step 6: Write YAML Test Plan
Create the test plan file:
mkdir -p docs/testingWrite to docs/testing/test-plan.yaml:
version: 1
metadata:
branch: <current_branch>
base: <base_branch>
generated: <ISO timestamp>
changes_summary: |
<Summary of what this PR changes based on commit messages and diff>
setup:
stack:
- type: <rust|node|python|go|elixir|docker>
package_manager: <cargo|pnpm|npm|yarn|uv|poetry|mix|none>
prerequisites:
# Services or infrastructure the tests need running
- name: <e.g., PostgreSQL>
check: <command to verify it's available, e.g., "pg_isready -h localhost">
build:
# Commands to build the project artifacts (binaries, assets, etc.)
- <build command, e.g., "cargo build --workspace">
services:
# Long-running processes to start before tests (servers, watchers, etc.)
# Omit if the project is a CLI tool or library with no server component
- command: <start command>
health_check:
url: http://localhost:<port>/health
timeout: 30
env:
# Environment variables needed by tests (use ${VAR} for secrets)
DATABASE_URL: "${DATABASE_URL}"
tests:
# CLI test example — run the built binary with real arguments:
- id: TC-01
name: <CLI test name>
context: |
<Why this test exists, which changes affect it>
steps:
- run: <command that a human would type in their terminal>
- run: <follow-up command to verify the effect>
expected: |
<Expected behavior: exit code, stdout content, side effects>
# API test example:
- id: TC-02
name: <API test name>
context: |
<Why this test exists, which changes affect it>
steps:
- action: curl
method: GET
url: http://localhost:<port>/<path>
expected: |
<Expected behavior in natural language>
# Database verification example:
- id: TC-03
name: <Database test name>
context: |
<Why this test exists, which changes affect it>
steps:
- run: <command that writes to the database>
- run: psql "${DATABASE_URL}" -c "SELECT ... FROM ... WHERE ..."
expected: |
<Expected rows, schema state, or migration effect>
# Browser test example (always use agent-browser CLI commands):
- id: TC-04
name: <UI test name>
context: |
<Why this test exists, which changes affect it>
steps:
- run: agent-browser open http://localhost:<port>/<path>
- run: agent-browser snapshot -i
- run: agent-browser click @<ref>
- run: agent-browser snapshot -i
- run: agent-browser screenshot evidence/tc-04.png
expected: |
<Expected behavior in natural language>
evidence:
screenshot: evidence/tc-04.pngStep 7: Report Summary
After generating the test plan:
## Test Plan Generated
**File:** `docs/testing/test-plan.yaml`
**Branch:** <current_branch> → <base_branch>
### Detected Stack
| Component | Type | Port |
|-----------|------|------|
| <component> | <type> | <port> |
### Tests Generated
| ID | Name | Type | Affected By |
|----|------|------|-------------|
| TC-01 | <name> | curl/browser | <files> |
### Entry Point Coverage
- **Covered:** <N> entry points with tests
- **Unchanged:** <M> entry points not affected by this PR
### Next Steps
1. Review the generated test plan at `docs/testing/test-plan.yaml`
2. Adjust test values and expectations as needed
3. Run the tests by invoking the **run-test-plan** skill ([run-test-plan](../run-test-plan/SKILL.md))Step 8: Verification
Confirm Hard gates 1–5 are satisfied with evidence (see Hard gates above) before treating the plan as complete. Then run:
# Verify file was created
ls -la docs/testing/test-plan.yaml
# Validate YAML syntax
python3 -c "import yaml; yaml.safe_load(open('docs/testing/test-plan.yaml'))" && echo "Valid YAML"
# Check required fields
grep -E "^version:|^metadata:|^setup:|^tests:" docs/testing/test-plan.yamlVerification Checklist:
- [ ] Test plan file created at
docs/testing/test-plan.yaml - [ ] YAML is syntactically valid
- [ ] At least one test case generated
- [ ] Setup commands match detected stack
- [ ] Each test has id, name, steps, and expected fields
- [ ] No automated test duplication: Grep every
run:andcommand:step in the plan for test runner invocations (cargo test,pytest,npm test,go test,mix test,jest,vitest,mocha, etc.). If ANY step invokes the project's test runner, the plan fails verification. Remove those steps and replace them with real E2E actions. - [ ] Behavioral coverage: At least one test exercises the primary behavioral change described in
changes_summary. Re-read thechanges_summaryand commit messages — if they describe a capability (e.g., "adds a new LLM provider") but no test invokes that capability (e.g., sends a message through the provider), the plan fails verification. Add the missing core functionality test before completing. - [ ] No config-only plans: If all tests target configuration/admin entry points and zero tests target core functionality entry points, the plan is incomplete. Go back to Step 4, identify the core functionality entry points, and add tests for them.
Rules
- E2E only — every test step must exercise the real built artifact (binary, server, UI) as a human would. Never wrap automated test suites.
- Always create
docs/testing/directory if it doesn't exist - Generate at least one test per affected entry point
- Include context explaining why each test matters (trace from changes)
- Use natural language for
expectedfield (agent will interpret) - CLI projects: Test steps should invoke the actual binary with real arguments and verify stdout, stderr, exit codes, and side effects (files created, database rows written, processes spawned)
- Server projects: Start the server in setup, test via curl/agent-browser
- Library-only projects with no binary or server: If the change is purely internal library code with no user-facing entry point (no CLI, no server, no UI), state this explicitly and generate tests that exercise the library through its public API via a small driver script — not by running the test suite
- Default to conservative port detection (8000 for API, 5173/3000 for frontend)
- Browser automation steps MUST use `agent-browser` CLI commands (e.g.,
agent-browser open,agent-browser snapshot -i,agent-browser click @ref) — never use abstract action syntax - Always
agent-browser snapshot -ibefore interacting with elements and after navigation/DOM changes - Use
agent-browser screenshot <path>to capture evidence for browser tests - Use
${ENV_VAR}syntax for secrets, never hardcode credentials - If no user-facing changes detected, explain why and suggest manual verification
Tech Stack and Entry Point Discovery
This reference keeps the detailed stack-detection and entry-point tracing logic that would otherwise make SKILL.md too long.
Step 2: Detect Tech Stack
Scan for project configuration files to determine the stack:
# Rust detection
ls Cargo.toml Cargo.lock 2>/dev/null
# Elixir detection
ls mix.exs mix.lock 2>/dev/null
# Node.js detection
ls package.json pnpm-lock.yaml package-lock.json yarn.lock 2>/dev/null
# Python detection
ls pyproject.toml requirements.txt setup.py 2>/dev/null
ls uv.lock poetry.lock 2>/dev/null
# Go detection
ls go.mod 2>/dev/null
# Docker detection
ls docker-compose.yml docker-compose.yaml Dockerfile 2>/dev/null
# Makefile detection
ls Makefile 2>/dev/null && grep -q "dev:" Makefile && echo "has-dev-target"
# Database detection
ls migrations/ db/migrate/ priv/repo/migrations/ 2>/dev/null
grep -rl "DATABASE_URL\|postgres\|PgPool\|sqlx\|Ecto.Repo" --include="*.rs" --include="*.ex" --include="*.py" --include="*.ts" --include="*.go" 2>/dev/null | head -5Stack Detection Rules
| Files Found | Stack | Build Commands | Default Port |
|---|---|---|---|
Cargo.toml | Rust (cargo) | cargo build --release | N/A (CLI) or 8080 |
mix.exs | Elixir (mix) | mix deps.get && mix compile | 4000 |
package.json + pnpm-lock.yaml | Node.js (pnpm) | pnpm install && pnpm run build | 5173, 3000 |
package.json + package-lock.json | Node.js (npm) | npm install && npm run build | 5173, 3000 |
package.json + yarn.lock | Node.js (yarn) | yarn install && yarn build | 5173, 3000 |
pyproject.toml + uv.lock | Python (uv) | uv sync | 8000 |
pyproject.toml + poetry.lock | Python (poetry) | poetry install | 8000 |
go.mod | Go | go build ./... | 8080 |
docker-compose.yml | Docker | docker-compose up -d | Parse from compose |
Makefile with dev: target | Make-based | make dev | Infer from Makefile |
Determine Project Type
After detecting the stack, classify the project:
| Type | How to detect | E2E test approach |
|---|---|---|
| CLI tool | Has fn main / if __name__ / binary targets, no HTTP listener | Build binary, invoke subcommands with real args, check stdout/stderr/exit code/side effects |
| HTTP server | Has route definitions, listens on a port | Start server, hit endpoints with curl, verify responses and database state |
| Web app (frontend) | Has React/Vue/Svelte routes, serves HTML | Start dev server, use agent-browser for UI interactions |
| Full-stack | Has both server and frontend | Start both, test API + UI |
| Library only | No binary, no server, no main — only lib.rs/__init__.py/package exports | Write a small driver script that exercises the public API, or test through a downstream consumer |
Entrypoint Discovery
Rust (clap CLI):
# Find CLI subcommands
grep -rn "Subcommand\|#\[command\]" --include="*.rs" | head -20
# Find binary targets
grep -rn "\[\[bin\]\]\|fn main" --include="*.rs" --include="*.toml" | head -20
# Find HTTP routes (axum/actix/rocket)
grep -rn "Router::new\|\.route(\|#\[get\]\|#\[post\]\|HttpServer" --include="*.rs" | head -20Elixir (Phoenix):
grep -rn "get \"/\|post \"/\|pipe_through\|live \"/\|scope \"/\"" --include="*.ex" | head -20
grep -rn "def handle_event\|def mount" --include="*.ex" | head -20Python:
grep -rn "@app\.\(get\|post\|put\|delete\|patch\)" --include="*.py" | head -20
grep -rn "@router\.\(get\|post\|put\|delete\|patch\)" --include="*.py" | head -20
grep -rn "@click.command\|@app.command\|add_parser" --include="*.py" | head -20Node.js (Express/Fastify):
grep -rn "app\.\(get\|post\|put\|delete\)" --include="*.ts" --include="*.js" | head -20
grep -rn "router\.\(get\|post\|put\|delete\)" --include="*.ts" --include="*.js" | head -20React Router:
grep -rn "createBrowserRouter\|<Route\|path=" --include="*.tsx" --include="*.jsx" | head -20Go (net/http, gin, chi):
grep -rn "http.HandleFunc\|r.GET\|r.POST\|router.Get\|router.Post" --include="*.go" | head -20
grep -rn "cobra.Command\|AddCommand" --include="*.go" | head -20Build a map of:
- CLI subcommands: name + args + description + file:line
- API endpoints: method + path + file:line
- UI routes: path + component + file:line
- Database migrations: filename + tables affected
Port Discovery
grep -E "^PORT=" .env .env.example .env.local 2>/dev/null
grep -A2 "ports:" docker-compose.yml 2>/dev/null
grep -E "port:" vite.config.ts vite.config.js 2>/dev/nullStep 4: Trace Changes to Entry Points
For each changed file, determine if it affects user-facing functionality:
1. Direct entry point change - file contains route definitions 2. Import chain analysis - find what imports the changed file and trace up to entry points 3. Architecture-aware tracing - read CLAUDE.md, README, or architecture docs for module relationships 4. Document the trace path in test context
Import Chain Analysis by Ecosystem
# Rust — use/mod/crate references
grep -rn "use.*<crate>\|mod <module>" --include="*.rs"
# Also check Cargo.toml dependencies between workspace crates
grep -rn "<crate-name>" --include="Cargo.toml"
# Python
grep -rn "from.*<module>\|import.*<module>" --include="*.py"
# TypeScript/JavaScript
grep -rn "from.*<module>\|require.*<module>" --include="*.ts" --include="*.tsx" --include="*.js" --include="*.jsx"
# Elixir
grep -rn "alias.*<Module>\|import.*<Module>\|use.*<Module>" --include="*.ex" --include="*.exs"
# Go
grep -rn "<package>\." --include="*.go"If the ecosystem is not covered above, or grep results are inconclusive, read the project's CLAUDE.md, README, or architecture docs to understand the module graph and trace the data flow from changed files to user-facing entry points.
Classify Affected Entry Points
| Category | Description | Examples | Priority |
|---|---|---|---|
| Core functionality | Entry points where the feature does its actual work for the end user | Chat endpoint, API action, data processing pipeline, generation flow | High - test first |
| Configuration/admin | Entry points where the feature is set up, toggled, or configured | Settings page, admin dashboard, preference toggles, dropdown selections | Lower - test after core |
Requirement: At least one test must target a core functionality entry point before generating configuration/admin tests.
Test Case Generation
This reference keeps the long test-template examples and prioritization guidance out of SKILL.md.
Critical: E2E Only
Every test case must exercise the real built artifact — the actual binary, server, or UI — exactly as a human QA tester would. A test case that invokes the project's automated test runner (cargo test, pytest, npm test, go test, mix test, etc.) is never valid. Those tests already run in CI. The purpose of this plan is to verify behavior that automated tests cannot: real end-to-end workflows through the actual user interface (CLI, HTTP, browser).
Step 5: Generate Test Cases
Before generating test cases, answer: "What does this change do for the end user?"
Then ask: "How would a human tester — who has never seen the code — verify this works?"
Generate tests in this order:
1. Core functionality tests first - exercise the primary behavioral change through the actual user-facing interface. 2. Configuration/admin tests second - support the feature but do not replace the core test.
CLI Applications (shell tests)
- id: TC-XX
name: <Describe what user action this represents>
context: |
<Which files changed and why this subcommand is affected>
steps:
# Build the binary first (or reference it from setup.build)
- run: <invoke the CLI binary with real arguments>
- run: <verify the effect — check stdout, query database, inspect files>
expected: |
<Exit code, stdout content, side effects (files created, DB rows, etc.)>CLI test examples by scenario:
# Subcommand that writes to a database
- id: TC-01
name: "volant plan creates a workflow in PostgreSQL"
steps:
- run: ./target/debug/volant plan "Fix login bug" --description "Users can't log in" --sandbox local
- run: psql "${DATABASE_URL}" -c "SELECT id, state FROM workflows ORDER BY created_at DESC LIMIT 1"
expected: |
Exit code 0. A new workflow row exists with state 'pending' or further.
# Subcommand that outputs to stdout
- id: TC-02
name: "volant status lists workflows"
steps:
- run: ./target/debug/volant status --all
expected: |
Exit code 0. Outputs a table or list of workflows. Does not crash on empty database.
# Subcommand with --dry-run
- id: TC-03
name: "volant run --dry-run validates without executing"
steps:
- run: ./target/debug/volant run example.toml --dry-run
expected: |
Exit code 0. Prints the resolved workflow structure. Does not execute any nodes.
# Error handling — missing config
- id: TC-04
name: "volant plan errors gracefully without API key"
steps:
- run: env -u ANTHROPIC_API_KEY ./target/debug/volant plan "test" 2>&1 || true
expected: |
Non-zero exit code. Stderr contains a meaningful error about missing configuration,
not a panic or stack trace.API Endpoints (curl tests)
- id: TC-XX
name: <Describe what user action this represents>
context: |
<Which files changed and why this endpoint is affected>
steps:
- action: curl
method: <GET|POST|PUT|DELETE>
url: http://localhost:<port>/<path>
headers:
Content-Type: application/json
body: <JSON body if needed>
expected: |
<HTTP status code, response body shape, side effects>Database Verification
- id: TC-XX
name: <Describe what data change this verifies>
context: |
<Which migration or data-writing code changed>
steps:
- run: <command that triggers the data write>
- run: psql "${DATABASE_URL}" -c "<SQL query to verify>"
expected: |
<Expected rows, column values, or schema state>Database test examples:
# Migration applies cleanly
- id: TC-05
name: "Session tables migration creates expected schema"
steps:
- run: psql "${DATABASE_URL}" -c "\dt sessions"
- run: psql "${DATABASE_URL}" -c "\d sessions"
expected: |
The 'sessions' table exists with columns matching the migration definition.
# Data roundtrip through the application
- id: TC-06
name: "Checkpoint save and resume preserves workflow state"
steps:
- run: ./target/debug/volant plan "test checkpoint" --sandbox local
- run: psql "${DATABASE_URL}" -c "SELECT workflow_id, state FROM checkpoints ORDER BY created_at DESC LIMIT 1"
expected: |
A checkpoint row exists for the workflow with the expected state payload.UI Routes (agent-browser CLI tests)
- id: TC-XX
name: <Describe the user journey>
context: |
<Which files changed and why this route is affected>
steps:
- run: agent-browser open http://localhost:<port>/<path>
- run: agent-browser snapshot -i
note: Capture interactive elements with refs
- run: agent-browser fill @<ref> "<test value>"
- run: agent-browser click @<ref>
- run: agent-browser wait --url "**/<expected-path>"
- run: agent-browser snapshot -i
note: Verify final state
- run: agent-browser screenshot docs/testing/evidence/tc-XX.png
expected: |
<Natural language description of expected behavior>
evidence:
screenshot: docs/testing/evidence/tc-XX.pngProcess Lifecycle (TUI / long-running)
- id: TC-XX
name: <Describe the process behavior>
steps:
# Start the process in background, give it time to initialize, then verify
- run: timeout 5 ./target/debug/volant 2>&1 || true
- run: <verify it started correctly — check stderr output, temp files, etc.>
expected: |
Process starts without crash. Produces expected initial output.
Exits cleanly on timeout/interrupt.Test Case Guidelines
- Never invoke the project's test runner — every step must be a real user action
- At least one test per affected user-facing entry point
- CLI tests for command-line tools — invoke the binary directly
- curl tests for HTTP servers
- Browser tests for web UIs — always use real
agent-browserCLI commands - Database tests when migrations or data-writing code changed
- Include authentication/config steps if commands require credentials
- Test error paths — missing config, bad input, unreachable services
- Always snapshot before interacting and re-snapshot after navigation or DOM changes
Step 6: Write YAML Test Plan
Create docs/testing/test-plan.yaml with metadata, setup, prerequisites, and the generated tests.
Step 7: Report Summary
Report the generated file, detected stack, tests generated, entry-point coverage, and next steps.
Step 8: Verification
Verify the YAML file exists, parses successfully, and includes the required top-level keys.
Additional verification: Grep every run: step for test runner commands (cargo test, pytest, npm test, go test, mix test, jest, vitest). If any are found, the plan fails — replace with real E2E actions.